# Functional model for _unstructured_ grids

###### tags: `cycle 3`

shaped by: Enrique, Hannes

## Goals

In the previous cycle a functional execution model for the Cartesian GT4Py was developed (https://hackmd.io/@gridtools/SkEKFlGEO) and tested in a toy implementation (https://github.com/fthaler/gt4py_new_model). The model uses a local view (_accessor-view_) to express stencil computations. This model is, with extensions, also applicable to the unstructured GT4Py.
In the meantime also an alternative perspective of this model was developed. See the [current draft](https://hackmd.io/@havogt/ryaxJKX5_). Similarly ot the original model, a toy implementation exists in different flavors, see e.g. [here](https://github.com/fthaler/gt4py_new_model/tree/unstructured_v5). The goal of this project is to finalize the model description (time-boxed half-cycle) and implement a initial working version of the model.

In the development of these models we _found out_ that the models allow for a DSL which is directly executable in Python. Additionally, the embedded execution could be designed in a way to achieve decent prototyping performance (similar to the current NumPy backend).

This project targets the more general _unstructured_ GT4Py for several reasons. Cartesian is much more mature and several projects already rely on the current GTScript and planned extensions for the backends (FV3core, and [DaCe + GTScript](https://hackmd.io/QGThhWORQhaz2Mv8Sm-hfA) project). In the _unstructured_ GT4Py, we are still in a prototyping phase, therefore trying new approaches has less impact on other on-going activities. Additionally, _Cartesian_ is included as a special case and the concepts in a pure-Cartesian and the unstructured model have huge overlap.

## Appetite

Full cycle with contributions from CSCS developers involved in discussions and previous iterations of the model (Hannes, Anton, Felix, Enrique, Till). The discussions will be coordinated with developers from partner institutions (e.g. Vulcan, MeteoSwiss) providing actual use-cases in current weather models and getting first-hand knowledge of the new model rationale and features from the very beginning.

The project requires part-time contributions from many developers. Therefore, we propose to assign low-priority [side-projects](https://hackmd.io/nVXB169TRP-M95w8-tze8Q) for contributors to this project.

## Structure

### Finalizing the model specification (timeboxed 3 weeks)

In this phase we will collect a list of known use-cases (including challenging corner-cases) to test the model and refine the design document. During the discussions we will finalize prototyping a basic toy implementation of the specification to guide the specification.

The following questions should be answered:

- Frontend
+ Can we define an executable user-friendly syntax for our mostly-functional DSL?
+ Can we achieve performance numbers similar to NumPy in _eager mode_ execution?
- Are the axis/domain sizes required/useful/useless at the IR level?
- Vertical/sequential computational patterns:
+ Can we express all the known computations? does it make sense?
+ Should we provide specialized built-ins for specific patterns? which ones?
- Tensor-fields and linear-algebra operations:
+ Support per-point operations?
+ Element access with horizontal offsets??

### Model implementation

Once the specification is ready, the toy prototype will be expanded and cleaned up as needed to become a _reference implementation_ of the model, which means it should support all the patterns and features described in the model. The main goal of the prototype is to verify the feasibility and expressiveness of the specification, and to serve as reference implementation against future optimized implementations. There is not need for a minimum performance goal, as long as it is fast enough to execute arbitrarily complex stencils for (very) small domains in an interactive way.

A rough sketch of the implementation plan would be:

1. **(required)** Implement a pure Python executable front-end which does not require parsing the AST or any other source transformation mechanism.
2. *(optional)* Implement an alternative front-end for tracing Python/DSL code in order to generate an IR for the supported computations.
3. *(optional)* Implement a basic toolchain to generate code from the IR with a naive back-end that generates a valid Python/DSL code, which can thus be run with the executable frontend developed in _1_.

When the 3 implementation tasks are completed we will have a toolchain for the new model capable of round-tripping valid DSL code. It will be then possible to start adding alternative backends and transformations (like a translation to DaCe SDFGs).