[GT4Py] Batch of cleanups and refactorings

# [GT4Py] Batch of cleanups and refactorings  - Shaped by: Hannes - Appetite (FTEs, weeks): - Developers:  ## Problem Reduce technical debt and improve usability. Specific goals are to enable - embedded execution for icon4py, - preparation for implementing embedded execution of a combined view\*. \*\) Combined view refers to the language that combines local view and field view. ## Appetite Full cycle, ideally 2 developers that can discuss the conceptual difficulties. ## Solution ### Tasks to enable embedded for icon4py #### Finish `as_offset` Implementation is in https://github.com/GridTools/gt4py/pull/1397. It is missing the implementation of `inverse_image` for multiple dimensions. #### Boundary conditions `concat_where` In the previous cycle we prototyped `concat_where` with the old implicit domain semantics of `where`. This solution does not require changes to lowering as it is compatible with `where` after lowering. The implementation is in https://github.com/GridTools/gt4py/pull/1468. #### Bug: np.scalars with Fields The following code snippet is broken ```python! np.int32(1) < field ``` because the numpy comparison will wrap the field into a numpy array and return a single `bool`. The following solutions are possible: - introduce our own full implementation of scalars (a lot of boiler-plate) - investigate the use of `ufunc`s to avoid this problem - explore https://wrapt.readthedocs.io/en/master/wrappers.html#object-proxy for this purpose **Workaround: the current workaround is to put the field on the l.h.s. of comparison functions.** ### Conceptual cleanups #### Dimensions and Offsets The current specification of dimensions and offsets have conceptual problems or are inconvenient: 1. A remap with a connectivity offset, e.g. `f(V2E)`, creates a field with a local dimension, e.g. `V2EDim`. The offset is mapped to the connectivity via the `offset_provider` (later probably via some mesh property or by directly using the connecitivity). However, currently, the dimensions (`V2EDim`) don't have a link to the connecitivty. For a dimension created by a *sparse connectivity* (one that contains *skip values*), we need the information about the sparsity in neighbor reductions. Therefore, currently, we have the implicit requirement that the offset and dimension use the same tag, e.g. `V2Eoff = FieldOffset("V2E", ...)` and `V2EDim = Dimension("V2E")`. 2. In Cartesian, creating an additional object for offsetting is inconvenient. Cartesian dimensions could be factories for offset, e.g. `I = Dimension("I")` could be used as `I + 1` or `I[1]` to create the Cartesian offset. Providing the mapping in offset_providers is superfluous. 3. It would be convenient to find a mechanism to allow grouping of 1-to-1 connectivities to 1-to-N connectivities (TODO: find a name to distinguish 1-to-1 and 1-to-N). 1-to-1 means mapping a single index to another single index, e.g. what we currently represent with `E2V[0]` (doesn't create a new dimension in the field); 1-to-N refers to `E2V` which creates the local dimension `E2VDim`. It would be consistent if `(E2V[0], E2V[1])` as offset would allow to create a local dimension. #### Valid type annotations for field operators (`Field[Dims[D0, D1, ...], DType]`): Nikki Currently, we represent `Field` type annotations by `Field[[D0, D1], DType]`. The sequence `[D0, D1]` is not legal for python typing annotations. [PEP 646](https://peps.python.org/pep-0646/) introduced valid syntax for _variadic generics_ which matches this use-case, but it requires a slight change in gt4py syntax `Field[Dims[D0, D1], DType]`. Until recently `mypy` didn't implement the feature so there was no reason to switch to the new syntax, but now that is indeed supported gt4py typing annotations should be changed to be syntactically legal and checkable by type checkers. Subtasks: - Define a variadic generic type for annotations (either a new empty `Dims` type or consider to make the current `Domain` a variadic generic) and make `Field` first argument compatible with it. - Extend PAST & FOAST parser to support the new syntax (and still support the old one for now, maybe with a deprecation warning) -- open PR after this is done - Solve possible new `mypy` issues appearing after changing type annotations in the codebase (mostly examples and tests) #### Shaping: One field implementation The current way to implement a new kind of Field is to model the `Field` protocol. This includes everything from data handling to domain computation. However, for all fields the new domain should only depend on the domains of the input fields, not on the specific field implementation. Therefore, we concluded that the customization should be implemented at a lower level: there should be only one implementation of `Field` which contains the domain computation for all built-ins; the customization to different *NDArrayLike* (we use this name here to refer to the lower level implementation, although for a function field, it is not an NDArray) kinds would only operate on the data, e.g. different buffer implementations or function representation. Problems to discuss/shape: - infinite fields (e.g. constant field or function field) vs finite fields (buffer fields) - which are the required functions in an NDArrayLike #### Shaping: How to allocate for embedded execution For embedded execution with numpy and cupy we use the gtx.constructors. They allocate the buffers with a custom layout, which can make sense for performance of embedded execution (but probably is not really required, as embedded is not meant for performance mode and the gained speed-up, if at all, would be minor). Additionally, currently, after the first numpy/cupy operation, the resulting buffer is not anymore allocated with the custom layout. For jax, the allocation doesn't make sense at all, because we first create an empty buffer and then assign into it. For non-mutable jax buffers, we cannot do this, therefore we can only swap the buffer with a new buffer containing the desired data. Problem to discuss/shape: - Does it make sense for embedded to go through gtx.constructors? If not, how can we change the test infrastructure to do different buffer creation for embedded. The solution should be transferable to user code to make it convenient to switch between embedded execution and compiled backends. - If we conclude that using custom layout allocation makes sense, we should propagate it through builtins, by storing the allocation mechanism. #### Optional: Research: canonicalization or partial execution instead of parsing ##### Example: program -> fieldop call `domain` kwarg with same syntax as allowed in `domain()` For fields, we introduced the concept of `DomainLike`, which describes different way to construct a domain. The idea is that users can use compact syntax to construct a domain, while in the backend (here refering to everything that is not user-facing) we can use a single conventient representation for dealing with domains. The backend construction would be `Domain(NamedRange(I, UnitRange(0,4)), NamedRange(J, UnitRange(1,2)))` (note: this syntax assumes we introduced a proper representation of NamedRange, see [minor refactorings](#Minor-refactorings)). This verbose construction is not convenient for users, therefore we allow, e.g. `{I:4, J:(1,2)}`. The cannonicalization of `DomainLike` to `Domain` is implemented in `common.domain`. All possible ways to represent a `DomainLike` should be allowed where users specify domains. Currently, in a field_operator call from the program, we have a place where we describe a domain in a context that we parse, e.g. ```python! @program def foo(...): bar(..., domain={I:(0,4), J:(0,2)}) ``` There, we only allow the `dict[Dimension, tuple[int,int]]` syntax. **It is not desirable to duplicate all variants in our parser, therefore we propose to implement a mechanism to `exec` parts of the program specification instead of parsing.** The goal of this project would be to investigate how such a mechanism could be integrated in parsing with the domain example outlined above. #### Optional: Slice syntax for local dimensions Enables accessing the local dimension in icon4py and remove the Felix-Sparse-Field-representation. TODO expand ### Minor refactorings - dataclass instead of tuple for NamedRange (and refactor to use attribute access): Nikki - canonicalize `AnyIndexSpec` to `NamedSlice` instead of `NamedRange` and add `[I(1):]` semantics as a follow up to https://github.com/GridTools/gt4py/pull/1453 ## Rabbit holes  ## No-gos  ## Progress  - [x] Task 1 ([PR#xxxx](https://github.com/GridTools/gt4py/pulls)) - [x] Subtask A - [x] Subtask X - [ ] Task 2 - [x] Subtask H - [ ] Subtask J - [ ] Discovered Task 3 - [ ] Subtask L - [ ] Subtask S - [ ] Task 4