# Add support for cartesian GT4Py storages in experimental/functional GT4Py

###### tags: `functional cycle 13`

- Developers: Linus (after `functional` is merged to `main`)
- Appetite: 2 weeks

## Problem

In the stable release of GT4Py framework (AKA **cartesian**, v1.x, branch: [main](https://github.com/GridTools/gt4py/tree/main)) there is a submodule with utilities for device-indepent buffer allocation using functionality from NumPy/CuPy. However, this code is not used in the **functional** branch, which effectively limits the usage of this branch with GPU storages.

## Goals

The goal for this project is to use the `gt4py.storage` module with the **functional** branch.

**Optional goal**: if times allows it, it could be beneficial to additionally refactor the `gt4py.storage` implementation using the array creation functions in the Python Array Standard API instead of pure NumPy/CuPy calls. This refactoring would decouple the `gt4py.storage` implementation from NumPy/CuPy, since any library implementing the standard could be used. Since the NumPy and Array standard creation routines are very much aligned, the expected source code changes to make this goal should be really minimal (if any) and mostly dealing with module imports and names.

## Implementation

Currently, there is no memory allocation functionality in the **functional** branch, only `Field` creation from some previously allocated and filled `np.ndarray`, which is implemented in the `np_as_located_field()` function of the embedded Iterator IR backend `functional.iterator.embedded`. This routine wraps the given `ndarray` as the data content of a field, represented as an instance of any class implementing the `LocatedField` prototype, which also requires the information about the field _dimensions_ (`.axes`) linked to this buffer. The `.axes` attribute is accessed later at any point where knowledge of the field dimensions is required (e.g. shifting fields in embedded execution, passing data pointers to compiled backends, etc.).

Array creation routines in the `gt4py.storage` module implements a similar design as discussed in [GDP 3 — A New Storage Implementation using Duck Typing](https://github.com/GridTools/gt4py/blob/main/docs/GDPs/gdp-0003-duck-storage.rst), which builds on existing standards (`__array_interface__`, `__cuda_array_interface__`) to share buffer data, and additionally defines extra attributes to communicate the field dimensions and _origin_ point in the domain.

Implementation steps:

1. Adapt the `gt4py.storage` implementation to support arbitrary dimension identifiers, not only strings.

2. Modify the `functional.embedded.LocatedField` interface to adopt gt4py.storage interface using `__gt_dims__`.

3. Refactor all `LocatedField.axes` attribute references to use `__gt_dims__`.

4. Update all documentation and ADRs if needed to reflect the new interface.

## Non-goals

Extensive refactoring of the embedded Iterator implementation or other parts where storages are used in the **functional** branch are completely out of scope.

## Possible rabbit holes

Since the current implementation of the embedded Iterator IR execution is a bit cumbersome, there might be unexpected side effects of modifying private implementation details of Field-like classes, which could have been used inadvertently as public interfaces somewhere else.

Additionally, the same might happen in the `gt4py.storage` side: since the metadata contents of storage allocation are not really used or extensively tested, minor modifications of the storage utils in the current cartesian branch might unexpectedly break existing cartesian projects.