# [GT4Py] field_operator calls with multiple output domains

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Hannes
- Appetite (FTEs, weeks):
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem

Currently GT4Py requires that a field_operator (called from the program or from outside of the DSL) has a single output domain for all fields.

Example

@program
def prog(...):
    fop(..., out=(out_a, out_b), domain=a_domain)

This implies that all output fields are defined on that domain. Specifically, it's not possible to output fields on different horizontal locations (e.g. `Vertex` and `Edge`) or with a different number of vertical levels (e.g. vertically staggered and non-staggered fields).

For this reason, the user has to split field_operators which reduces readability or limits optimizations.

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->

## Solution

In this project we introduce the possiblity to specify an output domain per field.

@program
def prog(...):
    fop(..., out=(out_a, out_b), domain=(domain_a, domain_b))

A prototype is implemented in https://github.com/GridTools/gt4py/pull/2209.

The following is a sketch of the required steps

### Frontend

Extend parsing and type deduction (the prototype just removes checks) for the syntax shown above and the corresponding other variants:

- implicit domain `fop(..., out=(out_a[1:], out_b[2:]))`: here we can remove the limitation that all fields need to be on the same domain. Consequence is that with runtime domain sizes we need to assume that every field has a different size. Consider emitting a warning in case we cannot deduce if fields have same size.
- direct field_operator call (outside of a program)

Note that both implicit domain and direct field_operator call will be simpler after https://github.com/GridTools/gt4py/pull/1893. Because the implicit size args are replaced by the builtins to get the domain. (Parts of the changes are sketched in https://github.com/havogt/gt4py/tree/multiple_output_domains_prototype_direct_fop_call .)

Needs frontend tests including checks for invalid syntax.

### Lowering to GTIR and GTIR transformations

GTIR already allows tuples of domains in `SetAt`s, however support is not fully implemented.

Lowering and transformation need to be extended to deal with tuple of domains (already included in the prototype, but needs proper testing and cleanup).

### DaCe

Support for SetAt with tuple of domains needs to be added. This work is optional. If we don't have the knowledge to do the change, we'll leave it open for after Edoardo's vacation.

Need to remove the requirement that every SetAt produces a single output state?

### Testing

In addition to the already mentioned unit tests at frontend, lowering and transformation level we should add an extensive set of integration_tests that cover the most common cases and corner cases. The following is an incomplete list:
- example with staggering in the vertical aka _KDim and KHalfDim_ (but don't use this ugly names ;) )
- example where a field is both output as well as a temporary with extend on another output field (see discussion in https://hackmd.io/m__8sBBATiqFWOPNMEPsfg)
- unstructured test switching locations
- implicit domain calls with and without compile time domains
- direct field_operator calls

## Rabbit holes

## No-gos

Transformations to improve performance are not included in this project.

## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

- [x] Task 1 ([PR#xxxx](https://github.com/GridTools/gt4py/pulls))
- [x] Subtask A
- [x] Subtask X
- [ ] Task 2
- [x] Subtask H
- [ ] Subtask J
- [ ] Discovered Task 3
- [ ] Subtask L
- [ ] Subtask S
- [ ] Task 4