# [Blueline] Combined stencils for Dycore Granule

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Magdalena, Nikki
- Appetite (FTEs, weeks):
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem
<!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this -->
To unlock further optimization potential we want to use combined stencils in the SolveHonHydro granule (including VelocityAdvection).
The combining of stencils is limited by two factors:
- they cannot extend over halo exchange calls
- the can only combine FieldOperators that return Fields of the same dimensions.

Combined stencils should be verified inside `icon4py` with `datatests` by running on serialized input data and comparing against serialized output data, similar to the way we test the static fields computations in `icon4py`.

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->

The following steps need to be done
### serialized data handling

- add Savepoints (Init/Exit) for all combined stencils as we go along writing or verifying them.

### combined stencils:
- add checkbox at the end of this file under **Progress** with combined stencil name
For each combined stencil:
- write combined stencils
- verify combined stencils individually with `datatests`
- integrate in the granule
- verify entire granule with combined stencil (run existing data test)
- mark checkbox

#### cleanups in fused stencils
- inline trivial stencils, for example replace

  z_w_con_c = where(
        k < nlev,
        _copy_cell_kdim_field_to_vp(w),
        _init_cell_kdim_field_with_zero_vp(),
    )

by

      z_w_con_c = where(
        k < nlev,
        astype(w, vpfloat),
        broadcast(vpfloat("0.0"), (dims.CellDim, dims.KDim)),
    )

- remove unused arguments from `field_operator` interfaces.

#### combined stencils in velocity advection
The combined stencils were written for Fortran integration, some of them have a wrapper `field_operator` that takes an `istep` argument and decides based on this whether to call the predictor or the corrector version. In the Icon4Py we don't need that that wrapper.

The `_restricted` field operator runs on the `nlevp1` level they are called with different bounds and only return those fields in the tuple that have `nlevp1`.

- `fused_velocity_advection_stencil_1_to_7.py`
create a `program` a separate program for the predictor and one for the corrector:

@program
fused_velocity_advection_stencil_1_to_7_predictor(...)
   _fused_velocity_advection_stencil_1_to_7_predictor(...) # outputs 4 fields
   _fused_velocity_advection_stencil_1_to_7_predictor_restricted(...) # only outputs 1 field, different vertical bounds


@program
fused_velocity_advection_stencil_1_to_7_corrector(...)
   _fused_velocity_advection_stencil_1_to_7_corrector(...) # outputs 4 fields
   _fused_velocity_advection_stencil_1_to_7_corrector_restricted(...) # only outputs 1 field, different vertical bounds

the `istep` argument can then be deleted everywhere.

- fused_velocity_advection_stencil_8_to_13 or 8_to_14
- - Only the `8_13` has been verified with Liskov. It is best to take that and add stencil_14 (`_compute_maximum_cfl_and_clip_contravariant_vertical_velocity`) to it.

Change to the same pattern concerning predictor/corrector (`istep` Argument) and `_restricted` as the one above.

- fused_velocity_advection_stencil_15_to_18
get rid of the `levelmask` argument in `_add_extra_diffusion_for_w_con_approaching_cfl`(it is strictly obsolete as the information is available in `cfl_clipping` ) then potentially all stencils from 8 to 18 can be fused into one, as they all operate on cells.

- fused_velocity_advection_stencil_19_to_20

#### combined stencils in solve_nonhydro
Some of them are available in open branches in `icon4py` but except for
`fused_solve_nonhydro_30_to_38` they have never been tested.

The combined stencils for `Velocity Advection` are already in `icon4py`.
## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->

## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

Combined stencils integration - velocity advection:
- [x] Stencils 1 to 7 (The file name is changed to `compute_edge_diagnostics_for_velocity_advection.py`. There are two separate programs, `compute_derived_horizontal_winds_and_ke_and_horizontal_advection_of_w_and_contravariant_correction` is called in the predictor step, while `compute_horizontal_advection_of_w` in the corrector step)
- [X] Data set serialized
- [X] Code update wrt to latest dycore version
- [X] Datatest implementation
- [X] Datatest passed
- [X] Stencil test adjustments
- [X] Cleanup
- [x] Integration in solve_nonhydro
- [x] Stencil merged
- [x] Stencils 8 to 13 (The file name is changed to `compute_cell_diagnostics_for_velocity_advection.py`. There are two separate programs, `interpolate_horizontal_kinetic_energy_to_cells_and_compute_contravariant_terms` is called in the predictor step, while `interpolate_horizontal_kinetic_energy_to_cells_and_compute_contravariant_corrected_w` in the corrector step)
- [X] Data set serialized
- [X] Code update wrt to latest dycore version
- [X] Datatest implementation
- [X] Datatest passed
- [X] Stencil test adjustments
- [X] Cleanup
- [x] Integration in solve_nonhydro
- [x] Stencil merged
- [x] Stencils 15 to 18 (new name: `compute_advection_in_vertical_momentum_equation`. Predictor and corrector steps call the same stencil.)
- [X] Data set serialized
- [X] Code update wrt to latest dycore version
- [x] Datatest implementation
- [x] Datatest passed
- [x] Stencil test adjustments
- [X] Cleanup
- [x] Integration in solve_nonhydro
- [x] Stencil merged
- [x] Stencils 19 to 20 (new name: `compute_advection_in_horizontal_momentum_equation`. Predictor and corrector steps call the same stencil.)
- [X] Data set serialized
- [X] Code update wrt to latest dycore version
- [x] Datatest implementation
- [x] Datatest passed
- [X] Stencil test adjustments
- [X] Cleanup
- [x] Integration in solve_nonhydro
- [x] Stencil merged

Combined stencils integration - solve nonhydro:
- [ ] Stencils 1 to 13
- [x] Data set serialized
- [x] Code update wrt to latest dycore version
- [x] Datatest implementation
- [x] Datatest passed
- [x] Stencil test adjustments
- [ ] Cleanup
- [x] Integration in solve_nonhydro
- [ ] Stencil merged

- [ ] Stencils (14) 15 to 28 (The file name is changed to `compute_edge_diagnostics_for_dycore_and_update_vn.py`. There are two separate programs, `compute_theta_rho_face_values_and_pressure_gradient_and_update_vn_in_predictor_step` is called in the predictor step, while `apply_divergence_damping_and_update_vn_in_corrector_step` in the corrector step)
- [x] Data set serialized
- [X] Code update wrt to latest dycore version
- [X] Datatest implementation
- [x] Datatest passed
- [x] Stencil test adjustments
- [x] Cleanup
- [x] Integration in solve_nonhydro
- [ ] Stencil merged

- [ ] Stencils 30 to 38
- [ ] Data set serialized
- [x] Code implemented
- [x] Datatest implementation
- [ ] Datatest passed
- [ ] Cleanup
- [ ] Integration in solve_nonhydro
- [ ] Stencil merged

- [ ] Stencils 41 to 60
- [x] Data set serialized
- [X] Code update wrt to latest dycore version
- [x] Datatest implementation
- [ ] Datatest passed
- [x] Stencil test adjustments
- [x] Cleanup
- [ ] Integration in solve_nonhydro
- [ ] Stencil merged

- [ ] Stencils 61 to 65 (Not relevant for now, not much performance gain)
- [ ] Data set serialized
- [ ] Code update wrt to latest dycore version
- [ ] Datatest implementation
- [ ] Datatest passed
- [ ] Stencil test adjustments
- [ ] Cleanup
- [ ] Integration in solve_nonhydro
- [ ] Stencil merged