# [Greenline] CI for datatests

- Shaped by: Magdalena
- Appetite (FTEs, weeks):2 FTE, 1 week
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem
In icon4py we have test verifying components by comparing them to data serialized from ICON runs. Those test can range from verification of granules like `diffusion` and `solve_nonhydro` to simpler functions that calculate interpolation data. While running CI on github runners we had no way to make the necessary amount of data available to the runners hence those test were not integrated into the CI runs and we relied on developers running them manually in order not to break anything.

As we switched to the CSCS infrastructure last cycle we can now store the data there and mount it into the runners.

Up to now the granule `SolveNonHydro` and `Diffusion` were only verified against the data of the MeteoSwiss experment `mch_ch_r04b09_dsl` which means the code paths specific to global models were never verified. We want to improve on this situation.

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->
~1 week, maybe involve 2 people to distribute know how further.

## Solution

### CI
#### Make data available on CI
- Publish the serialized data on CSCS servers (ask Andreas Fink how to do it)
- mount the data to `icon4py/testdata/ser_icondata/mpitask{1,2,4}/mch_ch_r04b09_dsl/ser_data`
#### run tests with serialized data
- consolidate test markers and pytest options that are available for running different sorts of test: We have several pytest markers and pytest options to run different groups of tests: `--datatest` to run tests that need serialize data, `--with-mpi` to run parallel tests, `--grid` `--backend` to run stencil unit and benchmarks with a given grid or `gt4py` backend. The idea is to consolidate these options and come up with a set options that allows enough flexibility to run different sub groups of tests as needed.

- add jobs in the CI specification cscs.yml

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->

## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

- [ ] Verification of EXCLAIM APE experiment
- [ ] Subtask A
- [ ] Subtask X
- [ ] Add datatests to CI
- [x] Subtask H
- [ ] Subtask J