# [Blueline] Py2F infrastructure: Verification, CI/CD on alps/balfrin, muli-process runs

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Christoph, Sam, (Anurag?)
- Appetite (FTEs, weeks): 1 cycle
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem

Currently missing:
* Way to verify each run of a granule against Fortran OpenACC
* Building icon-exclaim with the granule in one step (scripts, spack recipe)
* jenkins/(gitlab?) CI/CD on balfrin/santis based on venv
* successful verification of diffusion granule with multiple GPU

## Appetite
1 cycle

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->

Verification and debugging approach:

- Re-use verification interface (`verify_field`) of Liskov blue line integration.
- Probtest after the full experiment
- Document the way we do verification.
- Test the (ITIR -> dace) backend instead of (ITIR -> gtfn)
- Build recipe based on uvenv
- spack based?
- non-spack based?
- both?
- jenkins CI/CD on balfrin
- verification mode (compare granule at every execution to Fortran)
- substitution mode (only execute icon4py granule, probtest)
- gitlab on santis?
- Switch from mch_ch_r04b09 to small production experiments of MCH (icon1, icon2, kenda1)
- successfully verify diffusion granule for multiple GPUs (MCH production and aquaplanet)

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->
Do not try to replicate any workflow and data processing platform. Keep it simple.

## No-gos

No performance optimizations yet.
## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

- [x] Task 1 ([PR#xxxx](https://github.com/GridTools/gt4py/pulls))
- [x] Subtask A
- [x] Subtask X
- [ ] Task 2
- [x] Subtask H
- [ ] Subtask J
- [ ] Discovered Task 3
- [ ] Subtask L
- [ ] Subtask S
- [ ] Task 4