# CSCS-ci for ICON4Py

- Shaped by: Magdalena
- Appetite (FTEs, weeks): 2x (1 week , 1 FTE)
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem
<!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this -->
We have reached a limit with the current setup of the ICON4py CI at the same time new requirements show up:

1. We currently run the ICON4py unit tests and QA actions on github runners. During the last cycle it turned out that we were consuming too many of the C2SM github action minutes, so as an immediate measure we reduced the CI triggers on github. As a consequence of this updates to PRs are not anymore automatically tested.

2. In addition, there are no branch protections rules setup in ICON4py. This leads to the situation that a developer eventually merge breaking code.

3. We have been working towards running benchmark tests for ICON4py stencil they are currently exempt from the CI runs and should be run on meaningful hardware (GPUs).

5. The greenline uses verification tests where a substantial amount of serialized test data from ICON runs is used to compare against. Those tests are currently only run manually but should be integrated in the CI. In order to do this need an environment where we can store the test data persistently.

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->
Definitely the first 3 points below should be tackled. Since the current situation in ICON4py is very unsatisfying. (~1 week) Points 4 to 6 could also be addressed in follow up projects. (~1 week)

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->
1. Reconfigure the CI actions currently in place (tox tests, QA) in ICON4py to run on CSCS CI and re-enable on update triggers.
2. Introduce branch protections rules, such that PRs can only be merged if tests and QA actions are green.
3. test CPU, GPU backends

4. Run benchmark tests on CI
5. Once the greenline is fully merged transfer the testdata to CSCS servers and run the `pytest.mark.datatests` on the CI. Data should be available on the container where the tests are run.

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->