---
# System prepended metadata

title: Consolidation of CI
tags: [cycle 16]

---

- Shaped by: Abishek, Christoph
- Appetite (FTEs, weeks):
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem

Carry over tasks from last cycle, see:
https://hackmd.io/0-yiWyJoTsCWOHql-7SkLw

We want what was discussed in Davos: 

As summary of what we discussed yesterday:
To prevent unintended effects downstream of icon4py two additonal tests need to be in place:
- check existing spack-recipe
- check build of icon-exclaim with icon4py of PR

If the spack-recipe breaks (ideally) a new tag is created beforehand and added to the spack-recipe in spack-c2sm.

Additionally the dev of icon4py makes sure his changes are reflected in the recipe before merge.

If icon-exclaim fails the dev has to find the reason for the failure in icon-exclaim/icon4py.
If a PR is merged regardless of failing tests, users/devs downstream are blocked!
A test to check new developments in gt4py repo with compatability of icon4py is added too.
Once Sam is back from service, Jonas could come to Oerlikon to do a training-session on Spack for the icon4py-devs.

In addition:
- Create gt4py PR test that tests icon-exclaim

See also:
https://docs.google.com/presentation/d/1-Br_BQeBEp-UXZRB-EbQUWG_8UsSnu9Zui6QwZb4YR0/edit#slide=id.g1e484e40b86_3_123

## Tasks

## Spack
- [ ] add Python dependencies as upstream in Balfrin (requires to transfer all deps to the spack instance of CSCS!) - talk to Ben Cumming
- [x] Add test to spack-c2sm testing (Jonas)
- [x] Adding a development environment (Jonas, Christoph)
- [ ] Adding a development environment (using rsync)

## CI 
- [ ] Jenkins pipeline to recreate similar functionality as on `tsa` on `balfrin` and `daint`, but using spack (Abishek)
- [ ] Make `exp.mch_bench_r019b07_dev_sppt.run` run on daint
- [ ] add tests for global conservation of quantitites of interest (Anurag)

- [ ] Find a consistent way to build the dependent projects (like `icon-exclaim` depending on `icon4py` and `gt4py`) with both, rolling release branches (`main`) or specific tags
    - For users there should always be a stable supported version of `icon-exclaim` and dependencies to run simulations
    - For developers there should be a "green" version which is fairly recent, so they can create PR branches
    
- [x] Add icon4py PR Jenkins plan which triggers icon-exclaim integration test
- [ ] Add gt4py PR plan which triggers icon4py tests
- [ ] (optional) Add gt4py PR plan which triggers icon-exclaim integration test


## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->
