# OVERVIEW - Cycle 29 05/25

- Cycle:
- **Betting table: 13.05.2025**
- **Review meeting: 03.06.2025** ([slides](https://docs.google.com/presentation/d/1UQZKAHdHQHpVRJfJ0hiuShCM-BlAxSobAVgVFx3gUjg/edit?usp=sharing))
- Participants:
+ CSCS: Edoardo, Enrique, Will
+ EXCLAIM: Anurag, Magdalena, Mauro
+ MCH: Christoph

## Available people:

+ GridTools:
* Christos: 50% or a bit less (rest on PASC projects -push a bit more at the beginning-)
* Edoardo: 100%
* Enrique: 100%
* Hannes: 0%
* Philip: 60% (because of LUMI)
* Rico: 80-90 % (Anticipate sporadic SwissTwins work)
* Sara (ECMWF): 0% (working on distributed PMAP-GO + PASC poster)
* Till: 66% (12.05-23.05 only, rest vacation)
* Ioannis: 50% (because of cornerstone)
+ CSCS:
* Mikael (Simberg): 50%
* Giacomo: 50% (rest on Flang investigation and Tabascal)
* Will: 50% (rest: performance of GT4Py muphys, EXCLAIM Symposium)
* Prashanth: 40% (mainly for I/O bits of ICON4Py)
+ EXCLAIM:
* Chia Rui:
* Jacopo: 5-10% (helping out with Hannes' open PRs)
* Magdalena: Full Cycle
* Nikki: Full Cycle
* Yilu: almost full cycle (will be away for 4 days)
+ MCH:
* Christoph: 100%
* Daniel (50%):

## Cycle Goal

**Meaningful performance baseline for the Dycore granule**

Same as Cycle 28, but also improving performance beyond what we see right now. ICON-EXCLAIM running with ICON4Py and GT4Py-DaCe `main` branches, with minimal Python overhead and meaningful performance benchmarks. This also requires finishing the combined programs and merging all the half-finished features like `concat_where`, and complete the benchmarking infrastructure to support writing more optimizations in next cycles.

## Tasks

| Project | Appetite | Developers | Support |
|:------------------------------------------------------------------------------------- |:---------- |:------------------------------- |:--------- |
| [DaCe] Optimization VIII [#][dace-optimization-22] | full cycle | Philip, Ioannis | Edoardo |
| [DaCe] Toolchain Runtime Support [#][dace-toolchain-run30] | full cycle | Edoardo, Giacomo | Enrique |
| [GT4Py] Concat where cleanup & merge [#][gt4py-concat-where32] | full cycle | Till | Enrique |
| [ICON4Py] Benchmarking infrastructure [#][icon4py-benchmarki35] | full cycle | Enrique, Christos, (Magdalena?) | Chia Rui |
| [ICON4Py] Combined solve non_hydro stencils [#][icon4py-combined-s41] | 1 week | Christoph (until done) | Nikki |
| [ICON4Py] Hannes' fixes 1: Static variants of programs [#][icon4py-hannes-fix50] | 1 week | Nikki | Edoardo |
| [ICON4Py] Hannes' fixes 2: Fix skip values [#][icon4py-hannes-fix38] | 1 week | Magdalena | |
| [ICON4Py] Hannes' fixes 3: Fix validation errors [#][icon4py-hannes-fix44] | 1 week | Jacopo, Daniel | Christoph |
| [ICON4Py] Small CI improvements [#][icon4py-small-ci-i29] | 1 week | Magdalena | Enrique |
| [ICON4Py] Standalone greenline [#][icon4py-standalone28] | full cycle | Mikael, Yilu | Magdalena |
| [ICON4Py] Improving performance for CFL condition in dycore [#][icon4py-improving-57] | 2 weeks | Christoph, Chia Rui | Magdalena |
| [Blueline] ICON with Granule deployment [#][blueline-icon-with37] | full cycle | Rico, Will | Christoph |
| Data Compression Project [#][data-compression-p24] | 1 week | Nikki, Christos | |

<!-- Links -->
[dace-optimization-22]: https://hackmd.io/@gridtools/ByFhRhFlll "[DaCe] Optimization VIII"
[dace-toolchain-run30]: https://hackmd.io/@gridtools/Syu217kZge "[DaCe] Toolchain Runtime Support"
[data-compression-p24]: https://hackmd.io/@gridtools/BJNR2DyZxg "Data Compression Project"
[gt4py-concat-where32]: https://hackmd.io/@gridtools/SyIdscjxge "[GT4Py] Concat where cleanup & merge"
[icon4py-benchmarki35]: https://hackmd.io/@gridtools/BkeHccillg "[ICON4Py] Benchmarking infrastructure"
[icon4py-combined-s41]: https://hackmd.io/@gridtools/ryyRmUfCyg "[ICON4Py] Combined solve non_hydro stencils"
[icon4py-hannes-fix38]: https://hackmd.io/@gridtools/rJpqIw1-gl "[ICON4Py] Hannes' fixes 2: Fix skip values"
[icon4py-hannes-fix44]: https://hackmd.io/@gridtools/SytNU9ybee "[ICON4Py] Hannes' fixes 3: Fix validation errors"
[icon4py-hannes-fix50]: https://hackmd.io/@gridtools/SJNzBUyWle "[ICON4Py] Hannes' fixes 1: Static variants of programs"
[icon4py-improving-57]: https://hackmd.io/@gridtools/Bk01fFlbll "[Icon4Py] Improving performance for CFL condition in dycore"
[icon4py-small-ci-i29]: https://hackmd.io/@gridtools/SJb0TIjegx "[ICON4Py] Small CI improvements"
[icon4py-standalone28]: https://hackmd.io/@gridtools/BkvlzXjggx "[ICON4Py] Standalone greenline"
[blueline-icon-with37]: https://hackmd.io/@gridtools/Bys907bAke "[Blueline] ICON with Granule deployment"

----------------------------------------------------------------
----------------------------------------------------------------

## Planning Action points

- ~~@egparedes sync with @mluz about the icon4py tasks~~
- ~~combined stencils~~
- ~~RBF & standalone grid status~~
+ ~~CI improvements~~
- ~~@egparedes ask for possible dates for the group brainstorming meeting (if needed) and betting table and send the invites~~
- ~~@egparedes send betting table invite~~
- sync and discuss with all people involved on different spack+venv projects (@ricoh @muellch @Will Sawyer, ....)

## Brainstorming

- benchmarking infrastructure:
1. StencilTests with larger grids (either standalone or with serialized data)
- validate with small grid
- benchmark with large grid
- accept static args
2. green-line JW
- intermediate step: serialize APE with R02B07
- switch to standalone grid when ready
- MCH experiment: ICON-CH2-small for validation, Christoph s ICON-CH2 for benchmark
- add to bencher benchmark diffusion, dycore
3. blue-line + granule (ape large grid?)
- add to bencher
- ~~profiling & identifying the low performance in the blueline + granule~~
- gt4py
- merge performance counters as soon as they are merged in gt4py
- finish the concat_where
- dace
- reduce overhead of the program calls (dace)
- toolchain instrumentation
- continue with DaCe optimizations
- parallel runs with DaCe
- icon4py
- take over and merge PRs from [Hannes handover document](https://hackmd.io/lIheBruuSNOSd6NMqppUFQ)
- finish the combined stencils (one week?)
- ~~consolidate the serialization points in icon-EXCLAIM (required for the serialization of the larger grids needed for benchmarking) (few days?)~~
- standalone: (1 cycle, both tasks could go in parallel )
- finish RBF validation
- glue code to start using them and merge and integrate z_ifc branch.
- Simple CI improvements: integrate nox utils to skip uneeded test execution (few days?)
- fix CFL reduction in icon4py dycore
- ~~PASC Poster~~
- ICON uenvs --> needs larger discussion ??
- missing experiments:
- is it possible to run spack-installed uv in the install phase of a dependent spack package in the context of a uenv build? (locally -> uv does not run because spack does not expose libbz2 to it during staging of the dependent package, why?). Possible fix: depend on `bzip2` as a build dependency in the icon4py spack package?
- IF YES -> only needed to add a venv with icon4py for runtime in uenv post-install
- IF NO -> consider alternatives: `pip install` instead of `uv sync`, install `icon4py` and then `ICON` both in post-install
- uenv deployment ?
- ICON CI (Will, Daniel) ?