- Shaped by: Magdalena - Appetite (FTEs, weeks): 2 FTE, 1-2 weeks - Developers: <!-- Filled in at the betting table unless someone is specifically required here --> ## Problem Halo exchange calls with GHEX have been added to the python diffusion module in cycle 15. There are some leftovers: - GHEX is cloned and built manually, it cannot yet be installed via pip install - the code uses serialized ICON data (domain decomposition information, parallelized input/output fields for diffusion, parallelized metrics and interpolation fields) this data currently still is kept locally. - verification of the parallel diffusion run against the serialized data has not been finished. <!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this --> ## Appetite <!-- Explain how much time we want to spend and how that constrains the solution --> It is mostly a clean up project. - GHEX python build ~1 week - ICON4py branch cleanup/verification ~1 week ## Solution <!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand --> Bring the current experiment into a state that it can be merged and is usable for other developers (no/close to no manual build steps). There will probably remain a dependency on a system boost and MPI installation. - [x] GHEX - [x] add a python build cmake extension to GHEX (use pybind11/nanobind as blueprint) see example in https://github.com/pybind/scikit_build_example - [x] ICON4py - [x] add mpi4py and GHEX dependency to requirements.txt - [x] mpi4py - [x] pytest-mpi - [x] GHEX (see above) - [x] verify parallel diffusion run - [x] make serialized data for parallel runs available from online storage - [x] make sure all code runs in serial mode if no MPI/GHEX is present - [x] dummy driver - [x] tests - [x] documentation (README): pre requisites, run of MPI tests, dummy driver (?) - [x] fix logging configuration ## Rabbit holes <!-- Details about the solution worth calling out to avoid problems --> As always verification/debugging can be much more trouble some as expected. ## No-gos <!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->