# [Blueline] Py2F: Support multinode runs and dycore

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Sam, Abishek
- Appetite (FTEs, weeks): Full-cycle
- Developers:

## Problem
<!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this -->

In order to run full-scale simulations multi-node runs need to be supported in Py2F via GHEX. This project is a continuation of the work done in cycle 21 to fully support multi-node runs and verify results from such runs. Furthermore after diffusion the next step is to support calling the dycore from ICON, however a wrapper for the dycore is missing and the current dycore in icon4py needs to be adapted to work with py2f.

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->
Full-cycle

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->

- Enabling multi-node runs:
- Support running experiments on multiple nodes and be able to verify those (probtest, cdo). Currently runs on two cores with MPI on CPU framework.
- Run on >2 cores and then multiple nodes (CPU)
- Try multi-node runs on GPU
- Does GHEX work out of the box for GPU?
- Measure overhead of multi-node runs
- Try possible speed ups.

- Integrate dycore granule:
- Run the ICON4Py Dycore using the GTFN GPU backend, ideally with an integration test (datatest) for that in the CI.
- Write a Python interface for a dycore granule and create a unit test to test it from Python.
- Use Py2F to generate corresponding bindings for the dycore wrapper.
- Integrate this into Fortran, writing a Fortran call to Py2F bindings, passing all the correct parameters and arrays.
- Verify dycore results.

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->
[Fill in any potential complications or areas where additional attention may be needed]

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the appetite or make the problem tractable -->
[Fill in any functionalities or use cases intentionally excluded]

## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As the first task during building, add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->
[This section will be filled during the project implementation phase]

- [x] Run ICON4Py Dycore using GTFN GPU backend (after several fixes)