# [Greenline] Halo Construction II

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Magdalena
- Appetite (FTEs, weeks):
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem
<!-- The raw idea, a use case, or something we’ve seen that motivates us to work on this -->
Individual granules (dycore, diffusion) can be run in parallel and do halo exchange. However icon4py model infrastructure lacks the feature of distributing the grid and setting up the the necessary halos and decomposition information.
Some steps towards this goal were taken roughly a year ago and there is a [draft PR](https://github.com/C2SM/icon4py/pull/540) that should be taken up and the work concluded.

See also this [old project](https://hackmd.io/rsVLV8q0S-imunbY5DuLag).

## Appetite
<!-- Explain how much time we want to spend and how that constrains the solution -->
full cycle.

## Solution
<!-- The core elements we came up with, presented in a form that’s easy for people to immediately understand -->
remaining task from [PR-540](https://github.com/C2SM/icon4py/pull/540):

- add tests:
- 1) distribute fields
- compute global to local indices (for local offsets, etc)
- compute (local) start/end indices
- 2) run stencil operation
- choose functions that require different halos (cell, edge, vertex) access
- 3) gather fields and compare to icon single node run.
- generalize for flexible number of halo lines
- generalize to LAM grids.

- finally fix and merge [PR-692](https://github.com/C2SM/icon4py/pull/692).

## Rabbit holes
<!-- Details about the solution worth calling out to avoid problems -->

## No-gos
<!-- Anything specifically excluded from the concept: functionality or use cases we intentionally aren’t covering to fit the ## appetite or make the problem tractable -->

## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

- [x] Halo Construction ([PR-540](https://github.com/C2SM/icon4py/pull/540))
- [x] compute halos for cells, edges, vertices (same size as ICON)

- [x] Halo Construction
- [x] global to local indices
- [ ] start end indices computation from refinement control
- [x] start end indices for LAM regitions (lateral boundary, nudging), [PR-845](https://github.com/C2SM/icon4py/pull/845)
- [x] start end indices for interior points, [PR-845](https://github.com/C2SM/icon4py/pull/845)
- [ ] start end indices for halo region
- [ ] distribution test: distribute-compute-gather ([PR-540](https://github.com/C2SM/icon4py/pull/540))
- [x] read field distribute - gather (no computation)
- [x] C2E: `compute_geofac_div` (field_op on embedded)
- [ ] fix GT4Py issue "disconnected" regions - "raise ValueError("Restriction generates non-contiguous dimensions.")" in`nd_array_fields.py._hyperslice...` order local dimensions?, change skip values config
- [ ] E2C:
- [ ] V2C:
- [ ] C2V:
- [ ] V2E2V:

- [ ] CI [PR-692](https://github.com/C2SM/icon4py/pull/692)
- TODO (magdalena): extract standalone project for somebody who is more skilled with the CSCS-CI and MPI setups than me (Maybe Mikael or Rico)

- [ ] generalize halo construction for flexible number of halo lines