# [Blueline] Examine icon-exclaim memory usage

<!-- Add the tag for the current cycle number in the top bar -->

- Shaped by: Christoph
- Appetite (FTEs, weeks):
- Developers: <!-- Filled in at the betting table unless someone is specifically required here -->

## Problem

icon-exclaim uses more memory than icon-nwp, part of the problem is known (`nblocks_e vs nblocks_c`) but some unnecessary allocations might be done.

## Goal

Develop knowledge about icon-exclaim's memory usage, develop unnecessary usages and measure and justify any other usage that exceeds the icon-nwp behaviour (example `nblocks_e` is necessary for icon-exclaim to work with icon4py).

## Steps

- Ask Andreas Jocksch about the knowledge he developed
- Check git diff between icon-nwp and icon-exclaim, examine all memory allocations
- Measure icon-nwp memory usage with `nblocks_e`, compare to icon-exclaim memory usage with `nblocks_e` with and without the DSL.
- Investigate lifetime of local fields (manual temporaries) in icon4py.
- Push Dmitry's memory pool PR.
- Investigate if the backends (dace/gtfn) allocate and de-allocate temporaries in the desired way (too often, too rarely)

## Progress

all in double precision

### ICON-CH1
on two nodes
| version | nblocks_c=1 | nblocks_e=1 |
| ---------------------------------- | ----------- | ------------ |
| icon-nwp (42a24ad3ef) | 68.30GB | 82.43GB |
| icon-dsl (17d75a220, build_acc) | 70.81GB*| 85.01GB |
| icon-dsl (17d75a220, build_gpu2py) | - | ** |

The runs marked with an asterisk (*) crashed (one rerun has been attempted).

The runs marked with an asterisk (**) crashed with out-of-memory.


### ICON-CH2
on one node
| version | nblocks_c=1 | nblocks_e=1 |
| ---------------------------------- | ----------- | ------------ |
| icon-nwp (42a24ad3ef) | 59.54GB | 66.85GB |
| icon-dsl (17d75a220, build_acc) | 60.73GB | 68.17GB*|
| icon-dsl (17d75a220, build_gpu2py) | - | 73.56GB*|

The runs marked with an asterisk (*) crashed (one rerun has been attempted).

The runs marked with an asterisk (**) crashed with out-of-memory.