---
# System prepended metadata

title: NV on NEMO (NVIDIA & BSC-ES)

---

# NV on NEMO (NVIDIA & BSC-ES) 5 February 2025

Meeting Agenda:

---

- BSC: status and discussions
  - NEMO 4:
    - Rommel works on `ice_thd`, separate memory mode for MN5 seems to be functionally OK
    - Now it is time for performance study for `ice_thd`
    - Restore the functioning on LUMI-G: it is complicated...
  - NEMO 5:
    - preparing a patch for current NEMO5: unified approach to array compaction, easily allowing for GPU transition
    - this patch introduces couple of temp arrays and couple of extra fill/copy operations with them, but it seems to be a fair trade-off
  - nemo-build and psubmit:
    - let's commit the scripts
    - I can think of making dnb+psubmit combination work with containers in the most direct way -- but I need some input
  - Nsight Systems, cuda-gdb and compute-sanitizer:
    - issues on MN5: nsys profiling is difficult: for most of my current branches nsys is disfunctional
       - one case: https://forums.developer.nvidia.com/t/nvhpc-25-7-nsys-fails-when-collecting-a-profile-for-simple-fortran-program/347031 
       - another case: nsys just hangs on execution
    - also have difficulties with cuda-gdb and compute-sanitizer. 
    - can we approach the issues with tools systematically, i.e. by having NVIDIA Engineers access to MN5-ACC? There is also an issue with NVHPC 25.x and NEMO5 (crash in MPI_Wait).
  - Action required for BSC:
    - [ ] Integrate the Serailbox into nemo-build?
    - [ ] Share nsys logs with Lukas -- will do after merging in the NEMO code into IFS-NEMO

---

- NVIDIA: status and discussions
  - 1
  - 2
  
<!--
- NVIDIA:
    - 1st branch of investigations:
        - Looking into`-mp=gpu` reproducibility issue
        - Looking into `TANH` and `EXP` reproducibility ( affects CUDA Graphs )
        - CUDA graph perf check
        - Tab kernels
    - 2nd branch of investigations:
        - LBCLNK
    - 3rd branch of investigations:
        - PSyclone 
    - 4th branch of work:
        - NEMO build nvcluster changes cleanup
-->











