---
# System prepended metadata

title: icon-dsl with seriaization build on balfrin

---

# icon-dsl with serialization build on balfrin

Below is a guide of how to compile icon-dsl on balfrin with serialization. Currently, serialbox library is not in the user environment. You have to install it manually.

## Before you start

For the build described in this hackmd, you need to have `dsl/env.sh`, `dsl/setup.sh`, and `config/cscs/alps_nospack`.
They were introduced in this PR: https://github.com/C2SM/icon-exclaim/pull/283 so until they are not merged in main, borrow them from there.


## Steps

1. Prepare the uenv image. Details about uenv can be found on https://confluence.cscs.ch/display/KB/UENV+user+environments
```bash
git clone https://github.com/eth-cscs/uenv.git
cd uenv
# run the installation script.
# if this is the first time that you are installing uenv,
# type "yes" or "y" when prompted whether to update your ~/.bashrc file.
./install --local
```

2. activate the environment (you need this also for running ICON, so put it in your run scripts or bashrc)
```bash
uenv start --view=icon-wcp:icon /scratch/mch/leclairm/uenvs/images/icon.v1.rc4.sqfs
```
You should see something as shown below when you type ```uenv status```
```bash
/user-environment:icon-wcp
  The tools required to build ICON
  views:
    icon (loaded)
    modules: activate modules
    spack: configure spack upstream
```

3. clone the repo and move to the branch you need to work with
```bash
git clone git@github.com:C2SM/icon-exclaim.git
cd icon-exclaim
git submodule update --init --recursive
```

4. Prepare the serialbox library

4.1. **Spack**
```bash
git clone --depth 1 --recurse-submodules --shallow-submodules -b v0.21.1.3 https://github.com/C2SM/spack-c2sm.git
cd spack-c2sm
source setup-env.sh # this line should actually be `source setup-env.sh /user-environment`, but neither Chia Rui nor Jacopo have tested it
spack install serialbox%nvhpc@24.5 +fortran # this will take quite some time
```
if spack does not automatically detect the compiler, you can do the following to find the correct compiler and rerun the command above
```bash
spack compiler find
```

In the script ```setup.sh```, you need to add the following lines
```bash
source /scratch/mch/<your-user-name>/spack-c2sm/setup-env.sh # wherever this is
spack load serialbox%nvhpc@24.5
```

and in `env.sh` change this variable

```bash
SERIALBOX2_ROOT=$(spack find --format "{prefix}" serialbox@2.6.1%nvhpc@24.5 | head -n1)
```

4.2. **No spack**
Or, if you do not want to use spack, you can manually install serialbox by (which is much faster than spack install above)
```bash
git clone https://github.com/eth-cscs/serialbox2.git
cd serialbox2/
mkdir build
cd build/
export CC=mpicc
export FC=mpif90
export CXX=mpicxx
cmake ../ -DSERIALBOX_ENABLE_FORTRAN=ON
cmake --build .
cmake --build . --target install
```

Now you have to change `SERIALBOX2_ROOT` to your serialbox installation path in the script `env.sh`, for example,
```bash
SERIALBOX2_ROOT=/scratch/mch/cong/serialbox2/install
```

5. Build icon-dsl. 

Building on cpu:
In `setup.sh`:
 - add an option for a new `build_type` at the top: `build_serialize_cpu`;
 - in `build_for_type()` copy the build_cpu options as follows
```bash
build_for_type() {
    local build_type=$1
    if [[ "$build_type" == "build_cpu" || "$build_type" == "build_serialize_cpu" ]]; then
        build_dir="build_cpu"
            gpu_option=""
        liskov_option="--enable-dsl-local"
        ACC_FC_FLAGS=""
        CUDA_LDFLAGS=""
        CUDA_LIBS=""
        NVCFLAGS=""
        if [ "$build_type" == "build_serialize_cpu" ]; then
            build_dir="build_serialize_cpu"
            liskov_option="--enable-serialization"
        fi   
```

Finally, copy `alps_nospack` to `icon-exclaim/dsl/icon-exclaim/config/cscs/`. Then, run the `setup.sh` script (note that the setup script and `env.sh` must be at the same directory with icon-exclaim, icon4py, and gridtools)
```bash
./setup.sh build_serialize
```


## Troubleshooting:

a. If you encounter this error:
```bash
NVFORTRAN-F-0004-Unable to open MODULE file mo_nonhydro_state.mod (/scratch/mch/jcanton/icon-exclaim/icon-exclaim/src/shr_horizontal/mo_icon_interpolation_scalar.f90: 35)
NVFORTRAN/x86-64 Linux 24.5-1: compilation aborted
```
you need to remove the serialization code (lines with `!$ser`) from `mo_icon_interpolation_scalar.f90`.

b. Similar as above:
```bash
make[1]: Entering directory '/scratch/mch/jcanton/icon-exclaim/icon-exclaim/build_serialize_cpu'
  FC       src/shr_horizontal/mo_math_gradients.pp-fpp.pp-sb2.o
NVFORTRAN-F-0004-Unable to open MODULE file mo_nonhydro_state.mod (/scratch/mch/jcanton/icon-exclaim/icon-exclaim/src/shr_horizontal/mo_math_gradients.f90: 45)
NVFORTRAN/x86-64 Linux 24.5-1: compilation aborted
```
remove serialization code in `mo_math_gradients.f90`