---
# System prepended metadata

title: ExCALIBUR H&ES Benchmarking demo

---

# ExCALIBUR H&ES Benchmarking demo

## June 23 Cambridge RSE seminar notes

### Local Demo

1) Clean spack chache
   ```
   rm -rf ~/.spack/cache
   ```
3) Create virtual environment
   ```
   python3 -m venv demo_python_env
   source demo_python_env/bin/activate
   ```
3) Install spack
   ```
   git clone -c feature.manyFiles=true https://github.com/spack/spack.git
   source spack/share/spack/setup-env.sh
   ```
4) Install framework
   ```
   git clone https://github.com/ukri-excalibur/excalibur-tests.git
   pip install -e ./excalibur-tests
   export RFM_CONFIG_FILES=$(pwd)/excalibur-tests/behcmarks/reframe_config.py
   ```
5) Setup spack environment (Done automatically on supported systems)
   ```
   spack env create -d ./demo_spack_env
   spacktivate ./demo_spack_env
   spack compiler find
   spack external find openmpi
   spack external find python
   spack config blame [packages, compilers]
   export EXCALIBUR_SPACK_ENV=$(pwd)/demo_spack_env
   ```
6) Run examples
   ```
   reframe -c excalibur-tests/benchmarks/examples/sombrero -r --performance-report
   ```
   ```
   reframe -c excalibur-tests/benchmarks/apps/imb -r --performance-report --tag pingpong
   ```
7) Look at output files
   ```
   less output/default/default/default/SombreroBenchmark/rfm_build.out
   less perflogs/default/default/SombreroBenchmark.log
   ```

### Demo on CSD3

1) SKIP Clean spack chache
   ```
   rm -rf ~/.spack/cache
   ```
2) Create virtual environment
   ```
   module load python/3.8
   python3 -m venv demo_python_env
   source demo_python_env/bin/activate
   pip install --upgrade pip
   ```
3) SKIP - Install spack
   ```
   git clone -c feature.manyFiles=true https://github.com/spack/spack.git
   source spack/share/spack/setup-env.sh
   ```
4) Install framework
   ```
   git clone https://github.com/ukri-excalibur/excalibur-tests.git
   pip install -e ./excalibur-tests
   export RFM_CONFIG_FILES=$(pwd)/excalibur-tests/behcmarks/reframe_config.py
   ```
5) SKIP - Setup spack environment (Done automatically on supported systems)
   ```
   spack env create -d ./demo_spack_env
   spacktivate ./demo_spack_env
   spack compiler find
   spack external find openmpi
   spack external find python
   spack config blame [packages, compilers]
   export EXCALIBUR_SPACK_ENV=$(pwd)/demo_spack_env
   ```
6) Run example
   ```
   reframe -c excalibur-tests/benchmarks/examples/sombrero -r --performance-report
   ```
   NOTE: Building was very slow on skylake -- should probably run on icelake or cascade lake nodes.
7) Look at output files
   ```
   less output/default/default/default/SombreroBenchmark/rfm_build.out
   less perflogs/default/default/SombreroBenchmark.log
   ```


## July 22 Met Office meeting -- Presenter notes

1) Clean up
   - clear spack cache if it exists `rm -rf ~/.spack/cache`
   - unload python module(s)
   - deactivate conda
2) Install reframe
   - git clone [reframe](https://reframe-hpc.readthedocs.io/en/stable/started.html#github) 
     ```
      git clone -q --depth 1 --branch v3.10.1 https://github.com/reframe-hpc/reframe.git
      cd reframe
      ```
   - run reframe bootstrap script `./bootstrap.sh`
   - optionally, add reframe to path `export PATH=$(pwd)/bin:$PATH`
3) install spack
   - git clone [spack](https://spack.readthedocs.io/en/latest/getting_started.html#installation)
   ```
   git clone -c feature.manyFiles=true https://github.com/spack/spack.git
   ```
   - run spack setup script `source ./spack/share/spack/setup-env.sh`
   - check spack is in `$PATH`, for example `spack --version`
4) install excalibur-tests
   - git clone excalibur-tests
   ```
   git clone https://github.com/ukri-excalibur/excalibur-tests.git
   ```
   - set reframe config file `export RFM_CONFIG_FILE="$(pwd)/excalibur-tests/reframe_config.py"
`
5) run benchmarks :tada: 
   - run sombrero example
   ```
   reframe/bin/reframe -c excalibur-tests/examples/sombrero/ -r --performance-report --job-option='--account=do006'
   ```
   - run Intel MPI benchmarks
   ```
   reframe/bin/reframe -c excalibur-tests/apps/imb/ -r --performance-report --job-option='--account=DIRAC-DO006-CPU' --system csd3:icelake -t intel-mpi -t biband
   ```

### Hacks
 - On csd3, we had to manually request sombrero to be built with openmpi (instead of impi) using `-S spack_spec=sombrero@2021-08-16^openmpi` to reframe
 - If a benchmark has defined tags, pass `-t` to include a tag, `-T` to exclude a tag
### Project codes
- cosma8: `do006`
- csd3: `DIRAC-DO006-CPU`