---
# System prepended metadata

title: 'SCC Training: Benchmarking GPAW'

---

# SCC Training: Benchmarking GPAW

Training reservation in Mahti: `SCC22` (partition `medium`)
CSC user documentation: https://docs.csc.fi
GPAW documentation: https://wiki.fysik.dtu.dk/gpaw
GPAW presentation (50 min, no need to watch for this training): https://www.youtube.com/watch?v=-z2M3g-o_sA

## Groups
- Ilmari, Leopekka, Roope: https://meet.csc.fi/SCC1
- Matias, Niklas, Huy: https://meet.csc.fi/SCC2

## Building GPAW

GPAW is a Python based software with a few Python dependencies, together with some other library dependencies. CSC's standard Python environments are container (Apptainer) based, which can sometimes complicate custom software installations, thus in this training we use a "special" Python environment in Mahti.

1. Load the Python module in Mahti:
```
module use /projappl/project_2005493/modules
module load python-gpaw
```

2. Get the source code

Download the version 22.1.0 along the instructions at https://wiki.fysik.dtu.dk/gpaw/install.html#getting-the-source-code

3. Install GPAW with `pip`

The downloaded code can be installed by issuing `pip install .` in the upmost directory (`gpaw`). As the default installation directory Python packages does not have write permissions, you need to provide install location either with 
  
  - by setting `PYTHONUSERBASE` environment variable and using `--user` option to `pip` 
  - `--prefix` option to `pip`

:::info 
Note! It is not recommended to use `--user` without `PYTHONUSERBASE` as it is very easy to get conflicting versions of packages under the default "user" directory in $HOME
:::

Missing Python requirements will be installed automatically, however, external library dependencies might require loading modules and customizing the installation. Look at https://wiki.fysik.dtu.dk/gpaw/install.html#customizing-installation
for some more details.

Note: in Mahti **libxc** (required) and **ELPA** (optional) are not available as modules but can be found at
```
/appl/spack/v017/install-tree/gcc-11.2.0/libxc-5.1.5-oa6ihp/
```
and
```
/appl/spack/v017/install-tree/gcc-11.2.0/elpa-2021.05.001-a3dh2f/
```

4. Testing the build

In order to run GPAW `bin` directory in the installation tree needs to be added to `PATH`. Also, `PYTHONUSERBASE` needs to be set (or the `installation_base/.../site-packages` added to `PYTHONPATH`). Furthermore, the PAW datasets need to be installed as described at https://wiki.fysik.dtu.dk/gpaw/install.html#install-paw-datasets

Once everything is set, a simple serial test calculation (can be done in the login node) can be run as
```
gpaw test
```
and the same thing in parallel (in a batch job) with
```
srun gpaw test
```

## Running the benchmark

The input file `si-divacancy.py` is available at `/scratch/project_2005493/gpaw-input` in Mahti, copy the file to directory where you carry out the benchmark runs. The GPAW input files are Python scripts themselves, and are run as `gpaw python input.py`, *i.e* here with batch job as
```
srun gpaw python si-divacancy.py
```
The main output file `output_si_divacancy.txt` which contains timing information at the end, look for total time, *i.e* `Total:`

:::warning
If you perform multiple runs in the same directory, the output file will be overwritten. Thus, make each run in separate directory or copy the output file between the runs.
:::

Running time with one node should be less than three minutes.

### Enabling ELPA

Even if GPAW was built with ELPA support, it is not automatically used, but needs to be enabled in the input file. See how to enable ELPA at https://wiki.fysik.dtu.dk/gpaw/documentation/parallel_runs/parallel_runs.html#parallelization-options
and investigate the effect of ELPA options on performance.




