In the repository:
https://github.com/AaltoSciComp/hpc-software-tests/

# Test naming convention:

Each test is named like this:
`SOFTWARE/TEST-NAME-WITHOUT-UNDERSCORE/run_TEST-NAME-WITHOUT-UNDERSCORE.sh`

find . -name 'run_*.sh'

The folder hpc_tests has the configs.

# Misc notes
python wrapper to run sbatched tests

The scripts are based on **pytest**.

Everything runnable as normal user.

sbatch runs with --wait

# Each test
Each test has zero return value if successfull/correct.

Tests take list of modules as command line parameter

runnable separately:

sbatch run_hello-world.sh openmpi/4.1.6

Each test will run in the git repo folder of the test.

# configuration file for tests
`pytest_generate_tests()`

`hpc_tests/aalto_dev.yaml`

- *extra modules* are loaded first
- __then one run for each *module*__ (for loop over modules)
- there is also *global* where you can set extra parameters/requirements
- *localrun*: instead of sbatch run them on the submit node or workstation

naming convention similar to file structure

# Creating a new test
Take an example case, such as hello-world from mpi

Often download example input if it doesn't exist yet.

If there is a cleanup needed, it usually done at the beginning of run.

The whole `run_*.sh` is submitted as a sbatch job.

# Running via the suite

bash run_tests.sh --help
bash run_tests.sh --conf hpc/tests/aalto-desktop.yml --list
bash run_tests.sh --conf hpc/tests/aalto-desktop.yml --keyword gcc_hello-world-gcc_8.4.0

(--keyword is grepping to match tests, don't use --test)
(--dev will run all the aalto-dev side test, whole tests in the order of 10mins)

# osu-micro-benchmark MPI tests are outside the suite/framework

`compile_osu-micro-benchmark.sh` is a helpful wrapper to compile using a specific module, e.g.:

`./compile_osu-micro-benchmarks.sh mpicc mpic++ openmpi/4.1.6`
(Creates bins in the build folder, Check the Known issues in the README.md and remove the failing tests)

`sbatch --constraint=milan run-osu-microbenchmarks.sh openmpi/4.1.6`

## Output

Pretty much manual work to look at the output.
The bandwidth should be ~10GB/s.
Latency should be low... few or few tens of microseconds.

# Simppa's workplan

1. convert this document to wiki
2. create a image-test-suite
3. ior test suite is missing for simppa (module exists already)