# OpenMP Benchmarks (and some others)

I have a [build script](https://github.com/focs-lab/tsan-benchmarks/blob/main/setup.sh) that contains commands for building various benchmarks used by earlier works on data race detection.

Usage. (You may want to [comment out benchmarks](https://github.com/focs-lab/tsan-benchmarks/blob/82fd3cbf524381aad89b918e72352afb1b0ed539/setup.sh#L839-L862) that you don't need.)

export CUSTOM_LLVM_BUILD_PATH=/home/daniel/llvm-project/build
./build.sh

There is [another build script](https://github.com/focs-lab/tsan-benchmarks/blob/82fd3cbf524381aad89b918e72352afb1b0ed539/build.sh) that is supposed to rebuild all programs without downloading the source, but it has not been maintained. Ignore it.

### TSan without linking

The build script compiles all programs with the following flags.

-fsanitize=thread -fno-sanitize-link-runtime -fno-sanitize-link-c++-runtime -lclang_rt.tsan

This is so that the TSan runtime is not linked into the compiled binary (i.e. it will be dynamically linked during runtime). By default, the runtime is statically linked into the binary.

This is so that we don't need to recompile the benchmarks each time we make a change to the TSan runtime.

### Running

[Here are some example commands](https://github.com/focs-lab/tsan-benchmarks/blob/14a5834ab989f053c23f158fa7a38f873cd5cc70/experiment-template/testcases.yml) for running the benchmark programs (they come after `cmd: `). They do not cover all programs in the benchmark.

Before running a benchmark, you will need to set the `LD_LIBRARY_PATH` environment variable to contain the paths to the TSan and OpenMP libraries, since they are not statically linked into the binary (in this setup). For example:

export LD_LIBRARY_PATH=~/llvm-project/build/lib/clang/19/lib/x86_64-unknown-linux-gnu:~/llvm-project/build/runtimes/runtimes-bins/openmp/tools/archer:~/llvm-project/build/runtimes/runtimes-bins/openmp/runtime/src

Note that in the command above, we provide the paths to the TSan, OpenMP, and Archer libraries. [Archer](https://reviews.llvm.org/D45890?id=210043) is an OpenMP tool that adds TSan annotations to an OpenMP program.

:::warning
It is important to have Archer in the `LD_LIBRARY_PATH` for OpenMP programs, as otherwise all the synchronizations will be handled wrongly.

For OpenMP programs, also make sure to set `export TSAN_OPTIONS="ignore_noninstrumented_modules=1"` to avoid false positives from the OpenMP runtime.
:::

For evaluating performance, you may prefer to set `export TSAN_OPTIONS="report_bugs=0"` to turn off race reporting which contains additional locks and I/O that may cause some irregularities in the program run duration.

To have multiple TSan options, separate them with a space, e.g.

export TSAN_OPTIONS="ignore_noninstrumented_modules=1 report_bugs=0"

### Remarks

:::info
We should have a better build script. In particular:

1. Don't `rm -rf` everything at the start of the script.
2. Only download the source code if it is not present. Save time and also don't waste other people's server costs (e.g. GNU's FTP server).
:::