Install Seissol on Taiwania3 (Intel Node)
=============
### Module load GCC
```bash=
ml gcc/13
```
### Install Spack
```bash=
git clone https://github.com/spack/spack.git
. ~/spack/share/spack/setup-env.sh
```
### Download Dependencies for Seissol
```bash=
spack install seissol convergence_order=4 gemm_tools_list=LIBXSMM,OpenBLAS,PSpaMM graph_partitioning_libs=parmetis
```
### Fail on easi(110/111)
This cmake would automatically check the compiler is work or not.
Ashole just can't fetch the fucking GCC.
=> Specify path for it.
=> Noted that we should specify the `openmpi` path for it. Or this would happen again when it starts to detect `MPI_C_COMPILER` ==
Edit the cached install package in `~/spack/var/spack/cache/_source-cache/git/SeisSol/easi.git`
```bash=
cp ~/spack/var/spack/cache/_source-cache/git/SeisSol/easi.git/0d87b1a7db31e453d52c7213cb9b31bda88cbf40.tar.gz ~
tar xvf 0d87b1a7db31e453d52c7213cb9b31bda88cbf40.tar.gz
```
After decompress it, you should get a `easi` directory on your `~`
Append this two line into the CmakeList.txt
(`~/easi/CMakeLists.txt`)
```bash=
set(CMAKE_C_COMPILER "~/spack/opt/spack/linux-cascadelake/openmpi-5.0.7-eee5iqsn5ncps7x3jnlf64334v5mq4yg/bin/mpicc")
set(CMAKE_CXX_COMPILER "~/spack/opt/spack/linux-cascadelake/openmpi-5.0.7-eee5iqsn5ncps7x3jnlf64334v5mq4yg/bin/mpic++")
```
Like this:
Finally, tar this modified `~/easi` directory back to the same name as the .tar.gz in spack cache.
```bash=
tar -cvf 0d87b1a7db31e453d52c7213cb9b31bda88cbf40.tar.gz easi
```
Drag this .tar.gz back and replace the original one.
OK, now we deal with this ashole `easi` again.
```bash=
spack install seissol convergence_order=4 gemm_tools_list=LIBXSMM,OpenBLAS,PSpaMM graph_partitioning_libs=parmetis
```
This time it should be OK.
### Fail on Seissol(111/111)
Then this should failed, bitch ==
But not a big deal.
We could have it manually.
### Build Seissol from source
```bash=
git clone https://github.com/SeisSol/SeisSol.git
mv SeisSol seissol_tmp
cd seissol_tmp
git submodule update --init --recursive
mkdir build && cd build
```
### Spack load ...
```bash=
spack load gcc openmpi cmake eigen easi hdf5 \
netcdf-c py-pspamm openblas openmpi parmetis \
yaml-cpp libxsmm
```
### Add netcdf-c into LIBRARY_PATH and LD_LIBRARY_PATH
```bash=
export LD_LIBRARY_PATH="/home/jason3410/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/lib:$LD_LIBRARY_PATH"
export LIBRARY_PATH="/home/jason3410/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/lib:$LIBRARY_PATH"
export C_INCLUDE_PATH="/home/jason3410/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/include:$C_INCLUDE_PATH"
export CPLUS_INCLUDE_PATH="/home/jason3410/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/include:$CPLUS_INCLUDE_PATH"
```
### Python Dependencies
```bash=
python -m ensurepip --upgrade
pip3 install --upgrade pip
python -m pip install numpy
python -m pip install pkg_resources
python -m pip install setuptools
python -m pip install scipy
```
### Install
```bash=
#hsw: haswell, supports AVX2
cmake -DCMAKE_BUILD_TYPE=Release \
-DNUMA_AWARE_PINNING=ON \
-DASAGI=ON \
-DPRECISION=double \
-DORDER=4 \
-DEQUATIONS=elastic \
-DGRAPH_PARTITIONING_LIBS=parmetis \
-DCMAKE_PREFIX_PATH="~/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/lib
..
make -j$(nproc)
```
### Update Linker
Another shitty error here is:
```bash=
/usr/bin/ld: /home/jason3410/spack/opt/spack/linux-cascadelake/libxsmm-1.17-mzyyr5lb6rvpo4jywp6ckziho5wwilah/lib/libxsmm.a(libxsmm_main.o): unable to initialize decompress status for section .debug_info
/usr/bin/ld: /home/jason3410/spack/opt/spack/linux-cascadelake/libxsmm-1.17-mzyyr5lb6rvpo4jywp6ckziho5wwilah/lib/libxsmm.a(libxsmm_main.o): unable to initialize decompress status for section .debug_info
/home/jason3410/spack/opt/spack/linux-cascadelake/libxsmm-1.17-mzyyr5lb6rvpo4jywp6ckziho5wwilah/lib/libxsmm.a: error adding symbols: File format not recognized
collect2: error: ld returned 1 exit status
```
It is due the the out-of-date linker version.
To solve this, we install latest version of linker with spack.
```bash=
spack install binutils
spack load binutils
```
Now:
### Done
Finally, we could finish this.
```bash=
make -j$(nproc)
```
### Testing
```bash=
[jason3410@cpn3006 build]$ ~/seissol/build/SeisSol_proxy_Release_dhsw_4_elastic 100000 100 all
Allocating fake data... ...done
=================================================
=== PERFORMANCE SUMMARY ===
=================================================
seissol proxy mode : all
time for seissol proxy : 2.51268
cycles : 0
GFLOP (libxsmm) : 0
GFLOP (pspamm) : 0
GFLOP (libxsmm + pspamm) : 0
GFLOP (non-zero) for seissol proxy : 507.137
GFLOP (hardware) for seissol proxy : 867.065
GiB (estimate) for seissol proxy : 193.119
FLOPS/cycle (non-zero) : inf
FLOPS/cycle (hardware) : inf
Bytes/cycle (estimate) : inf
GFLOPS (non-zero) for seissol proxy : 201.831
GFLOPS (hardware) for seissol proxy : 345.075
GiB/s (estimate) for seissol proxy : 76.8577
=================================================
```
### Env-setup.<span>sh (for future usage)
```bash=
#! /bin/bash
ml gcc/13
spack load gcc openmpi cmake eigen easi hdf5 \
netcdf-c py-pspamm openblas openmpi parmetis \
yaml-cpp libxsmm binutils python
export LD_LIBRARY_PATH="/home/jason3410/spack/opt/spack/linux-cascadelake/netcdf-c-4.9.2-bi6acrct4tzme7nyxagyj4udl6erfzzt/lib:$LD_LIBRARY_PATH"
```
Sign in with Wallet
Connect another wallet