# EESSI hackathon Jan'22 - sync meeting notes (2022-01-19) Task progress ## [02] Installing software on top of EESSI * starting to implement to TODO items in https://github.com/easybuilders/easybuild-framework/issues/3918 * (Kenneth) support for conditional statements in module file guarded by check for defined environment variable * (Martin) modify `Toolchain.prepare_rpath_wrappers` to make it easier to leverage elsewhere * (???) enhance `rpath_wrapper_template.sh.in` to pick up `$RPATH_INCLUDE_DIRS` and `$RPATH_FILTER_DIRS` + provide control over location of log files * (Jacob) refactoring `create_rpath_wrappers` from enhanced `GCCcore` easyblock into framework * (Martin, Kenneth) installed custom `GCCcore/9.3.0` module that includes RPATH wrappers (using https://github.com/easybuilders/easybuild-easyblocks/pull/2638) * to use: `module use /mnt/shared/02_software_on_top/easybuild/modules/all` * (Frank) experimenting with manually building LAMMPS with CMake on top of EESSI, using custom `GCCcore/9.3.0` that includes RPATH wrappers * several small issues popped up * RPATH wrapper scripts have hardcoded path in `/tmp` for log file * wrapper scripts didn't have correct permissions (only execution rights for owner) * building LAMMPS manually was problematic, but working now * quite easy to mix up modules (build vs run), requires better module naming scheme * RPATH section is quite big currently * could consider using `patchelf` to shrink the RPATH section * maybe to include in RPATH wrapper scripts in EasyBuild framework * or a script that walks over install dir and runs `patchelf` on all binaries and shared libraries (which is what Compute Canada does) * TODO * also test with a configure-make-make install build? * good example? ## [03] Workflow to propose additions to EESSI software stack * (Kenneth) wrote some wrapper scripts around the software build script to facilitate autonomously installing software using the EESSI container: https://github.com/EESSI/software-layer/pull/163 * (Bob) tested these scripts on the CitC cluster using a Slurm job, and using a simple Easystack file * next step: let the app submit this job * TODO: * add CI for bot in https://github.com/EESSI/eessi-bot-software-layer ## [05] GPU support * (Alan) updated the [installation script to work with 2021.12](https://github.com/EESSI/hackathons/blob/05_gpu/2021-12/05_gpu/working_script_2021.12.sh) * will still be needed even if we include CUDA in EESSI (for compat libs) * (Alan) installed GROMACS with GPU support - seems to work * Also tried the CUDA-enabled OSU benchmarks but these segfault (I suspect because there is no RDMA capable device on `eessi-gpu`...but it might be that I am running them wrong) * Still needed: * EB plugin to add property to module files * Lmod plugin to hide modules with the property by default * Use path created by installation script to expose * Also allow explicit override via envvar * Port installation script to much cleaner and flexible version with lots of checks (space, write permissions, more error checking...) * TODO * to fix: `foss/2021a` is missing for `skylake_avx512` in `2021.12` EESSI pilot?! * EasyBuild hook to inject properties in modules for CUDA-enabled software * Lmod plugin to dynamically hide/expose modules based on properties * tedious details for script (disk space, latest driver check, etc.) ## [06] EESSI test suite * nothing to report :-( * Hugo's goal: verify test suite on IB CitC cluster on Azure * Kenneth would like to look into running 16-node GROMACS test on CitC in AWS ## [07] Monitoring (Terje) * Some poking at CVMFS monitoring options, nothing that feels productive as of yet. * need `/cvmfs` mount on Stratum-1 ## [16] Export a version of the EESSI stack to a tarball and/or container image (Jure) * local `/tmp` is indeed much faster * singularity `--fakeroot` still requires admin to set things up * user ID mapping in `/etc` (see https://sylabs.io/guides/3.5/user-guide/fakeroot.html#requirements-configuration) * singularity `%files` chokes on broken symlinks * to report upstream to have option to allow broken symlinks? * eessi init as part of `%post` or `%runscript` seems like too much (duplicated) effort * EESSI init script fails in container post/runscript, but fails because `$BASH_SOURCE` is not defined * EESSI init script could define `$BASH_SOURCE` if it's not defined yet? * Jörg: would sourcing `/etc/profile.d` first help? * `eessi-init` command may help here (see https://github.com/EESSI/software-layer/issues/147) * current state: polishing things up * how to switch loaded modules to equivalent `generic` modules?