# Getting Scientific Software Installed VSC training

-------------------------------------------------------------------------------------------

# Questions (20251103)

## Central software stack

- Will modules be affected (or dissapear) with the migration to Rocky9? If yes, are there some modules planned to be reinstalled?
- how to choose the module version?

## EESSI

- Is LIMA still the best workaround for using eessi on macOs?
- admin access to use EESSI, install software
- EESSI @ KUL?
- If you dont module purge, will EESSI modules interact with VSC modules?
- Should software requests still happen the traditional way for HPC-UGent clusters?
- more (bioinformatics) software in EESSI to make it easier to jump from Tier-2 to Tier-1

## Using containers

- How to run tblite example with Apptainer?
- Where can container images be stored other than DockerHub?

## conda & co

- Reasons for discouraging of conda on VSC cluster?
- Building conda packages from source

## pip + venv

- Should recommend uv?
- Is it possible in the environment to load any type of modules, or only python packages?
- Is it possible to activate the enivronments using an interactive session e.g. Jupyter Notebook or Rstudie?
- https://docs.vscentrum.be/compute/portal/ondemand/jupyterlab.html#virtual-environments-for-python
- Does vsc-venv import packages from (Lmod) modules?
- Does this work with job scripts?

## R, Julia

- When I tried using RStudio (OnDemand app) with RLibs on VSC_DATA, RStudio kept crashing. Is the “RProject way” the only way to work in RStudio and RLibs on VSC_DATA (and not VSC_HOME)? Could you explain how RStudio relates to R on the HPC? (Sorry if there is documentation that already answers this, I could not find it.)
- If installing R packages in environment, how do we share it with colleagues? (A: Copy the folder, or use project/VO directory)

## Software compilation

- Difference between CMake and Ninja (A: CMake configures, Ninja builds; Ninja is a drop-in replacement for GNU make)
- Do we have to re-compile if we want to use it on a different node?

## Building containers

- Could the wrapper scripts be used with bind mounts?
- Once such conda-type image is created, can it be modified dynamically afterwards to conda install extra packages for example? Or is it read-only?
- What if your tool in your container needs another CUDA version than your host, is that possible in any way? (A: ranges of compatibility between CUDA versions, depends on code)
- Is (re)building a container reproducible? (A: depends on the definition file, keeping/distributing the image is the most reproducible)
- How can a container or an App inside the container run across multiple nodes using SLURM? Should slurm be defined and slurmd be installed inside the sif file? (A: no Slurm required in containers, install MPI in container or bind-mount host MPI; perhaps also --container Slurm option)

## EasyBuild + Spack

- can you use spack on top of the installed modules, such that centrally installed modules (by EasyBuild, for example) are used when resolving dependencies? (A: yes, spack has support for external packages, _theoretically_)
- Is there a difference between using EasyBuild as a module and installing EasyBuild with pip? A: control, customization, but no specific advantage)

-------------------------------------------------------------------------------------------

## Sync meeting 20251023 (15:00)

- Catering OK
- @Sam: pass dietary restrictions
- registration hands-on session fo Thu 6 Nov
- UGent: coming soon
- VUB: open office style
- UA: ?
- KUL: link to registration form will be sent to participants
- on-site directions to room for training? => Sam
- will be available on site
- name of room? floor?
- access control at Belpaire building? => Sam
- no access control on visitors floor
- demo script for tblite: https://github.com/EESSI/eessi-demo/tree/main/tblite
- performance test for tblite?
- add demo scripts to https://github.com/vscentrum/gssi-training

## Sync meeting 20251017 (13:00-15:00)

attending: Kenneth, Steven, Maxime, Robin, Sam, Alex, Cintia

- review complete draft of presentation
- https://docs.google.com/presentation/d/1NSR0BDCOtn75IDv9mP5wFj20abrSWs3sB0AA0i20Pek
- clean up formatting
- @Alex/Cintia: on Fri 24 Oct 2025
- practical aspects
- send email with practical information to all registered attendees
- when?
- a couple of days before -> Wed 29 Oct'25?
- who takes care of it? => Alex
- Kenneth provides Excel sheet with registrations on 23 or 24 Oct to Alex & co
- mention hands-on session (incl. links with more info per site)
- UGent:
- UA:
- VUB:
- KUL:
- in-person vs online
- catering => Sam
- coffee at 9am
- lunch at 12:00
- coffee break at 2.30pm
- recording via Teams
- speaker in view via spare laptop => VUB
- with webcam => Sam
- backup with separate smartphone => Alex
- on-site directions to room for training? => Sam
- access control at Belpaire building? => Sam
- do we need to share list of in-person attendees up front?
- GitHub repo @ https://github.com/vscentrum/gssi-training
- changing laptops per speaker => because of hands-on
- share Teams on large screen
- let each speaker share their screen in Teams
- relies heavily on the WiFi
- hands-on session on Thu 6 Nov
- registration?
- @UGent:
- 2h slots (am + 2x pm)
- send invite to everyone from UGent who registered
- week before (Thu 23 or Fri 24 Oct)
- @VUB: without registration, only in afternoon
- @UA: unclear, TBD
- @KUL: probably with registration, TBD
- update on registrations (Kenneth)
- 23 in-person (9 seats left)
- KUL: 3 (incl. 1 via GMail)
- UA: 3 (+Franky)
- UGent: 11
- UHasselt: 1
- VUB: 3
- other: 1 (INBO)
- 55 virtual
- KUL: 13
- UA: 2
- UGent: 29
- UHasselt: 1
- VUB: 5
- other: 5
- VSC staff organising this training does not need to register
- 8/40 seats reserved for VSC staff
- one more sync meeting?
- Thu 23 Oct'25 15:00 CEST

## Sync meeting 20250926

attending: Kenneth, Maxime, Robin, Cintia, Lara, Alex

- review slides + feedback
- detailed agenda: https://hackmd.io/53rv-_D9STCwVGS4pm8VKg?view
- add "homework" slide with exercises for every section
- update on registrations (Kenneth)
- Lara can also access registrations in `event.ugent.be`
- VSC staff organising this training does not need to register
- 8 seats reserved for us
- 11 in-person (21 seats left)
- KUL: 1
- UA: 1 (+Franky)
- UGent: 6
- UHasselt: 0
- VUB: 2
- other: 0
- 36 virtual
- KUL: 4
- UA: 2
- UGent: 24
- UHasselt: 1
- VUB: 3
- other: 2
- practical aspects
- for Mon 3 Nov
- for hands-on session Thu 6 Nov
- registration (per site separately)
- 30min/1h time slots to let attendees select
- max 3-5 people per time slot?
- email with practical information [Alex]
- send on Thu/Fri 30-31 Oct
- details on venue, start, agenda, prerequisites (?), ask to inform us if they can't attend (in-person)
- for online attendees: Teams meeting invite
- **who can take care of this?** (Kenneth on leave)
- => Alex
- logistics
- make sure there's coffee at 9am (Sam)

## Sync meeting 20250828

attending: Kenneth, Lara, Sam, Alex, Cintia, Maxime, Steven, Robin

- date(s) & venue
- Mon 3 Nov'25 @ FWO (room booked via Sam)
- Belpaire building, room with 40 people max.
- (Sam) catering (coffee, lunch)
- hands-on session
- Thu 6 Nov @ UGent (Multimediazaal)
- (TBD) @ KUL [Steven not available Fri 7]
- Thu 6 Nov @ VUB
- (TBD) @ UA
- right after school holiday week (Herfstvakantie), not ideal in terms of (last-minute) preparation
- clash with EOSC Symposium (https://eosc.eu/eosc-symposium-2025/#programme)
- but program on Tue 4 Nov is more interesting than Mon 3 Nov, so not a big issue
- set date&venue in stone, announce + open registration?
- registration via event.ugent.be => OK (Kenneth/Lara)
- registration capped at 32 (to leave seats for VSC staff)
- info
- in-person vs online
- allow indicating interest for hands-on during registration
- VSC account
- name/affiliation/email
- goal for this training
- level of experience
- announce via mailing list of VSC sites + inform GJB+Rafal to add to VSC website
- detailed agenda being fleshed out in https://hackmd.io/53rv-_D9STCwVGS4pm8VKg?view
- "homework" assignment at end of each section
- example software package to use in hands-on demos?
- manual installation, container, Spack+EB, etc.
- should be quick, not a lot of dependencies, etc.
- STREAM? HPL? some Python package?
- tblite: https://pypi.org/project/tblite
- supported in Spack/EasyBuild/conda
- easy to "pip install" thanks to pre-built wheels in PyPI?
- next steps
- (Kenneth) open registration early Sept
- check with Rafal+GJB first
- start working on slides: https://docs.google.com/presentation/d/1NSR0BDCOtn75IDv9mP5wFj20abrSWs3sB0AA0i20Pek/edit?slide=id.g24a30fc37c1_10_0#slide=id.g24a30fc37c1_10_0
- Kenneth will flesh out structure
- next meetings:
- Fri 26 Sept'25 10:00-12:00 CEST
- (before meeting) step through prepared slides, give feedback
- discuss status of slides
- Fri 17 Oct'25 13:00-14:00 CEST
- finalise presentation + practical details

## Sync meeting 20250703

*(during VSC Central Software Installations meeting)*

- no room available at FWO for Mon 20 Oct
- organise presentation session at each VSC site?
- option 3
- find alternative location at one of the VSC sites?
- option 2: UAntwerp City Campus
- week of 20 Oct + 3/17 Nov
- ~40 people minimum
- option 1: find alternative date at FWO (big room, 40 people)
- week of 3-7 Nov (clash with KUL HPC intro)
- week of 17-21 Nov (SC'25 week)
- Sam to reply to FWO, someone else to follow up
- detailed agenda
- WIP via https://vscdocumentationbeta.readthedocs.io/compute/software/index.html + https://github.com/lexming/VscDocumentation/tree/software/source/compute/software
- preparing contents
- set up pairing session to prepare contents per session
- person assigned speaker is responsible for planning with sparring partner
- aim to have docs ready for overall review by mid Sept
- set up table of contents ASAP
- via https://hackmd.io/53rv-_D9STCwVGS4pm8VKg
- each speaker+sparring partner per section
- hands-on session
- optional, one at each VSC site
- short time slots, max 4 people per 30min slot
- make goal of this session clear: help with practical problems, give suggestions, push in right direction, hands-on help
- setting expectations will be important
- bring your own problem
- keep registration open until day before hands-on session
- require/recommend that people follow session on Monday first
- high-level question in registration form
- what software?
- which software installation mechanism are you using?
- what have you tried?
- which problems did you encounter?
- next meeting (on training session)
- Thu 28 Aug 09:00 CEST
- goal:
- table of contents + bullet-point slides per session
- have pairing session scheduled/ongoing

---

## Sync meeting 20250606

attending: Kenneth, Lara, Maxime, Robin, Sam

### Agenda

- see also notes of last VSC Software meeting: https://github.com/vscentrum/vsc-software-stack/blob/vsc/meeting-notes/2025/2025-05-08.md
- agree on dates
- best option: Mon 20 Oct + Thu 23 Oct 2025 (OK'ed by GJB)
- Mon 20 Oct (full day): presentations w/ remote participation
- large physical location: at FWO?
- => (Sam) check with Caroline if top room is free
- => (Kenneth) talk to Rafal for recording & technical support + promotion
- all speakers in same room?
- physical attendance of users
- max. attendance? 50/60? => Sam asks Caroline
- tables? (not required)
- Thu 23 Oct (half or full day): at each VSC site
- work with slots, organised per topic?
- 2-4 people to provide hands-on help
- max. 20 attendees per slot?
- let each VSC site decide on details of practical organisation, book room, etc.
- save the date
- via page @ vscentrum.be (via Rafal)
- direct contact with local users for hands-on session
- open registration mid Sept'25
- practical: set up Google Slides deck + structure for VSC docs
- https://docs.google.com/presentation/d/1NSR0BDCOtn75IDv9mP5wFj20abrSWs3sB0AA0i20Pek
- goal is to update https://docs.vscentrum.be/compute/software/index.html
- dedicated separate site w/ MkDocs? (no need to)
- flesh things out via https://vscdocumentationbeta.readthedocs.io/compute/software/index.html
- dedicated `software` branch in https://github.com/lexming/VscDocumentation
- MarkDown or RST, up to person who writes those pages
- Alex will add initial structure
- rework schedule (hands-on session on separate day)
- split up work to prepare sections
- focus
- taxonomy of tools: what's out there, how can they be used, pros/cons, use cases, required expertise, hands-on demos

### Draft schedule

#### 09:30-10:00 Welcome

#### 10:00-10:10 Introduction [speaker: Kenneth, sparring: Alex]

- general points: goal of the training, agenda, practical org, ...
- CPU vs GPU
- CPU-specific builds, $VSC_* environment variables
- use vs install vs compile (from source)
- use: central modules, EESSI, containers
- basic install: conda+mamba+pixi, language pkg mgrs (Python venv/R/Julia/Rust+cargo)
- expert install: manual build, build own container images, EasyBuild, Spack

#### 10:10-10:30 Central modules [speaker: Kenneth, sparring: Alex]

- basics of environment modules, Lmod, why we use them, ...
- incl. hands-on demo (5min)

#### 10:30-11:00 EESSI [speaker: Lara, sparring: Sam]

- VM in AWS with `sudo` for hands-on demo of native installation
- single-node without CernVM-FS: access via container vs cvmfsexec

#### 11:00-11:30 Containers (using) [speaker: Sam, sparring: Kenneth]

- performance, trust/security
- GPU use case

#### 11:30-12:00 Conda+mamba+pixi [speaker: Maxime, sparring: Alex]

- start creation of conda env first, then talk
- can't be combined with modules
- issue with reproducibility of conda envs

#### 12:00-13:00 lunch break

#### 13:00-13:30 Language package managers: Python [speaker: Steven, sparring: Robin]

- Python virtual environments, pip/venv/uv
- vsc-venv

#### 13:30-14:00 Language package managers: R/Julia/Rust/Java [speaker: Alex, sparring: Kenneth]

also R/Julia/etc. (15m + 15m hands-on)

- combo with modules
- simple Python/R/Julia example for hands-on exercise (pick one)

#### 14:00-14:30 Manual installation [speaker: Alex, sparring: Robin]

- `cmake` + `make` example
- constraints: which CPU you're targetting, link to OS, all manual steps (harder to reproduce)
- `build-env` modules
- compilers (GCC/LLVM/Intel/NVHPC/...), toolchains (MPI/BLAS/LAPACK/...), CMake, Makefile, environment ($PATH, ...)
- required dependencies
- from scratch vs on top of modules
- CPU-specific builds
- link to other VSC trainings

#### 14:30-15:00 Coffee break

#### 15:00 building containers [speaker: Sam, sparring: Robin]

#### 15:30 EasyBuild+Spack [speaker: Kenneth, sparring: Steven]

- combo with existing modules
- using existing an easyconfig

#### 16:00-16:15 Creating patch files ??? [Kenneth]

#### 16:15-17:00 Q&A

### Next meeting

- to be planned via VSC Slack

-------------------------------------------------------------------------------------------

## Kickoff meeting 20250130

- scope: overview of tools (conda, containers, Python venv, R, EasyBuild, Spack, EESSI), discuss the pros & cons of each, give some recommendations;

- hands-on hackathon: get your software installed in your VSC account;

- full-day?

- when?

### Notes

- lots of room with docs @ https://docs.vscentrum.be/software/index.html
- https://www.vscentrum.be/_files/ugd/5446c2_ce37458f4a1b438d8ab1d8269350b680.pdf as a base
- extra levels: EESSI, containers, using EasyBuild, ...
- try to find common recommendations across VSC sites
- w.r.t. conda recommended usage
- making EESSI available
- clearly highlight stuff that's specific to VSC systems

### Agenda

#### 10:00-00:15 General points (15m talk)

#### 10:15-00:45 Central modules (10m talk + 20m hands-on)

#### 10:45-01:15 Using EESSI (15m talk + 15m hands-on)

- access via container/cvmfsexec

#### 11:15-01:30 coffee break

#### 11:30-02:00 Using existing container images (15m + 15m)

- performance, trust/security
- GPU use case

#### 12:00-12:30 lunch break

#### 13:00-13:00 Conda (15m + 15m) [Alex]

- start creation of conda env first, then talk
- can't be combined with modules
- issue with reproducibility of conda envs

#### 13:00-13:30 Python virtual environments, also R/Julia/etc. (15m + 15m hands-on)

- combo with modules
- simple Python/R/Julia example for hands-on exercise (pick one)

#### 13:30-13:45 EasyBuild (talk + hands-on demo)

- combo with existing modules
- using existing an easyconfig

#### 13:45-14:00 Manual installation (talk + hands-on demo)

- `cmake` + `make` example
- constraints: which CPU you're targetting, link to OS, all manual steps (harder to reproduce)
- `build-env` modules

#### 14:00-14:15 Creating container images (talk + hands-on demo)

#### 14:15-14:30 Native installation of EESSI (talk + hands-on demo)

- VM in AWS with `sudo` for hands-on demo

#### 14:30-14:45 Creating patch files

#### 14:45-15:00 coffee break

#### 15:00-18:00 hands-on session: bring your own software! (3h)

### Next steps

- set up HackMD pages for each of the section + assign to someone
- general example: something on top of PyTorch
- target: early Sept'25?
- next sync meeting: week of 24-28 Feb?
- Mon 24 Feb'25
- Wed 26 Feb'25