# TTT4HPC 07/05/2024 ARCHIVE DOCUMENT ## Tuesday Tools & Techniques for High Performance Computing - Episode 3 :: Containers on HPC clusters :::danger ## Infos and important links - This is the archive for episode 3 of TTT4HPC - Program for the day: https://coderefinery.github.io/ttt4hpc_containers/ - Materials: https://coderefinery.github.io/ttt4hpc_containers/ ::: *Please do not edit above this* # Tuesday Tools & Techniques for High Performance Computing - Episode 3 :: Containers on HPC clusters ## Icebreakers ### 1. How many times have you seen snow since last summer? - 00 - Too many :D - Twice: Once covering all winter, and once two weeks ago! ### 2. Where are you watching from and how many people are watching with you? - Helsinki, Finland (Aalto!) - Bergen, Norway - Delft, The Netherlands - Geneva, Switzerland - Espoo, Finland - Jyväskylä, Finland - Ioannina, Greece - Modena, Italy - Lund, Sweden - one more Lund, Sweden - Stockholm, Sweden - Gothenburg, Sweden ### 3. Which cluster are you using? - Triton (Aalto) - Saga (NRIS) - DelftBlue - Leonardo - Tetralith - UPPMAX - Cosmos (LUNARC) - Alvis - CSC ### 4. What's the most annoying software you've used? - awk - 👑 vim - emacs - MS Word +1 - notepad - VSCode ### 5. Have you used containers before (vote by adding o)? - Yes: oooooooo - No: oooooooo ## Intro to containers https://coderefinery.github.io/ttt4hpc_containers/intro_and_motivation/ - Should we make software more easy to install instead of using containers? - When you have apps with conflicting requirements for Python or Java, it's easier to maintain each one independently as containers than trying to install and maintain multiple parallel versions. - What about virtual machines? And once there was something called "chroot", are containers in the same spirit of that somehow? - The VM creates virtual hardware where it will run on. There is more strict difference between the physical resources and the virtual machine. - What is the typical overhead for building the guest system at runtime? - I think it's usually quite low: this is all built into the Linux kernel and basically is only a different view of the system resources. - Memory requirement is usually few MB and there's practically no extra CPU overhead. - Do containers exist natively on other operating systems (other than Linux)? - You can run both Linux and Windows programs and executables in Docker containers. The Docker platform runs natively on Linux (on x86-64, ARM and many other CPU architectures) and on Windows (x86-64). - What do you mean by storing images using Apptainer? Is this a way around using volumes/mounts? - "container image" is the file that contains the containerized OS. (but yes, Apptainer can mount files as volumes, but I think that wasn't the point they ment to make right now) - Are Apptainer containers lightweight, or are Apptainter containers "normal" and Docker etc heavyweight? - Edit - yes, except maybe the building process. ~~Not sure if they are more lightweight, but the way they run is better suited for HPC as they typically run as a normal user, where docker typically runs as root user (privileged). You will hear more about some differences later today. It depends on what one defines as lightweight also - as building can be heavier. But yes, running/using/shipping apptainer is more lightweight than docker. ~~ - In a way yes: more options available for docker containers, but you need higher privilegdes (e.g. you need to be the root / admin of the machine where the container is running). This means also more overhead. - Advantages compared to virtual environments (e.g. requirements.txt)? - Containers also have operating system libraries in them. So stuff that needs things that are *not* expressible in requirements.txt (or environment.yml), then you need containers. (example: needs a certain version of libc, or some OS library) - Containers also exist as one file, so filesystem i/o performance, and moving them around without re-installing, is better. - If we created an app with a corresponding apptainer image but wanted to update a single component/layer of it, is the standard to fully recreate a new apptainer image for the updated version? - Usually yes. You can create writable container images, but that will then reduce the reproducibility aspect of the technology. If you're developing an application (e.g. Python code), you can also bring your application into the container and run the application with the interpreter (e.g. python) from within the container. We'll talk about these when we get to mount bindings. - Do I have to request memory for running the entire guest OS when I submit something container-based as a slurm job? - The container will run in the same memory request and it is very lightweight. The guest OS is mainly used to access libraries so no OS services such as desktop services etc. are started. Memory overhead to running non-containerized applications is ususally few MB. - Is this true also for docker images? - What is a namespace? - You can think of it as a "enclosed" environment - or an environment which belongs to a certain namespace and not another, all objects (files etc) that belong to the namespace can be reached within that namespace. Other objects might be hidden or masked by this namespace. A use-case is using a namespace to load the correct python version. https://en.wikipedia.org/wiki/Namespace :::success Question for everyone following: "What is your (potential) use case of Apptainer?" - software that *only* works on Ubuntu but has to run on the cluster - running old versions of OpenFOAM on a cluster - trying out software to decide whether its work the pain of installing it on HPC - ::: ## Basics of running containers https://coderefinery.github.io/ttt4hpc_containers/basics_running_containers/ - Amazingly well prepared lesson materials! +1 ❣️ ## Intro to container images https://coderefinery.github.io/ttt4hpc_containers/container_images/ - . ## Building container images (on HPC) https://coderefinery.github.io/ttt4hpc_containers/building_images/ - How do I check what is the cache folder? In my node $APPTAINER_CACHEDIR is empty: - Seems like the default location if that isn't set is `$HOME/.apptainer/cache` (https://apptainer.org/docs/user/main/build_env.html) - Does the definition file has some specific naming rules? I.e. simply: how should I name it? - It has to have `.def`name extension I think. It would be good to have a name that reflect the image - Yeah, unlike docker the name usually reflects the final image name. - HPC environmental modules are typically implemented as containers. What is the recommended way to install additional packages when using a pre-installed container? - ^-- Please discuss on stream - You can mount in extra directories inside the container, so basicalyl add more files into it. This isn't necessarily hard but is advanced and requires a good understanding of how the software works. - From the stream: also use an existing container as a base and build a new container containing the extra files. - In the system I'm running, I get an error: `FATAL: could not use fakeroot: mapping entries for user NNN found in /etc/subuid but all with a range count lower than 65536`. Is this something common? If not, never mind :-) - This is something you should ask the cluster admins about for help: Apptainer/etc might not be set up properly. - It's related to the setup and how it would manage permissions. Understanding it probably beyond the scope of this course, but at least know it shouldn't be your fault. - What system is it? ## Binding folders into your container https://coderefinery.github.io/ttt4hpc_containers/binding_folders/ - On my computer `apptainer exec python.sif pwd` shows my home directory, not the CWD. Is it an installation/configuration issue? (I am not in my home dir) - Same for me. - Hm. I'm not sure, my guess is a configuration issue, or maybe a different version. (this is new to be so I have no idea) Anyone else have a better idea? - . - . ## Feedback :::info - The next lesson is next week, about parallization - Exercise session in Zoom (registered participants) in one hour. - Video will be online by later tonight ::: Today was: - too fast: - too slow: - right speed:ooooooo - too advanced: - too basic: - right level:ooooo - Overall rating of today (number 0-9): One good thing about today: - excellent explantation about differences btw docker (et al) and apptainer +1 - nice to have interactivity - Excellent material, e.g. upsides/downside, mount binding figure, etc. +1 - great to have questions answered and comments added by instructores during the lecture simultaneously +1 One thing to be improved for next time: - Loved the Kleenex analogy, maybe include this earlier in the material? - . - . Other comments: - Just amazingly well done! - Very well done, appreciated the prompts and explanations regarding points that may cause confusion for the audience! - .