# HPC course notes - Day 4 ## Presentation material **Docker Training GitHub**: https://github.com/abdulrahmanazab/docker-training-neic ## Questions - In the Matlab + R example: If there are shared dependencies, can they be shared and thus would you get a smaller size than 300 MiB? - Could you comment on the difference between an image, a Dockerfile and a layer? - source : https://phoenixnap.com/kb/docker-image-vs-container - A Docker image is an immutable (unchangeable) file that contains the source code, libraries, dependencies, tools, and other files needed for an application to run. This you should treat read only and not supposed to modify - Container is a virtualized run-time environment based on the image - The Dockerfile is the recipe for creating the image, basically a text file with commands - I have found this useful: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008316 - I have heard someone say "containers" about .yml files with Python instructions/versions. Is this not technically correct? - you mean like `environment.yml` files in Conda environments? or is this something else? - Yes! - OK I would not call this a container but I would call it a recipe to create an environment. It provides some isolation but not the same isolation as a container. A container is an isolated operating system whereas `environment.yml` creates an envionment in the same operating system. This can also mean that precisely the same `environment.yml` can produce slightly different results on different machines if it links to system libraries "outside" of that environment. - Also the "clean way" looks like an imperative way to build an image. Is it possible to do it declaritively? - Using a Dockerfile, you'd get the newest software, right? Since you don't specify versions, this approach seems to give unpredictable results, so the exact docker image won't be reproducible. Is that correct? - On the other hand, the "dirty way" would give something static, and thus reproducible but not really transparent (not known how it was created). - If possible, could you show a simple implementation how to build or use the Docker to benefit for data analyzing? - In order to register at training.play-with-docker i need to respond to a verfication email. I get no such email. I have looked into all spam folders. I am stuck here... OK got the email now, but this took a loooong time.... - I was able to logon without confirming from an e-mail. - Why does sigma2 use quay.io instead of docker hub as a default for publishing the images? - I find quay has faster builds (in automatic builds when coupled with a recipe file stored on GitHub) - Can we run out of memory on our computer if we run a lot of containers? - I think so but I also think that it is not typical to run several containers at the same time but rather to run one at a time but to have lots of containers/images on the disk. So a more typical issue is to fill up the disk but there are cleanup commands to get rid of old/unused layers/images. - How do we 'cleanup' properly after a session? - For me this worked: `docker image rm -f ourfiglet`, but maybe there are more subtle ways. - Can you please show an example on SAGA or FRAM. Do you have singularity on LUMI ? - Why do we need these virtual machines for the exercices I mean. - Mee too, what is the point of this demo ? - Do we have to install Singularity ourselves on SAGA before we use it, I do not see it as a module ? - Info here https://documentation.sigma2.no/software/containers.html?highlight=singularity - Thank you. This would have been more usefull, VM is just a waste of time - Saga it is not working what you are showing, can you please show atleast the pull step on saga - How do I access project directory from container - Why would we like to compile something on saga in a contrainer? The point with having all the modules precompiled on Saga is that the developer does not have to compile herself? Is the motivation reprodusability?