Using CSC HPC environment efficiently, Part 2: Next steps 7.-8.11.2024

tags: puhti mahti allas

This is the collaborative "notebook" for the "Using CSC HPC environment efficiently, Part 2: Next steps" course organised online in Novemver 2024 by CSC -IT center for Science.

LINKS

Prerequisite course page in eLena platform
Course page in eLena platform
Collection of slides and material
Zoom-link for COURSE sessions

Table of contents:

• Using CSC HPC environment efficiently, Part 2: Next steps 7.-8.11.2024
  • How we work
    • General Zoom instructions
    • Code of conduct
  • 📅 Agenda
    • Prerequisite / support sessions
    • Day 1: Thursday 7.11.
    • Day 2: Friday 8.11.
  • ☃️ ICE BREAKER (HackMD -practice)
    • Ice breakers
  • 📝 Q & A
  • ✏️ Add your questions here
  • Feedback day 1
  • Feedback day 2
  • Course feedback
  • Useful links

How we work

General Zoom instructions

• Link to the zoom room was sent to you via e-mail, and can be found in the top of this page
• Please arrive 5-10 minutes before to test your microphone setup.
• Use your full name (Firstname Lastname)!
  
  
• During the course, please remember to always mute your microphone when you are not speaking.
• A headset or other external microphone is preferred in order to avoid echo (a simple phone headset is just fine).
• You can find all the controls (mic, video, chat, screen sharing) at the bottom of the Zoom window (when you bring your mouse there).
• Note: for questions and answers about the course topics, we will be using this living document
• Break-out rooms: More info:https://support.zoom.us/hc/en-us/articles/115005769646-Participating-in-Breakout-Rooms#collapseWeb Breakout rooms are smaller sessions that are split off from the main Zoom meeting. They are completely isolated in terms of audio and video. When the rooms are opened, you can click "Join" button over the chosen breakout room. Mouse over the number of people in the breakout room to reveal the "Join" button.
  
• Emojis & other indicators: First row emojis can be used to express your feelings :) They will disappear automatically after a while. Second row icons are used to indicate whether you are done with the practicals (green symbol), still need some time to work with the practicals (red symbol), or wish us to slow down or speed up, or whether you are having coffee. These you need to turn off yourself. To indicate that you have a question, you can use the "raise hand" button.

Code of conduct

We strive to follow the Code of Conduct developed by The Carpentries organisation to foster a welcoming environment for everyone. In short:

• Use welcoming and inclusive language
• Be respectful of different viewpoints and experiences
• Gracefully accept constructive criticism
• Focus on what is best for the community
• Show courtesy and respect towards other community members

📅 Agenda

Prerequisite / support sessions

On Wednesdays 14:00-15:00, feel free to join the CSC's weekly user meetings to get extra help and discuss with us, also about the course matters!

Day 1: Thursday 7.11.

Time	Content
8:45	Login, check connections work, test microphone
9:00	Welcome, course practical info (15 min)
9:15	Topic 8: Working efficiently with data (25 min)
9:40	Intro to breakout rooms & hosts, joining the breakout rooms, accessing materials (5 min)
9:45	Topic 8: Hands on 8.1 (15 + 15 min)
10:00	Break (10 min)
10:10	Topic 8: Hands on 8.1 (15 + 15 min) continued
10:25	Going through the hands-on(s) together (10 min)
10:35	Topic 9: Installing applications (25 min)
11:00	Break (10 min)
11:10	Topic 9: Hands on 9.1 and 9.2 (30 min)
11:40	Going through the hands-on(s) together (15 min)
11:55	Recap + preparations for the next day (5 min)
12:00	Finish

Day 2: Friday 8.11.

Time	Content
8:45	Channel open, "morning coffee"
9:00	Re-cap, feedback/expectations + some Qs from HedgeDoc (5 min)
9:05	Topic 10: Containers & Apptainer (25 min)
9:30	Topic 10: Hands on 10.1 (30 min)
10:05	Break (10 min)
10:15	Going through the hands-on(s) together (15 min)
10:30	Topic 11: How to speed up jobs (30 min)
11:00	Break (10 min)
11:10	Topic 11: Hands ons 11.1 (35 min)
11:40	Going through the hands-on(s) together (15 min)
11:55	Recap (5 min)
12:00	Finish

---

☃️ ICE BREAKER (HackMD -practice)

Ice breakers

Let's learn how to use this HackMD document by answering an ice breaker question!

Q1: What topic of this course are you most excited about?

Answers:

• the ones related to Allas and containers
• type here
• ..
• ..

📝 Q & A

Your questions are answered here. We will answer them, and this document will store the answers for you for later use!

Scroll

to the bottom of the page to submit a question

✏️ Add your questions here

Please type your questions ON THE BOTTOM OF THIS SECTION. We will answer them, and organise the document topically.

• Q0: I have difficulty pasting my questions into HackMD (here). Do you have some instructions on how to write here?

  • A: HackMD can look a bit differend depending on the view you are in. Can you see the three icons on top left corner, next to HackMD text? There’s pencil, this side-by-side symbol, and an eye (see below for a screenshot). In eye view, you can’t edit, you are just viewing. The other two reveal the markdown (MD) version of the page, which you can edit. I find it easiest to edit with the side-by-side view, but it can be a bit slow sometimes. First time opening the page, the small Edit (pencil) button might be on right next to the table of contents. Please note that it might make things faster if you switch to view only mode when not editing!

The bigger HedgeDoc/HackMD toolbar for switching between modes later on.

Before editing for the first time, you might find the pencil on the right side:

Hint: You can also apply styling from the toolbar at the top of the editing area.

• Q1: Slides available after the course?

  • A: They sure are! You will also have the access to this HedgeDoc document.
  • We encourage you to share and use the material also in your own courses. The material is in git, and we welcome all feedback and edit suggestions (pull requests)!

• Q2: Has the Zoom meeting already started? I'm just gettin "The meeting has not started" -note.

  • A: Make sure you have the full Zoom link when connecting! It's looong.

• Q3: Should we be able to access the slides in e-elena? I can only see the first slide.

  • A: Try navigating with the arrow keys :)

• Q4: So many credendials… Which ones do I need and where?!?

  • A: You need
    • CSC credentials to Puhti/Mahti/Allas
    • Haka OR Virtu credentials (=your university/institution credentials) to my.csc.fi and eLena platform
    • contact us if you don't have HAKA/Virtu in use

• Q5: I have forgotten my CSC username and password

  • A: In my.csc.fi you can check your CSC username. If you have forgotten your CSC password, check this link: https://docs.csc.fi/accounts/how-to-change-password/ -Note that it takes some time for the password to update to Puhti, if you need to change that :)

• Q6: Depending on the place you say to obtain acces to Puhti and Allas or Puhti and Mahti, should we have all three of them (just in case)?

  • A: On the course there is one tutorial in which you can try moving data from Puhti to Mahti. It is an optional task so you don't necessarily need Mahti.

• Q7: Do you record the sessions?

  • A: No. However, the lecture videos, slides and hands-ons are available in the github page, and all the material + the quizzes etc can be accessed in eLena platform, so this course works well as self-study also!

• Q8: Why are there so many different sites for the same information?

  • A: We are using the same material in several ways.
  • If you just wish to participate in the live online course, it's easiest to follow the schedule here (above) and the slides and exercises in the github page. This material is also used by our colleagues and collaborators, and github provides a nice way to suggest edits!
  • If you wish to test your learning and perhaps get a course certificate, do also the quizzes in eLena platform, where you can also download the certificate. The e-Lena platform is also used when taking the self-learning course.
  • This HackMD document is used for all discussion related to this specific live course. It also allows us to scale up, as multiple questions can be asked and answered simultaneosly, and everyone can learn from them. Some people also find it easier to ask questions anonymously in text format, and we of course want to make that possible!

• Q9: Maybe not related to this particular training, but which are the best ways to transfer files to and from Allas? Prefereably easy ways, since not all the users are "computer-persons".

  • A: The easiest way for non "computer savvy" persons is probably to use the graphical user interface in the HPC web interfaces we offer, e.g. www.puhti.csc.fi. See https://docs.csc.fi/computing/webinterface/file-browser/
  • A: On the command-line, using the a-tools is easier to use than e.g. rclone. See https://docs.csc.fi/data/Allas/using_allas/a_commands/

• I have trouble finding these first slides from linked website.

  • A: They are all linked here https://csc-training.github.io/csc-env-eff/part-2/data-io/

• I use Puhti. Data size is 1GB. How should I input that data for efficient GPU usage? Local scratch, /scratch, allas, what else to take into notice?

  • A: Data to be used in computing workloads should be transferred first to /scratch. Then depending on the nature of the data and workload (a lot of small files or few big ones, a lot of IO being done or not) it is ok to use /scratch or then local scratch. So short answer it depends.

• Q10: What is the practical difference of using a-conmand, rclone and s3cmd?

  • A: a-command and rclone can be used in swift and s3 protocols. S3cmd command uses s3 protocol. a-commands are bit safer to use and will not overrides the objects in allas. Rclone may not ask before overriding it. Please read more about different allas clients here: https://docs.csc.fi/data/Allas/using_allas/

• Q11: If I were to use th $LOCAL_SCRATCH in my computing nodes, what is actually more efficent, transfering from scratch or from allas object?

  • A: If the data is available in both, copying from /scratch is faster. If the data is in Allas only it's best to copy it directly to $LOCAL_SCRATCH.

• Q12: I have recently used allas-backup. Some time ago I create a backup of one folder. Yesterday, I used allas-backup folder again, but it seems it does the entire backup again. What is the command to do the incremental copy?

  • A: allas-backup automatically creates a project-specific backup repository in the Allas storage service at CSC and uses it for cumulative backups. – different versions of a dataset can be stored so that in the case of a new dataset version, only the changes copared to the previous version needs to be stored. https://docs.csc.fi/data/Allas/using_allas/a_backup/
  • More on how that happens: https://restic.readthedocs.io/en/stable/040_backup.html#file-change-detection

• Q13: Can allas-backup be used from ePouta and cPouta?

  • A: I am not sure about ePouta, but cPouta works as long as you install the allas-cli-utils developed by CSC in your VM in cPouta. The repo is here: https://github.com/CSCfi/allas-cli-utils/tree/master and installation instructions: https://github.com/CSCfi/allas-cli-utils/tree/master?tab=readme-ov-file#installing-allas-cli-utils
  • A: Comment: ePouta instances live in a separate network and may not have access to allas which is on the public network. As epouta service is a self-service model, you can obviously relax the firewalls or netwroks at your own risk. CSC has no control on it. In recent SD environment (SD Desktop), you can only access encrypted data on allas.

• Q14: Can tha Allas commands be used in the exact same way from Mahti as from Puhti?

  • A: Yes. One can use a-commands in the same way, both on Mahti and Puhti.

• Q15: As a general question, what is the difference between wclean libso / wmake libso and wclean / wmake ?

  • A: I guess this is some custom maketool for OpenFOAM? Seems like it is a wrapper script based on regular make. I am not an expert on OpenFOAM, but there is documentation available here: https://doc.cfd.direct/openfoam/user-guide-v12/compiling-applications
  • A: From the docs: "The wmake script is generally executed by typing wmake <optionalDirectory>. The <optionalDirectory> is the directory path of the application that is being compiled. Typically, wmake is executed from within the directory of the application being compiled, in which case <optionalDirectory> can be omitted."
  • A: Furthermore: "wmake builds a dependency list file with a .dep file extension, e.g. newApp.C.dep in our example, in a $WM_OPTIONS sub-directory of the Make directory, e.g. Make/linuxGccDPInt64Opt. If the user wishes to remove these files, e.g. after making code changes, the user can run the wclean script by typing: wclean <optionalDirectory>. Again, the <optionalDirectory> is a path to the directory of the application that is being compiled. Typically, wclean is executed from within the directory of the application, in which case the path can be omitted."
  • A: Note that OpenFOAM is already available on Puhti, Mahti and LUMI, so you do not need to install it yourself unless you specifically want to. See https://docs.csc.fi/apps/openfoam/

• Q16:conda-containerize new --prefix /projappl/<project>/$USER/tykky-env env.yml # replace <project> with your CSC project, e.g. project_2001234 should this be deleted after?

  • A: Yes. you can delete the directory. It will not be needed on the course.

• Q17: Do we need to add the path of our tykky container to the environmental variable path only once?

  • A: The export command will only be effective for the current session. If you for example log out and log back in, all changes will be lost. If you wish to make some change permanent, you can e.g. add the export comamnd to your $HOME/.bashrc file. It should be noted that doing so may cause conflicts with other software, so it's not generally recommended. This is especially true for Tykky installed software. It's best to have only one such software in your $PATH at any one time, as they often conflict with each other. So only add them to your $PATH when using them. You can e.g. add the export command to your batch job script.

• Q18: Im interested to get my some of my codes from puhti to be stored in private gitlab repo. What should I do? I am familiar with ssh keys but I am not sure how it goes between supercomputer and gitlab?

  • A: Git is available on Puhti, so pushing code to a private gitlab repo should be no issue. To use SSH keys to authenticate, you should create a key pair on Puhti and register the public key in your gitlab. Please remember to keep your private key safe (don't change its permissions, it should be -rw-------, i.e. 600 in numeric form)!
  • Note: git is available directly in the login nodes. If working in a compute node (e.g. interactive session) you need to load it as a module module load git.

• Q19: Having a "database" consisting of 100k directories, each containing about 10 files, what would be the recommended way of storing these?

  • A: If you are working with flatfiles in database and the need for queries is very less, you might use it under Lustre (generally not advised). if the need grows, the better approach would be is to use $LOCAL_SCRATCH. Use of database in $LOCAL_SCRATCH will not cause any issues on parallel file system. It just requires that you have to stage and unstage it from local_scratch, otherwise database will be lost. However, if there is a great need for querying those files from database and you need this database more frequently, the recommendation is to use this database on cPouta/Rahti and query it from there.
  • Follow-up Q: Do I understand correctly, during use, I stage the data to $LOCAL_SCRATCH and access it from there. When not in use, I keep it stored on Mahti/Puhti normal scratch?
  • A: Yes. you can store in normal scratch if you are activley using it. Otherwise, it makes sense to move the database to ALLAS

• Q20: Would you recommend using Apptainer (vs docker) also in cPouta/ePouta?

  • A: Both work fine in cPouta/ePouta environment. If the VM instance is shared by many users, I would go with Apptainers. Just FYI, Apptainer is more suitable for scientific applications on HPC systems. Docker is more for microservices in more isolated environments.

• Q21: Are there any geographic restrictions in accessing CSC services? In other words can I work with all of these services from abroad?

  • A: Generally, there are no restrictions as such. But there can be some restrictions due to some state regulations from time to time (https://csc.fi/en/news/csc-prohibits-the-use-of-its-services-based-on-the-regulations-of-the-european-union/). Also note that CSC does lot of services of which some of them may be restricted (sensitive data or similar services) worldwide for security reasons (e.g., https://docs.csc.fi/cloud/rahti/security-guide/).

---

Write your questions above this line

---

Feedback day 1

One thing you liked:

Feedback day 2

One thing you liked:

One thing that could be improved:

• ..
• ..

Course feedback

Please give us also "official" feedback at: https://ssl.eventilla.com/feedback/101HdmQNk2504Vk
(This will be used to develop our courses further. Note, that if you need the course certificate, please access this same form through the e-learning platform :) )

Useful links

• Elements of Supercomputing online course
• Training links by us and our friends
• Weekly research user meetings (come and ask anything!)
• Data management self-learning course
• Sensitive Data (SD) services webpage
• Sensitive Data (SD) services user guide