# CSC Computing environment Q&A session 26.4.2022

###### tags: `puhti` `mahti` `allas`

> This is the common "notebook" for the CSC Computing Services Q&A session" webinar organised in April 2022 at CSC -IT center for Science. [Course page in e-Lena platform](https://e-learn.csc.fi/course/view.php?id=76)
>

:::danger
This is the place to ask questions about the workshop content! We use the Zoom chat only for posting links, reporting Zoom problems and such.
:::

:::info
:bulb: **Hint:** HackMD is great for sharing information in this kind of courses, as the code formatting is nice & easy with MarkDown! Just add 3 ticks (``` ` ```) for the``` code blocks ```. Otherwise, it's like a Google doc: it allows simultaneous editing.
:::

[ToC]

## Code of conduct

We strive to follow the [Code of Conduct developed by The Carpentries organisation](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html) to foster a welcoming environment for everyone. In short:
- Use welcoming and inclusive language
- Be respectful of different viewpoints and experiences
- Gracefully accept constructive criticism
- Focus on what is best for the community
- Show courtesy and respect towards other community members

---

## TO DO before the course
- Please make sure you have a CSC user account, project with Puhti and Allas as services, and have accepted the terms of use for Puhti and Allas in my.cs

## :memo: Q & A

You can type your questions here. We will answer them, and this document will store the answers for you for later use! :rocket:

### General and practical matters

- [x] **Q1: I have difficulty pasting my questions into HackMD (here). Do you have some instructions on how to write here?**

- A: Can you see these three icons on top left corner, next to HackMD text? There’s pencil, this side-by-side symbol, and an eye. In eye view, you can’t edit, you are just viewing. The other two reveal the markdown (MD) version of the page, which you can edit. I find it easiest to edit with the side-by-side view.
:::info
:bulb: **Hint:** You can also apply styling from the toolbar at the top :arrow_upper_left: of the editing area.
:::

- [x] **Q2: I cannot access most of the slides from the course page. These are marked with an earth logo.**
- A: Try to use the arrow keys or space bar to switch the slide!

- [x] **Q5: Do we get ECTS credits from this course for our university transcripts?**
- A: After you have completed the course including all the quizzes, you can download a certificate where we recommend 1 credit for this course. We are not allowed to give credits, but we can recommend them, and by taking the certificate to your university, they usually accept :)

- [x] **Q6: Should I get my "own" project for the course, can't I just use a project you provide?**
- A: The course project will be available only during the course, and before and after that you need to do the exercises on your own projects anyway :) Also, we wish you to help you get past this first obstacle of setting up everything in the beginning! So yeah, better get your own project!

- [x] **Q7: So many credendials... Which ones do I need and where?!?**
- A: You need CSC credentials to login to Puhti, whereas for my.csc.fi and e-Lena you can nowadays login also with Haka credentials (=your university credentials). In my.csc.fi you can check the CSC username. If you have forgotten your password, check this link: https://docs.csc.fi/accounts/how-to-change-password/ -Note that it takes some time for the password to update to Puhti, if you need to change that :)

- [x] **Q: How long are the course materials available online? Tutorials, slides etc.?**
- Tutorials and slides will be there indefinitely (https://a3s.fi/CSC_training/README.html and https://csc-training.github.io/csc-env-eff/). Access to elearn (and the quizzes there) may discontinue. The main material will thus be available.

- [ ] **Q: To whom (what level student) should the CSC services be recommended?**
- Well, the level is not so critical but the need. Already now it's possible to set up an account for yourself (as a teacher/lecturer) and ask for training accounts for the participants (available for the duration of the course). To access the supercomputers the participant needs to be a member of a CSC project and setting up a project needs a Project manager with some requirements (a bit longer term position in a University). If you're interested send email to [e-mail]. A new setup is also in the pipeline to enable "student accounts" which don't need a "supervisor" which will make this easier.

- [ ] **Q: Tykky container service. How to install python / local packages?**
- https://docs.csc.fi/apps/python/#tykky
- https://docs.csc.fi/computing/containers/tykky/

- [x] **Q: I registered as a student but I can't found COMSOL multiphysics software. are available softwares are limited for student?**
- No such limit. Where did you look for comsol? How?

- [ ] **Q: How to access huge data (very big! :) from ALLAS to work with Matlab from cPouta? Are there examples online (I couldn't find them)?**
- It's possible to mount Allas to cPouta, but not sure if it would more wise to copy the data locally and work with that. This is a good use case! We are looking for documentation! https://docs.csc.fi/data/moving/disk_mount/
- https://docs.csc.fi/support/faq/how-to-mount-os-bucket-for-cpouta/#configuring-and-using-allas-ubuntu-and-centos

- [ ] **Q: Hi, I have a basic question. I used to get email notif when my batch job status is failed. But now I only get notified when it successfully ran. Is there some changes in csc with this?**

#!/bin/bash
#SBATCH --job-name=Jacque
#SBATCH --account=project_2000967
#SBATCH --time=06:00:00
#SBATCH --ntasks=1
#SBATCH --nodes=1
#SBATCH --output=Jacque
#SBATCH --error=Jacque-err
#SBATCH --mem=64G
#SBATCH --partition=small
#SBATCH --gres=nvme:300
#SBATCH --cpus-per-task=8
#SBATCH --mail-type=END
#SBATCH --mail-user=[e-mail]

- Great! You're asking `--mail-type=END` i.e. send email when the job ends. If you want mail also when it starts --> `--mail-type=BEGIN,END`. If you also want notification when job fails, also specify `FAIL` --> `--mail-type=BEGIN,END,FAIL`

- [ ] **Q: I am trying to run a batch job, basically doing a diamond blast to one database (non-redundant, nr). Previously I do it with swiss-prot, which took around 16min/sample and between 8-10G of memory. But with nr database the job is cancelled after 10h because exceeds the maximum memory (16G), which cannot be increased. I checked and any output file of the 40 desired was created before the job was cancelled. How can I run the job? How can I by-pass the memory limitation? I send this question by email hpcplatforms #538239**

#!/bin/bash
#SBATCH --job-name=myTestdiamond2
#SBATCH --account=project_2005233
#SBATCH -o output_%j.txt
#SBATCH -e errors_%j.txt
#SBATCH --time=15:00:00
#SBATCH --mem-per-cpu=16G
#SBATCH --partition=small
##SBATCH --mail-type=BEGIN #uncomment to enable mail

- Try sinfo command to learn about partitions, and about max mem
- Check chapters 5 and 6 from the self-learning course, there we are talking about this! It's a common difficulty!
- A good tutorial, where the memory runs out, and how to figure that out: https://csc-training.github.io/csc-env-eff/hands-on/batch_resources/tutorial_sacct_and_seff.html
- How much memory is available? List of Puhti's partitions (queues) https://docs.csc.fi/computing/running/batch-job-partitions/
- Try: ```#SBATCH --mem-per-cpu=32G```
- Don't use max mem, if you don't need that much -the "why" for this is explained in the training material

- [ ] **Q: Does the '#SBATCH --time' starts when the job starts or when you submit the batch job?**
- When the job actually starts. So that's the limit of the actual time of running the job, not the queuing time

- [ ] **Q:**

Action points:
- add a quiz/FAQ for how much memory I can ask / is available?
-