# BioMonth 2021

###### tags: `bioinformatics`

> This is the common "notebook" for the "BioMonth" organised in March 2021 at CSC -IT center for Science. [Course page in e-Lena platform](https://e-learn.csc.fi/course/view.php?id=57)
>

:::danger
This is the place to ask questions about the workshop content! We use the Zoom chat only for posting links, reporting Zoom problems and such.
:::

:::info
:bulb: **Hint:** HackMD is great for sharing information in this kind of courses, as the code formatting is nice & easy with MarkDown! Just add 3 ticks (``` ` ```) for the``` code blocks ```. Otherwise, it's like a Google doc: it allows simultaneous editing.
:::

[ToC]

## Code of conduct

We strive to follow the [Code of Conduct developed by The Carpentries organisation](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html) to foster a welcoming environment for everyone. In short:
- Use welcoming and inclusive language
- Be respectful of different viewpoints and experiences
- Gracefully accept constructive criticism
- Focus on what is best for the community
- Show courtesy and respect towards other community members

---

## TO DO before the course
- Please make sure you have accepted the terms of use for Puhti and Allas in my.csc.fi for the course project (2002389, "BioMonth Puhti Allas")! 
- Check & test that you have a functional terminal connection (https://docs.csc.fi/computing/connecting/, graphical connection not needed) 
- Login to e-Lena course page (https://e-learn.csc.fi/course/view.php?id=57) and self-register to the course (registration key was sent to you via e-mail)
- Check the Zoom instructions below

## Zoom instructions
- Link to the zoom room was sent to you via e-mail
- Please arrive 5-10 minutes before to test your microphone setup.
- During the course, please remember to always mute your microphone when you are not speaking.
- Please use a headset in order to avoid echo (a simple phone headset is just fine).
- You can find all the controls (mic, video, chat, screen sharing) at the bottom of the Zoom window (when you bring your mouse there).
- You can use the chat box for questions and comments, but please make sure you reply to "all panelists and participants" instead of just "all panelists", which is often the default.
- If you have a spoken question/comment, please use the "raise hand" button: we will then give the floor (and microphone rights) to you.
- Note: for questions and answers about the course topics, we will be using this living document

## :memo: Q & A

You can type your questions here. We will answer them, and this document will store the answers for you for later use! :rocket:

### General and practical matters

- [x] **Q0: I have difficulty pasting my questions into HackMD. Do you have some instructions on how to write here?**

- A: Can you see these three icons on top left corner, next to HackMD text? There’s pencil, this side-by-side symbol, and an eye. In eye view, you can’t edit, you are just viewing. The other two reveal the markdown (MD) version of the page, which you can edit. I find it easiest to edit with the side-by-side view.
:::info
:bulb: **Hint:** You can also apply styling from the toolbar at the top :arrow_upper_left: of the editing area.
:::

- [x] **Question 1: What is the schedule for today 16 March?**
- Thanks for asking! Forgot to mention this! :) So we have lectures & exercises until 12:00, but we will keep a bit longer "bio-break" (15min) in between. Exercise sessions are hands-on, and a good time to stretch a bit, get a cup of coffee, etc... too :) -MariaL

- [x] **Q13: I cannot access most of the slides from the course page. These are marked with an earth logo.**
- A: Try to use the arrow keys or space bar to switch the slide! (Thanks for sharing this info, we need to add this tip there below the slides!)

- [x] **Q17: Slides available after the course?**
- A: Yes, you can access e-lena after the course as well! You can download the slides from there (right click and save), and the exercises (and slides, in one format) are available in github: https://github.com/CSCfi/csc-env-eff You can also convert the html slides to pdf by adding '?print-pdf' at the end of html file like this : https://a3s.fi/CSC_training/03_disk_areas.html?print-pdf
and then type: control/cmd+p (on mac, cmd+p works)
- [x] **Q18: Course certificate?**
- A: Sure, send Maria an e-mail in case you need a course certificate!

### Using Puhti

- [x] **Question 2: How one knows how much memory an action requires?**
- Answer: Good question! We are checking this in more detail tomorrow! Here's a link to manual page about estimating the efficiency of your reservation with `seff` command: https://docs.csc.fi/computing/performance/
- [x] **Question 3: I failed to set the SHH key to CSC servers.**
- Answer: Let's take a look at this with you when there's the next hands-on break!

**Which machine are you using? Mac or windows...**
- [x] **Question 4: Should I be able to connect to puhti.csc.fi using the same user and password that I use when I login to the CSC web pages? -- I have a CSC login password as well as Haka password. I guess CSC login password should work for Puhti?**
- A: You need CSC credentials - for my.csc.fi (and e-Lena) you can nowadays login also with Haka credentials (=your university credentials), but to access Puhti, you need the CSC username and password. In my.csc.fi you can check the username, and if you have forgotten the password, check this link: https://docs.csc.fi/accounts/how-to-change-password/
- [x] **Question 5: Is it possible to add Graphab software to Puhti? https://sourcesup.renater.fr/www/graphab/en/home.html**
- A: We need to take a look! Our geoscience specialist Kylli Ek might be more familiar with this one! (Geoscience pages at research.csc.fi: https://research.csc.fi/geosciences)
- [x] **Question 6: I have successfully connected via windows Powershell, putty and NoMachine - which one should I use, is there some reason to use one rather than another?**
- A: What ever works is fine :) If there's need for graphical connection, then you need NoMachine. Tunneling connections via ssh e.g. to use Jupter Notebook or R-studio server I'd recommend MobaXterm for Windows users. It will be easier. Check the logging in hands-on tutorials.

- [x] **Q7: In putty what do we write in host name?**
- A: Check the instructions in docs.csc.fi: https://docs.csc.fi/computing/connecting/ But type there:puhti.csc.fi, see below: 
-

- [x] **Q8: Can you demonstrate NoMachine login too?**
- A: Check these docs.csc.fi pages: https://docs.csc.fi/apps/nomachine/
https://docs.csc.fi/support/tutorials/nomachine-usage/
- [x] **Q9: Do we need NoMachine (graphical access) during this course?**
- A: Not during this course!
- [x] **Q10: Do Puhti and Mahti rsync automatically or the user needs to do it?**
- A: Puhti and Mahti have separate file systems, so if you need same files on both, you'll need to copy them over yourself. We will talk more about handling data and Allas next week!
- [x] **Q11: Just to be sure: Is it so that files that are 90 days old are always removed from scratch, OR files that haven't been used in 90 days?**
- A: Files that haven't been used. The system looks at the last edit date of files. Just reading a file will not change this. If necessary, you can use command "touch" to update the edit date. https://docs.csc.fi/computing/disk/#scratch-directory
- [x] **Q12: It says after the log in that scratch cleaning is not yet active (contrary to the slide)… When will this be implemented? (This is crucial since there were some problems with Allas and I still have many files that may not be properly backed up.)**
- A: The clean-up is still not in use. We have not yet set a date, but /scratch is filling up fast, so we will have to turn it on soon.

- [x] **Q14: In the last lines of the disk areas tutorial, what does the $USER part mean?**
- A: That is your username, $USER is an *environment variable* that holds your username. Try to give this command `echo $USER`
- [x] **Q15: In the tutorial, you give permissions for the merged.fasta on the personal folder after copying. Do these permissions get also passed to the copied file in scratch?**
- A: Once the file is copied, you can modify the permissions of THAT file inside the scratch directory (in tutorial, file is actually copied to scratch).
- [x] **Q16: Is it possible to ask for modules/software to be added to the list?**
- A: Yeah, you can always send e-mail to [e-mail]! We'll take a look, and if it can be added and might be useful for others as well, we either install it for you or help you to install it!

- [x] **Q19: About the storage bonus excercise, https://github.com/CSCfi/csc-env-eff/blob/master/hands-on/disk-areas/disk-areas-exercise-fastdisks.md, how to send the script to compute node? There is no srun / sbatch specified in the script file, will it run in login node or in the compute one?**
- A: It should not be run on login node. The exercise is meant for sbatch submission as you can see from script. Once you finished modifying script you can submit the job as below to run the script on compute node:

`sbatch script_name.sh`
NOTE: At the moment, Puhti is under unscheduled break; Please do this exercise when Puhti is back to normal.
Although a command is not preceded by `srun` in a batch script, it will still be executed in the compute node. Not using `srun` may result in different action, though. See the template scripts per application for more use cases.
- [x] **Q20: Request for Relion: We would need it. It would be nice to put it to the list as it is now the most commonly used suite for cryo-EM.**
- A: It seems Relion 3.1.1 is available as a module in Mahti.
- [x] **Q21: Is it possible to take a moment to show how you would open a Jupyter Notebook using this sinteractive option?**
- A: Check the tutorial here, and the more detailed question 28 below: https://docs.csc.fi/computing/running/interactive-usage/#example-running-a-jupyter-notebook-server-via-sinteractive
- [x] **Q22: Having a "tree: command not found"**
- A: "tree" is not available in the interactive node. Instead of tree command, you can use “ls -l pythium”
- [x] **Q23: Hi. I wonder that is it possible to accidentally modify files that do not belong to my own project?**
- A: Not unless the files rights are modified that way, and that is likely not a good idea. If you need to share and work with others in Puhti, you need to invite them to the same project. Also, you can give read permissions but not write permissions. Note this also for files in Allas.
- [x] **Q24: I wonder how I can copy-paste into PuTTY?**
- A: Try: ctrl + insert = paste (or shift + insert)
- [x] **Q25: mkdir: missing operand ? what is the problem here?**
- A: After mkdir, you need to set the name of the directory you want to create. So for example: `mkdir my_new_folder`
- [x] **Q26: Also on the EnaDataGet "No WGS set file available for AKYA02000000, format fasta"**
- **Note: embl format is available, will that do for the excercise?**
- **Note: will not not work, need fasta format to compare sequences at the last step**
- **Can we try ```pythium_iwayamai.fasta``` from the first pythium excercise?**
- A: Try: `enaDataGet ERR1558694 -f fastq`
- **Works to download, but not in fasta format to perform the last step of the exercise - plus it seems like its short reads**
- A: For the locale problem, try: `export LC_ALL="en_US.UTF-8"`
- [x] **Q27: What is this `ls` command showing?**
- A: Lists the content of a directory. Check these out: https://docs.csc.fi/support/tutorials/env-guide/using-linux-in-command-line/#basic-commands-for-using-directories and https://docs.csc.fi/img/csc-quick-reference.pdf
- [x] **Q28: How to open Jupyter notebook on Puhti when you dont have ssh keys in place?**
On interactive node load module that has jupyter notebook and then launch notebook as below:

module load python-data
start-jupyter-server

If you have SSH keys set-up with Puhti, you can just follow the instructions that are displayed once jupyter is started. In case you use password to login to Puhti follow this ssh-tunneling process:

ssh -l <username> -L 8888:localhost:8888 puhti-login1.csc.fi    # Issue this command while being on local machine
                                                                # change port number if notebook is exposed on different port (default port is 8888 here); 
                                                                # choose login1 or login2 node 
                                                                                
ssh -l <username>  -L 8888:localhost:8888 <username>@$HOSTNAME       # Issue this command on login node; $HOSTNAME is compute node attached in batch job
                                                                     # hostname  of compute node attached to jupyter notebook 
                                                      

Point your browser to http://localhost:8888/?token=tokenkey (copy and paste this url from the instructions displayed once jupyter is launched)
If you want to use the server version, we recommend you set up the ssh-keys. Later this year, another setup called Open on Demand will at least partially provide another solution for this.

- [x] **Q29: Is there any lower limit for billing? For example, if one needs to explore data - thus using less resources - can this be done "for free"?**
- A: Once you reserve resources (like CPUs, Memory) even for testing, billing units are enforced. As such, smaller jobs don't cost that much in terms of billing units. You can actually check using 'seff' command after job is finished. Although CSC services for open research (academic use) are free of charge, all jobs are subjected to billing units and, no billing-free jobs.
- [x] **Q30: #SBATCH -e and #SBATCH -o for error and output?**
- A: Correct :) e = error, o = output
- [x] **Q31: Can you explain how file editing goes: 'Copy the example above into a file called my_serial.bash '**
- A: You need to use some editor, for example nano, vi or emacs. In login node: just type `nano my_serial.bash`. If you are on an interactive run, first type `module load nano` and then `nano my_serial.bash`. Using nano: https://www.nano-editor.org/dist/latest/cheatsheet.html
- [x] **Q32: When seff "perl: fatal: Bad job/step specified: slurm-5236590.out" MY BAD! bad input :)**
- A
- [x] **Q33: I created the command file in nano, run it and it is said 'submitted batch job XXX'. But then, nothring happens? I cannot see the job in the queue either. it cannot run that long! -- How to check the output then?**
- A: The example job 1 is very short, 1 minute only! So if you are not quick, `squeue -u $USER` won't show it anymore after it is completed. If you miss the output, you can however check output file (by default written into a file named slurm-XXXXXXX.out where XXXXXXX is a unique number corresponding to the job ID of the job), or use seff command to see what happened.
- [x] **Q34: Is there a difference between file types .bash and .sh?**
- A: Not really. File endings in Linux/unix are not functional in the same way as in some other operating systems.

- [x] **Q35: Handy trick, Kimmo. Can we have that somewhere in tutorials/hackmd? How to setup the script with sbatch variables with the sinteractive command again?**
- A: sinteractive -i -f new-batch-job-file
-
- [x] **Q36: I get this error, while trying to run sbatch my_serial.bash:**
sbatch: error: AssocMaxSubmitJobLimit
sbatch: error: Batch job submission failed: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)
- A: This error is usually caused by missing or incorrect --account parameter (although in this case it was due trying to run command directly instead of putting them in a file).
- [x] **Q37: In the interactive jobs tutorial, you recomend using NoMachine for tools with a graphical interface. Would this apply when using a Rstudio as an interactive job? Would it improve the performance?**
- A: RStudio Server is available on Puhti as part of the r-env-singularity module (https://docs.csc.fi/apps/r-env-singularity/). We wouldn't recommend NoMachine for this as the software is launched using SSH tunneling, with access to RStudio made available through one's local web browser (without a need for X11 forwarding). In this particular case, NoMachine would not afford a performance benefit and we recommend using either PuTTY or MobaXterm instead for setting up SSH tunneling (see https://docs.csc.fi/computing/connecting/#setting-up-ssh-keys).
- [x] **Q38: I had this experience that based on different sample size each time the mem usage was different, so sometimes I was allocating too much memory and sometimes the job would be stopped because of not enough memory! is there any option to know how much memory would be actually need?**
- A: Estimating memory beforehand is difficult. So normally you need to reserve some "extra" memory.
- **Q38b: what would be nice and decent amount of "extra" memory? 1GB? 200MB? Is it possible to give a figure?**
- I would reserve rather, say 20% extra, than just a fixed amount. 8 GB is still a rather small memory reservation.

### Data management

- [ ] **Question/comment 1: I like how the content in the RDM kit is organized as a reference manual. Nice example, perhaps something to adopt in a university web page as well?**
- Answer: Great, thanks! Feedback is very welcome :)
- [ ] **Question 2: To clarify a detail about the informed consent. If we want to e.g. process data outside Finland although the consent restricts analysis to Finland. Is it enough to inform the patient or should we get new written consent?**
- Answer: Francesca will talk to a colleague, and returns to this. It is a good idea to ask from your organisations legal personnel!
- [ ] **Question 3: How about the generation of the sequencing data? (Sequencing provider outside the lab)**
- Answer: They are a service provider, so there might already be an agreement between institutions (provider and institution). Good point!
-
- [ ] **Question 4: Can a private key ever be recovered?**
- A: No, if you lose your password, there is no way to de-encrypt the data. Be careful! :) Using our public key is a solution, as then there is a backup with us. There's a very strict policy in hosting our public key on our side.
- [ ] **Question 5: How will the encryption work in SD Connect, the tool to store sensitive data in Allas?**
- A: For now, you need to do it yourself, later on it will be automated. Timeline is unclear still, we are working on this!
- [ ] **Question 6: Is crypt4gh available in Puhti?**
- A: It is available in Puhti already, BUT, you are not allowed to import your sensitive data to Puhti un-encrypted. It is not yet possible to analyse sensitive data in HPC (Puhti, Mahti). You need to have the encryption tool installed locally, on your own computer. Here you can download the GUI: https://github.com/CSCfi/crypt4gh-gui/release

module load biokit
module load biopythontools
crypt4gh -h

### Using Allas

- [ ] **Question 1: If you do not remember which protocol (swift/s3) you have used, can it be determined from the allas object?**
- Answer: Not necessarily, but when not possible, it's not that much a problem. Small files you can use with both protocols. Bucket names can help: a-commands give by default the projectnumber-puhti-filename names. A-commands packages/compresses the files into one tar package.
- [ ] **Question 2: What if the bucket doesn't exist in Allas, when using rclone copy?**
- Answer: The command creates the bucket.
- [ ] **Question 3: Is there the 5G limit with a-put command?**
- Answer: No. Kimmo demonstrated what happens with a >5GB file: the file is split into several pieces, segments. Two buckets are created: In the main bucket, there's info where the segments are (size=0), and the actual segments are in the segments folder/bucket. Luckily, you don't really have to worry about this :)
- [ ] **Q4: Does the connection to allas get interrupted automatically after 8h? So is there a timer so we can check for how long we have been logged on to allas?**
- Answer: Yes. This might cause problems when moving large files (>500GB). Tip: the -k option = session remembers your password, so the Allas connection can be automatically re-created, when new a-put is launched. This slows down the process, so don't use when necessary! For batch jobs this is handy.
- [ ] **Q5: Is there a python/R interface for a-tools?**
- Answer: No.
- [ ] **Q6: Would the other big files (>5G) go to the same segment-bucket as the previously moved ones?**
- Answer: This depends on the bucket name, which you define when uploading the data.

- [ ] **Q6: Can I directly transfer all data in my poject_200XXX direcotry using a-put /scracth/Project_200XXX?**
- A: Good question! Don't try to move data within Puhti with Swift protocol/rclone. You need to pull it back to Puhti and then again to Allas.

- [ ] **Q7: No autocomplete in Allas? Tricky to navigate!**
- A: Nope, it is not a filesystem so there's no file hierarchy (even thou the filenames LOOK like file paths!).

- [ ] **Q8: What does -P mean in command rclone -P?**
- A: "Follow progress" = what is the transfer rate and how much have been moved already. BTW, commands have the help pages, -h! This is helpful when you don't know what a parameter means

- [ ] **Q9: When running backups, is it typically OK to do this from the login node? Do we need to go to compute node?**
- A: Normally, it's ok to use login node for this. If there's a very large object to be moved, the compression starts to require quite a lot CPU time, so then the interactive compute node would be the nice thing to do :) More resources there.

- [ ] **Q10: About publicity: Allas is not 'eternal' link, so it cannot be used as a way to share data in publications, or can it?**
- A: Yes and no :) Don't use the link in publications. It is valid as long as you have the project in Allas. This is a good way to share data with collaborators. DOI and such needed when really publishing data, check IDA at that point!

- [ ] **Q11: What does -b mean in command `a-put pythium/* -b xxxx-genomes-ap`?**
- A: Same as `--bucket`, "Define a name of the bucket into which the data is uploaded". `a-put -h` helps when wondering what these options mean

- [ ] **Q12: How can one save the big result files of sbatch-runs directly to Allas? Should the compression be avoided, if the result files are then later shared with other people in other institutions (published)?**
- A: Yes, you can make the sbatch runs store the data to Allas. `-k` is the trick here, before you launch the batch job. The you can use a-put with options --nc or --asis if you dont want to use the automatic compression or packing.

- [x] **Q12: I wonder if there is a command to empty the console in PuTTY?**
- A: Try `clear`

- [x] **Q13: To download data from outside CSC to Allas, how it is done? Just replacing the path with 'pythium'( as the command was 'a-put pythium')?**
- A: If you have user account, you can use the web interface (or Cyberduck). Or you can install a-tools or rclone to your environment, if the need is more continuous.

- [x] **Q14: Can you just upload file from your computer to Allas via cPouta?**
- A: Yes. You just need to do it one by one. We are developing the interface, so that it would be possible to upload a folder.

- [ ] **Q15: Is there a way to copy files from Funet FileSender link straight to Allas, i.e. without the need to first copy to your own computer?**
- A: You can use rclone copyurl for that. Copy the download link from FunetFilesender. Then use it in quotation marks with rclone copyurl. Like:

   rclone copyurl "filesender-download-link"  allas:bucket/object


### Containers


- [ ] **Q1: Are containers executed from the concole?**
    -  Answer: Yes. Actually, one can execute (or run) an image and then it becomes a container. Please check here (https://docs.docker.com/engine/reference/commandline/run/) for more details on different options. For docker you need sudo access rights and for singularity, you can launch a container with normal user rights

- [ ] **Question 2: Singularity for Mac? Could not find a command-line version just a “beta desktop version”**
    - Answer: Some virtual box was needed, so it wasn't super straight-forward... More detials if interested are here: https://singularity.lbl.gov/archive/docs/v2-3/install-mac
- [ ] **Question 3: So I guess for a Mac user the strategy for building hpc compatible singularity images would be to build using docker (have it installed) and convert to singularity on Puhti?**
    - Answer: That would work, or then installing a VM with some Linux OS. Note: Ready-made Docker images are sometimes written so that the applications are run as sudo which causes problems in Docker-to-Singularity-conversion, but in most cases this conversion works.

- [ ] **Q4: I have a notable lag inputting text with sinteractive. Is it my connection or a feature of the system?**
    - Answer: Many people are doing the same thing at the same time now on the interactive node, so this might be causing the lag. So usually the situation is more pleasant. With THESE EXERCISES, it's ok to use the login node (instead of the s interactive queue), as these can be considered light weight tasks, if this lag is terrible.

- [x] **Q5: I wonder what am I doing wrong. Trying to open the text editor by typing 'nano' in the console but nothing happens...?**
    - Answer: You need to `module load nano` first!

- [ ] **Q6: Follow-up Q to Q5: After creating test.sh, the following error appears:[llindfor@r07c52 cont]$ sbatch test.sh
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified**
    - Answer: Check that you have the correct project number determined!


- [x] **Q7: Where should the output come for the batch job test.sh?**
    - Answer: There should be a file called slurm-<jobid>.out in the folder where you sent the job.

- [ ] **Q8: When using R in Puhti, we use the command "module load r-env-singularity" to load it. Is there some connection to the singularity with this?**
    - Answer: Yes, this R has been installed using Singularity!
    - There are wrapper scripts using singularity_wrapper for commands R and Rscript. You can take a look at the scripts with:

module load r-env-singularity
less `which R`

- [ ] **Q9: Error writing test.sh: Permission denied. Why's that…?**
    - Answer: If you can't write a file, you may be under wrong directory where you don't have write permission. If execution is the problem, you can first try `chmod u+x test.sh` and then try again

- [ ] **Q10: The `singularity build pytorch_20.03-py3.sif docker://nvcr.io/nvidia/pytorch:20.03-py3 `command takes really long time I’ve been on `INFO:    Creating SIF file...` for almost 10 minutes**
    - Answer: It's a heavy operation, and it does take time... Maybe we should find a smaller image for the exercise! :) Like this one: `singularity build alpine.sif docker://library/alpine:latest` 
    - You can just ctrl+C to kill the process if it is taking long time, we don't really need it for anything, as this is just a tutorial :)

- [ ] **Q11: Is it faster ifs we use more cpus/threads? Like: `sinteractive -c 4`**
    - Answer: Yes, multiple cores makes it faster. In other partitions you can get more cores even. Also consider using local fast i/o drives; 

- [ ] **Q12: Feedback: I wish that in all tutorials there would be a comment e.g. on the right side of the commands that we are supposed to run in console. Its often difficult to understand what each line is actually doing.**
    - Answer: Thank you very much for the feedback! We are actively developing the material, so all feedback is highly valued!

- [x] **Q13: Do we have sudo privilige inside the singularity container to e.g. modify program settings?**

    - Answer: You have the same user rights inside the container as you have on the host. So if you are running the container as normal user, you also have only normal user rights inside the container. If you run it with sudo, you have sudo rights (not possible on Puhti, of course).

- [x] **Q14: Can we use the singularity inspect to see what tools are installed inside the container?**
    - Answer: Sadly no, not really. Open a shell in there, and see what is where. 


- [x] **Q15: Env?**
    - Answer: env is a command that prints out all the environment variables.


- [x] **Q16: I'm seeing lots of warnings?**
    - Answer: Try `unset XDG_RUNTIME_DIR` -this will hide the warnings :) We could add this to our tutorial!
  
- [x] **Q17: Singularity wrapper can be used as a replace to `--bind`, meaning that it binds the host dir with inside the container itself?**
    - Answer:   Singularity_wrapper does:  
        - -B /users:/users 
        - -B /projappl:/projappl 
        - -B /scratch:/scratch 
        - -B /appl/data:/appl/data

- [x] **Q18: So, outside the sinteractive it would be: `export SINGULARITY_CACHEDIR=/scratch/project_200XX` for both cache and tmpdir?**
    - Answer:  Yep!

- [ ] **Q19: Is there some good community web-page covering nextflow pipelines combined to containers? I mean something other than nf-core?**
    - Answer: Check this:  https://www.nextflow.io/docs/latest/index.html

- [ ] **Q20: At the uncomment step (`// publishDir 'results'`) I can't seem to find that line in fastqc.nf.
The only "uncommentable" line is `// publishDir params.outdir`. But `params.outdir = "results"` has been definied previously, so I think this should work the same wasy, no?**
    - Answer:  Yes, it was defined earlier, so it should work that way.

- [ ] **Q21: How to to define `<image_path>` in here n`extflow run <nextflow_script>  -with-singularity <image_path> ` How do I know which of the files is the image?**
    - Answer: ` shub://IARCbioinfo/nf_coverage_demo:v2.3`
    - Or if you are using a local image file (like we did yesterday), provide a full path for that.

- [ ] **Q22: I didn't know we cannot use "nano" in the sinteractive! Wondering how should I open txt files?**
    - Answer:  You can use nano, you just need to first load the nano module: `module load nano`

- [ ] **Q23: Can you explain again the thing about over-writting the param file?**
    - Answer: `reads` = hardcoded, `params.reads` = changeable

- [ ] **Q24: cPouta is free for researcher associated with University? And in future it will be good for sensitive data? But if project is over can it be still used e.g. Google cloud or amazon and how much is price (approx). --Do billing units apply in the case of Research in higher education institutions (=one of the free-of-use cases)? --Why do you call them ***billing*** units, that's confusing!**
    - Answer: cPouta, like many other CSC services (Puhti, Mahti, Allas...) are free-of-charge for academic research, education and training in Finnish higher education institutions and in state research institutes. ePouta is the current service for sensitive data. Francesca presented the SD-services that are atm under development on third session, that material is available in e-lena. Regarding the pricing, see: https://research.csc.fi/pricing
More about terms of use etc: https://research.csc.fi/terms-of-use-and-pricing?inheritRedirect=true
Billing units are these things we use to measure the usage of our services: if your use case is one of the free-of-charge use cases (like research in higher education institution here), you just apply for BUs, but they don’t cost really anything.
See the billing units chapter here: https://research.csc.fi/pricing
We call them billing units, because someone is paying for the use: if not the end-user (like in the free-use-cases), then it's the ministry :)

- [x] **Q25: How should I change the Singularity cache location to my Project directory**

    - Answer:  

export SINGULARITY_CACHEDIR=/some/path

- [x] **Q26:**
    - Answer: 


- [ ] **Q27:**
    - Answer: 

- [ ] **Q28:**
    - Answer: 

:::info
:bulb: **Hint:** You can also apply styling from the toolbar at the top :arrow_upper_left: of the editing area.

![](https://i.imgur.com/Cnle9f9.png)
:::

> Drag-n-drop image from your file system to the editor to paste it!