# Nextflow Executors - Slurm
<small>
Maxime U Garcia ▸ <a href="https://github.com/maxulysse" style="color:white;" ><i class="fa fa-github"></i> @maxulysse</a>
<a href="https://seqera.io/" style="color:white;" >Seqera Labs</a> ▸ Barcelona | Stockholm
</small>
---
## Disclaimer
* Only my own limited usage
---
## Be Kind, Read the docs
* [executor](https://www.nextflow.io/docs/latest/executor.html)
> The executor provides an abstraction between the pipeline processes and the underlying execution system. This allows you to write the pipeline functional logic independently from the actual processing platform.
---
## Running a pipeline with Slurm
Using the UPPMAX profile.
So this [config](https://github.com/nf-core/configs/blob/master/conf/uppmax.config) file.
```
singularity {
enabled = true
}
process {
executor = 'slurm'
clusterOptions = "-A $params.project"
}
```
---
## Let's go
```
nextflow run nf-core/rnaseq \
-profile test,uppmax \
--outdir results
```
---
## First failure
<small>
```
ERROR ~ Error executing process > 'NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (RAP1_UNINDUCED_REP1)'
Caused by:
Failed to submit process to grid scheduler for execution
Command executed:
sbatch .command.run
Command exit status:
1
Command output:
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified
Work dir:
/UPPMAX/franb/private/work/bb/2f52d4706d2a012c350ae491ecd64f
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
-- Check '.nextflow.log' file for details
```
</small>
---
## Quick debug
Let's have a look at the `.command.run`
<small>
```
#!/bin/bash
#SBATCH -J nf-NFCORE_RNASEQ_RNASEQ_FASTQ_FASTQC_UMITOOLS_TRIMGALORE_TRIMGALORE_(RAP1_UNINDUCED_REP1)
#SBATCH -o /UPPMAX/franb/private/work/bb/2f52d4706d2a012c350ae491ecd64f/.command.log
#SBATCH --no-requeue
#SBATCH --signal B:USR2@30
#SBATCH -c 2
#SBATCH -t 06:00:00
#SBATCH --mem 6144M
#SBATCH -A null
NXF_CHDIR=/UPPMAX/franb/private/work/bb/2f52d4706d2a012c350ae491ecd64f
# NEXTFLOW TASK: NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (RAP1_UNINDUCED_REP1)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x
NXF_ENTRY=${1:-nxf_main}
```
</small>
---
## Remember remember the config file parameter
```
singularity {
enabled = true
}
process {
executor = 'slurm'
clusterOptions = "-A $params.project"
}
```
---
## And the .command.run
<small>
```
#!/bin/bash
#SBATCH -J nf-NFCORE_RNASEQ_RNASEQ_FASTQ_FASTQC_UMITOOLS_TRIMGALORE_TRIMGALORE_(RAP1_UNINDUCED_REP1)
#SBATCH -o /UPPMAX/franb/private/work/bb/2f52d4706d2a012c350ae491ecd64f/.command.log
#SBATCH --no-requeue
#SBATCH --signal B:USR2@30
#SBATCH -c 2
#SBATCH -t 06:00:00
#SBATCH --mem 6144M
#SBATCH -A null
NXF_CHDIR=/UPPMAX/franb/private/work/bb/2f52d4706d2a012c350ae491ecd64f
# NEXTFLOW TASK: NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (RAP1_UNINDUCED_REP1)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x
NXF_ENTRY=${1:-nxf_main}
```
</small>
---
## Once more, with feelings
```
nextflow run nf-core/rnaseq \
-profile test,uppmax \
--outdir results \
--project SECRET_PROJECT_ID
```
---
## One more time
<small>
```
#!/bin/bash
#SBATCH -J nf-NFCORE_RNASEQ_RNASEQ_PREPARE_GENOME_GUNZIP_GTF_(genes_with_empty_tid.gtf.gz)
#SBATCH -o /UPPMAX/franb/private/work/dc/9f7a164a16ceb1bbddbc2882278d36/.command.log
#SBATCH --no-requeue
#SBATCH --signal B:USR2@30
#SBATCH -t 04:00:00
#SBATCH --mem 6144M
#SBATCH -A SECRET_PROJECT_ID
NXF_CHDIR=/UPPMAX/franb/private/work/dc/9f7a164a16ceb1bbddbc2882278d36
# NEXTFLOW TASK: NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GUNZIP_GTF (genes_with_empty_tid.gtf.gz)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x
NXF_ENTRY=${1:-nxf_main}
```
</small>
---
## Need help?
<!-- .slide: data-background="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/green_transparent_bg.svg" -->
Chat: [`https://nf-co.re/join`](https://nf-co.re/join) <img src="https://cdn.brandfolder.io/5H442O3W/at/pl546j-7le8zk-6gwiyo/Slack_Mark.svg" width=7.5%></img>`#help`
<div style="margin-top:0.1em"> </div>
<p align="center">
Follow nf-core on
<a href="https://www.twitter.com/nf_core"><img src="https://openmoji.org/data/color/svg/E040.svg" width=6%></a>
<a href="https://mstdn.science/@nf_core"><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/4/48/Mastodon_Logotype_%28Simple%29.svg/216px-Mastodon_Logotype_%28Simple%29.svg.png" width=5%></a>
<a href="https://github.com/nf-core"><img src="https://openmoji.org/data/color/svg/E045.svg" width=6%></a>
<a href="https://www.youtube.com/c/nf-core"><img src="https://openmoji.org/data/color/svg/E044.svg" width=6%></a>
</a>
</p>
<a href="https://nf-co.re/" style="color: #000000; font-family:Monaco, monospace; font-weight:bold;">https://nf-co.re/</a>
<div style="display: flex; justify-content: space-evenly; align-items:center;">
<img src="https://chanzuckerberg.com/wp-content/themes/czi/img/logo.svg" width=15%>
<div style="font-style:italic; font-size: 0.5em; color: #666;">Icons:<br><a href="https://openmoji.org">openmoji.org</a></div></div>
---
<style>
body {
background-image: url(https://raw.githubusercontent.com/nf-core/logos/master/nf-core-logos/nf-core-logo-square.svg);
background-size: 7.5%;
background-repeat: no-repeat;
background-position: 3% 96%;
background-color: #181a1b;
}
.reveal body {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h1 {
font-family: 'Roboto', sans-serif;
font-style: bold;
font-weight: 400;
color: white;
font-size: 62px;
}
.reveal h2 {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h3 {
font-family: 'Roboto', sans-serif;
font-style: italic;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal li {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal pre {
background-color: #272822 !important;
display: inline-block;
border-radius: 7px;
color: #aaaba9;
}
.reveal pre code {
color: #eeeeee;
background-color: #272822;
font-size: 100%;
}
.reveal code {
background-color: #272822;
font-size: 75%;
}
.reveal .progress {
color: #24B064;
}
.reveal .controls button {
color: #24B064;
}
.reveal blockquote {
display: block;
position: relative;
width: 90%;
margin: 20px auto;
padding: 5px;
background: rgba(255, 255, 255, 0.05);
box-shadow: 0px 0px 2px rgb(0 0 0 / 20%);
}
.multi-column{
display: grid;
grid-auto-flow: column;
}
</style>
{"title":"Nextflow Executors","contributors":"[{\"id\":\"fb193497-1111-470c-a594-827d34b6f673\",\"add\":7231,\"del\":150}]","description":"Maxime U Garcia ▸ <a href=\"https://github.com/maxulysse\" style=\"color:white;\" ><i class=\"fa fa-github\"></i> @maxulysse</a><a href=\"https://seqera.io/\" style=\"color:white;\" >Seqera Labs</a> ▸ Barcelona | Stockholm"}