---
title: nf-core/bytesize - DSL2 Coding style recommendations I
tags: DSL2, Nextflow, bytesize, nf-core, open-source, talk
---

<!-- .slide: data-background="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/green_white_bg.svg" -->

<a href="https://www.nf-co.re"><img src="https://raw.githubusercontent.com/nf-core/logos/master/byte-size-logos/bytesize-darkbg.svg" width="60%"><img></a>

# \#46: DSL2 Coding style recommendations - part I

Maxime U Garcia ▸ [<i class="fa fa-twitter" aria-hidden="true"></i>@gau](https://twitter.com/gau/) | [<i class="fa fa-github" aria-hidden="true"></i>@maxulysse](https://github.com/maxulysse/)
Karolinska Institutet | Science for Life Laboratory
Sweden

---

## Overview

- 🍬 What has changed?
- 🧙 What are modules?
- 🚀 What should we do?

---

## Disclaimer

- My own recommendations
- Other developers might have others
- We're still forging the best practices
- It might (and probably will) evolve

---

## What has changed with DSL2?

[From Paolo 2 years ago](https://www.nextflow.io/blog/2020/dsl2-is-here.html)
> A module file is nothing more than a Nextflow script containing one or more process definitions that can be imported from another Nextflow script.

---

## Modules!

<a href="https://www.nextflow.io/docs/latest/dsl2.html#modules">
<img src="https://raw.githubusercontent.com/maxulysse/my_memes/main/one_push-up/modules.jpg" title="Everytime I say modules I do one push-up" alt="Everytime I say modules I do one push-up meme"/>
</a>

---

## What does it mean?

- A module is a module
- A process can be a module
- A sub-workflow can be a module
- A workflow can be a module

---

## Modules¿?

<a href="https://www.nextflow.io/docs/latest/dsl2.html#modules">
<img src="https://raw.githubusercontent.com/maxulysse/my_memes/main/spidermen_pointing/modules.jpg" title="spidermen modules pointing at each others" alt="spidermen modules pointing at each others meme"/>
</a>

---

## [Some definitions](https://nf-co.re/developers/modules#terminology)

- Module: a single atomic process
- Sub-workflow: a few chained modules
- Workflow: an end-to-end pipeline

---

## What should be done IMO

- Code should be easy to:
- read
- understand
- share
- modify
- contribute

---

## At the module level

- All code for the process is in `modules`
- Either local or nf-core
- [@modules modules/ensemblevep/main.nf](https://github.com/nf-core/modules/blob/fa37e0662690c4ec4260dae282fbce08777503e6/modules/ensemblvep/main.nf)
- Modular set up in `configs`
- [Closures](https://www.nextflow.io/docs/latest/script.html#closures) are your best friends
- [`ext`](https://www.nextflow.io/docs/latest/process.html#ext) (custom namespace directive)
- `args`, `args2`, `args3`, `prefix`, `when`
- [`publishDir`](https://www.nextflow.io/docs/latest/process.html#publishdir)
- [`container`](https://www.nextflow.io/docs/latest/process.html#container)
- [@sarek conf/modules.config](https://github.com/nf-core/sarek/blob/551ead3df694a91ac1f55599bb4e5194cd44d566/conf/modules.config#L1142-L1169)

---

## At the sub-workflow level

- Keep the sub-workflow simple
- Chain modules
- Manipulate channels
- [subworkflows/annotation/ensemblvep/main.nf](https://github.com/nf-core/sarek/blob/551ead3df694a91ac1f55599bb4e5194cd44d566/subworkflows/nf-core/annotation/ensemblvep/main.nf)

---

## Still at the sub-workflow level

- A sub-worfklow can include others
- Chain modules and/or sub-workflows
- Manipulate channels
- Execution logic
- [subworkflows/annotate.nf](https://github.com/nf-core/sarek/blob/551ead3df694a91ac1f55599bb4e5194cd44d566/subworkflows/local/annotate.nf)

---

## At the workflow level

- We can do everything
- Chain modules and/or sub-workflows
- Execution logic
- Channel manipulation
- [workflows/sarek.nf](https://github.com/nf-core/sarek/blob/551ead3df694a91ac1f55599bb4e5194cd44d566/workflows/sarek.nf#L924-L941)

---

## [Some syntax recommandations](https://hackmd.io/HWt-WyNDTWWhutJUolJmMw)

- Indentation is your friend
- Collect files to use in a process
- `def foo = bar.collect{"-I $it"}.join(' ')`
- Channel Assignment
- `ch_foo = ch_bar.map{ meta, baz -> [baz] }`
- `ch_bar.map{ meta, baz -> [baz] }.set{ ch_foo }`

---

## Stay at home tips

- Please add comments
- Good for you
- Good for your colleagues
- Good for future you
- Recognize your [queue](https://www.nextflow.io/docs/latest/channel.html#queue-channel) and [value](https://www.nextflow.io/docs/latest/channel.html#value-channel) channels
- Don't hesitate to ask questions
- Discuss
- Have a fika

---

## Acknowledgements

<img src="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/acknowledgments_2021_dark.svg" title="Institutes acknowledgements" alt="Institutes acknowledgements"/>

---

<a href="https://nf-co.re/community#organisations">
<img src="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/institutes_2021_dark.svg" title="Institutes acknowledgements" alt="Institutes acknowledgements"/>
</a>

---

<a href="https://nf-co.re/community#contributors">
<img src="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/slides/nf-core_contributors_2021_03.png" title="Contributors acknowledgements" alt="Contributors acknowledgements"/>
</a>

---

<!-- .slide: data-background="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/green_white_bg.svg" -->

## Need help?

- bytesize 05: [DSL2 module development](https://nf-co.re/events/2021/bytesize-5-dsl2-module-development)
- bytesize 06: [All about modules](https://nf-co.re/events/2021/bytesize-6-dsl2-using-modules-pipelines)
- bytesize 24: [Where do I start writing my own DSL2 pipeline?!](https://nf-co.re/events/2021/bytesize-24-dsl2-pipeline-starter)
- bytesize 34: [Updates on the new DSL2 syntax](https://nf-co.re/events/2022/bytesize-34-dsl2-syntax)

Get involved:
[https://nf-co.re/join](https://nf-co.re/join)

[<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" focusable="false" width="1.6em" height="1.6em" style="-ms-transform: rotate(360deg); -webkit-transform: rotate(360deg); transform: rotate(360deg);" preserveAspectRatio="xMidYMid meet" viewBox="0 0 24 24" class="slack-svg"><path d="M5.042 15.165a2.528 2.528 0 0 1-2.52 2.523A2.528 2.528 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52v2.52zm1.271 0a2.527 2.527 0 0 1 2.521-2.52a2.527 2.527 0 0 1 2.521 2.52v6.313A2.528 2.528 0 0 1 8.834 24a2.528 2.528 0 0 1-2.521-2.522v-6.313zM8.834 5.042a2.528 2.528 0 0 1-2.521-2.52A2.528 2.528 0 0 1 8.834 0a2.528 2.528 0 0 1 2.521 2.522v2.52H8.834zm0 1.271a2.528 2.528 0 0 1 2.521 2.521a2.528 2.528 0 0 1-2.521 2.521H2.522A2.528 2.528 0 0 1 0 8.834a2.528 2.528 0 0 1 2.522-2.521h6.312zm10.122 2.521a2.528 2.528 0 0 1 2.522-2.521A2.528 2.528 0 0 1 24 8.834a2.528 2.528 0 0 1-2.522 2.521h-2.522V8.834zm-1.268 0a2.528 2.528 0 0 1-2.523 2.521a2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.528 2.528 0 0 1 2.523 2.522v6.312zm-2.523 10.122a2.528 2.528 0 0 1 2.523 2.522A2.528 2.528 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522h2.52zm0-1.268a2.527 2.527 0 0 1-2.52-2.523a2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.528 2.528 0 0 1-2.522 2.523h-6.313z" fill="white" class="slack-path"/></svg>](https://nfcore.slack.com/) [<i class="fa fa-twitter fa-2x" aria-hidden="true"></i>](https://www.twitter.com/nf_core) [<i class="fa fa-github fa-2x" aria-hidden="true"></i>](https://github.com/nf-core) [<i class="fa fa-youtube fa-2x" aria-hidden="true"></i>](https://www.youtube.com/c/nf-core)

<style>
.reveal section img { background:none; border:none; box-shadow:none; }

body {
background-image: url(https://raw.githubusercontent.com/nf-core/logos/master/nf-core-logos/nf-core-logo-square.svg);
background-size: 7.5%;
background-repeat: no-repeat;
background-position: 3% 96%;
background-color: #181a1b;
}

.reveal body {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}

.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}

.reveal h1 {
font-family: 'Roboto', sans-serif;
font-style: bold;
font-weight: 400;
color: white;
font-size: 62px;
}

.reveal h2 {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}

.reveal h3 {
font-family: 'Roboto', sans-serif;
font-style: italic;
font-weight: 300;
color: white;
}

.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;

}

.reveal li {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}

.reveal pre {
background-color: #272822 !important;
display: inline-block;
border-radius: 7px;
color: #aaaba9;
}

.reveal pre code {
color: #eeeeee;
background-color: #272822;
font-size: 100%;
}

.reveal code {
background-color: #272822;
font-size: 75%;
}

.reveal .progress {
color: #24B064;

}

.reveal .controls button {
color: #24B064;
}

.reveal blockquote {
display: block;
position: relative;
width: 90%;
margin: 20px auto;
padding: 5px;
background: rgba(255, 255, 255, 0.05);
box-shadow: 0px 0px 2px rgb(0 0 0 / 20%);
}

.fa {
color: white;
}

a .fa-github:hover { color: #000000; }
a .fa-spotify:hover { color: #1DB954; }
a .fa-twitter:hover { color: #4099FF; }
a .fa-youtube:hover { color: #FF0000; }
a .slack-svg:hover .slack-path {fill: #000000;}

</style>