<!-- .slide: data-background="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/green_white_bg.svg" -->
<a href="https://www.nf-co.re"><img src="https://raw.githubusercontent.com/nf-core/logos/master/byte-size-logos/bytesize-darkbg.svg" width="65%"><img></a>
# \#40: Software Packaging for nf-core
Alex Peltzer/ <img src="https://openmoji.org/data/color/svg/E040.svg" width=50> @alex_peltzer / <img src="https://openmoji.org/data/color/svg/E045.svg" width=50> @apeltzer
Boehringer Ingelheim Pharma GmbH & Co. KG.
---
# Overview
- Best Practices: Software packaging
- Bioconda / conda-forge: Whats the difference?
- How To Package a tool in Bioconda / conda-forge
- Biocontainers: Docker & Singularity at a glance
---
# Best Practices - Do's
 & 
- nf-core is heavily relying on upstream projects to package software / tools for pipelines (**bioconda**, **conda-forge** & **biocontainers**) ♻️
- We encourage people to contribute to these wherever possible
---
# Bioconda and Conda-Forge
<img src="https://i.imgur.com/Y7gMF3C.jpg" width=30%>
(well, obviously not :smirk:)
---
# Bioconda and Conda-Forge
- Bioconda ➡️ if you have a tool that is clearly life-sciences related
- Conda-Forge ➡️ if its general purpose tools, e.g. base R Packages
- Packaging relies on similar infrastructure, but there are differences
---
# How to package - Step I
- Step 1: Check if tool is already on [bioconda](https://bioconda.github.io/conda-package_index.html) / [conda-forge](https://conda-forge.org/feedstock-outputs/)
<img src="https://i.imgur.com/iw0OMTn.png" width=50%>
---
# How to package - Step II
- Step 2: Check contributor docs for adding to [bioconda](https://bioconda.github.io/contributor/index.html)/[conda-forge](https://conda-forge.org/#add_recipe)
- Bonus hint for Bioconda: Join the [Gitter channel for detailed questions](https://gitter.im/bioconda/Lobby) & join the Github organization immediately - makes your life easier
---
# How to package - Step III
- Step 3: Write your recipe, usually either rely on the [templates provided](https://bioconda.github.io/contributor/guidelines.html) or recycle a similar package recipe ♻️ (YMMV)
- Use skeleton templates where possible!
```bash
conda skeleton pypi <packagename> #for pypi
conda skeleton cran <packagename> #for R (CRAN)
```
- 🚨 You can test your recipe locally using `conda-build` which *should* build your recipe
---
# Example Recipe
- Usually `build.sh` and `meta.yml` - build sometimes very small
<img src="https://i.imgur.com/ccBFT4F.png" width=50%>
---
# How to package - Step IV
- Submit PR & wait for auto-build to check your recipe for success :crossed_fingers: (Bioconda / conda-forge are slightly different here)
- Once approved/reviewed by others in the community
<img src="https://i.imgur.com/zL2CPGQ.png" width=50%>
---
# So what is in it for me now?
- Bioconda/conda-forge recipes guarantee that you can install a package using `conda/mamba install -c bioconda conda-forge <packagename>` ✨
- Regular updates by community (including you!) - more people doing package updates for everyone - more frequent updates :100: :sunglasses:
- Ensure consistency across many different tools / packages, findability for tools
---
# So what about Docker and Singularity containers?
---
# So what about Docker and Singularity containers?
- Bioconda 🤝 Biocontainers - they are automatically built from conda recipes
---
# Containers continued
- Automatically generated from bioconda recipes, uploaded to Quay.io, e.g. [Samtools](https://quay.io/repository/biocontainers/samtools)
- Singularity containers are built by Galaxy Team and shared via [Galaxy Depot](https://depot.galaxyproject.org/singularity/) :bow: :rocket:
---
# Container Example :boom:
```bash
docker run -it quay.io/biocontainers/samtools:0.1.19--h20b1175_10 samtools --version
singularity run https://depot.galaxyproject.org/singularity/samtools%3A1.15--h1170115_1 samtools --version
```
---
# So what about *multi-tool* containers ?
- Bioconda/conda-forge recipes are "one tool per container" only :warning:
- If you need more than one per container -> [multi tool containers](https://github.com/BioContainers/multi-package-containers)
- Add your set of tools to the [hash.tsv](https://github.com/BioContainers/multi-package-containers/blob/master/combinations/hash.tsv), open a PR and wait for it to be merged :rocket: :heavy_check_mark:
---
# How to use containers efficiently in nf-core pipelines
- Much easier since DSLv2 - rely on **biocontainers** and **nf-core/tools**
```
nf-core modules install fastqc,samtools,multiqc
```
---
# How to use containers efficiently in nf-core pipelines
:information_source: The above installs three modules in your pipeline, with pre-configured Docker & Singularity containers that are automatically used by the pipeline. [Update works similar!](https://nf-co.re/tools/#update-modules-in-a-pipeline)
:information_desk_person: If you build a new module, tools will search biocontainers via API to query these URLs for you :sparkles: :male_mage:
---
# nf-core's approach to packaging software/tools
1. Check bioconda / conda-forge for existence of recipe of your tool :heavy_check_mark:
2. Add to the above if not present :building_construction: :female-construction-worker:
3. Rely on biocontainers + Galaxy to build Docker & Singularity containers :package:
4. Use nf-core/modules and nf-core/tools to auto-fetch/update URLs in modules :rocket:
5. Use `mamba` as drop-in replacement for `conda` where possible - better error outputs, faster dependency resolving
---
# Last words
- Software packaging *can get complicated* sometimes - but there are many here to help
<img src="https://i.imgur.com/eAq2dS1.png" width=30%>
- In case of doubt 🙋🤔: Ask! (Bioconda, Conda-Forge communities) on more detailed packaging questions or in the #help channel on [our slack](https://nf-core/join) for guidance / input!
---
# :warning: Remember
- Collaboration is key - everyone benefits from you adding recipes to bioconda / conda-forge - not only nf-core contributors / users!
<img src="https://i.imgur.com/WqMqEef.jpg" width=40%>
---
## Need help?
<!-- .slide: data-background="https://raw.githubusercontent.com/maxulysse/maxulysse.github.io/main/assets/img/svg/green_white_bg.svg" -->
Modules Tutorial: [`https://nf-co.re/developers/tutorials/nf_core_contributing_overview`](https://nf-co.re/developers/tutorials/nf_core_contributing_overview)
Tutorial: [`https://nf-co.re/usage/usage_tutorials`](https://nf-co.re/usage/usage_tutorials)
Conda-Forge: [`https://conda-forge.org/#add_recipe`](https://conda-forge.org/#add_recipe)
Bioconda: [`https://bioconda.github.io/contributor/guidelines.html`](https://bioconda.github.io/contributor/guidelines.html)
Chat: [`https://nf-co.re/join`](https://nf-co.re/join) <img src="https://cdn.brandfolder.io/5H442O3W/at/pl546j-7le8zk-6gwiyo/Slack_Mark.svg" width=7.5%></img>`#help`
<div style="margin-top:0.1em"> </div>
<p align="center">
Follow nf-core on
<a href="https://www.twitter.com/nf_core"><img src="https://openmoji.org/data/color/svg/E040.svg" width=6%></a>
<a href="https://github.com/nf-core"><img src="https://openmoji.org/data/color/svg/E045.svg" width=6%></a>
<a href="https://www.youtube.com/c/nf-core"><img src="https://openmoji.org/data/color/svg/E044.svg" width=6%></a>
</a>
</p>
<a href="https://nf-co.re/" style="color: #000000; font-family:Monaco, monospace; font-weight:bold;">https://nf-co.re/</a>
<div style="display: flex; justify-content: space-evenly; align-items:center;">
<img src="https://chanzuckerberg.com/wp-content/themes/czi/img/logo.svg" width=15%>
<div style="font-style:italic; font-size: 0.5em; color: #666;">Icons:<br><a href="https://openmoji.org">openmoji.org</a></div></div>
<style>
.reveal section img { background:none; border:none; box-shadow:none; }
body {
background-image: url(https://raw.githubusercontent.com/nf-core/logos/master/nf-core-logos/nf-core-logo-square.svg);
background-size: 7.5%;
background-repeat: no-repeat;
background-position: 3% 96%;
background-color: #181a1b;
}
.reveal body {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h1 {
font-family: 'Roboto', sans-serif;
font-style: bold;
font-weight: 400;
color: white;
font-size: 62px;
}
.reveal h2 {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h3 {
font-family: 'Roboto', sans-serif;
font-style: italic;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal li {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal pre {
background-color: #272822 !important;
display: inline-block;
border-radius: 7px;
color: #aaaba9;
}
.reveal pre code {
color: #eeeeee;
background-color: #272822;
font-size: 100%;
}
.reveal code {
background-color: #272822;
font-size: 75%;
}
.reveal .progress {
color: #24B064;
}
.reveal .controls button {
color: #24B064;
}
.reveal blockquote {
display: block;
position: relative;
width: 90%;
margin: 20px auto;
padding: 5px;
background: rgba(255, 255, 255, 0.05);
box-shadow: 0px 0px 2px rgb(0 0 0 / 20%);
}
</style>
{"metaMigratedAt":"2023-06-16T22:34:54.999Z","metaMigratedFrom":"YAML","title":"nf-core/bytesize 40 Software packaging","breaks":true,"contributors":"[{\"id\":\"6357fb8d-b0f8-4b7d-9fbe-6d813ad67700\",\"add\":173,\"del\":31},{\"id\":\"07755eb0-739a-4628-85e2-ad4a4a341d04\",\"add\":7725,\"del\":2327},{\"id\":\"361dff76-5a26-4d26-b3b2-56c5fdc58926\",\"add\":0,\"del\":4}]"}