<section data-background-image="https://raw.githubusercontent.com/MaxUlysse/maxulysse.github.io/master/assets/img/svg/green_white_bg.svg" data-background-opacity=0.9 >
<a href="https://www.nf-co.re"><img src="https://raw.githubusercontent.com/nf-core/logos/master/byte-size-logos/bytesize-darkbg.svg" width="65%"><img></a>
# \#16: Modules test data
Kevin Menden / <img src="https://openmoji.org/data/color/svg/E040.svg" width=50> @kevmenden / <img src="https://openmoji.org/data/color/svg/E045.svg" width=50> @kevinmenden
QBiC, University of Tuebingen
---
# Overview
- How to use test data for nf-core/modules :question:
- What if the required data is not available :question:
---
# Recap: Testing nf-core modules :thinking_face:
- Every module is unit tested with Docker, Singularity & Conda :memo:
- Nextflow code for tests is located at:
`modules/tests/software/fastqc/main.nf`
- All tests must be mentioned in the `pytest_software.yml`
`modules/tests/config/pytest_software.yml`
---
# Example: FastQC `main.nf` file
```groovy
nextflow.enable.dsl = 2
include { FASTQC } from '../../../software/fastqc/main.nf' addParams( options: [:] )
workflow test_fastqc_single_end {
input = [ [ id:'test', single_end:true ], // meta map
[ file(params.test_data['sarscov2']['illumina']['test_1_fastq_gz'], checkIfExists: true) ]
]
FASTQC ( input )
}
```
---
# Example: FastQC `test.yml` file
```yaml=
- name: fastqc single-end
command: nextflow run ./tests/software/fastqc/ -entry test_fastqc_single_end -c tests/config/nextflow.config
tags:
- fastqc
files:
- path: ./output/fastqc/test_fastqc.html
- path: ./output/fastqc/test_fastqc.zip
```
---
# Wait, what? :confused:
How do I know the dictionary keys?
```groovy=
file(params.test_data['sarscov2']['illumina']['test_1_fastq_gz'], checkIfExists: true)
```
---
# The one file to test them all :construction_worker:
```bash=
modules/tests/config/test_data.config
```
```groovy=
// Base directory for test data
def test_data_dir = "https://raw.githubusercontent.com/nf-core/test-datasets/modules/data"
params {
test_data {
'sarscov2' {
'genome' {
genome_fasta = "${test_data_dir}/genomics/sarscov2/genome/genome.fasta"
...
}
'illumina' {
test_single_end_bam = "${test_data_dir}/genomics/sarscov2/illumina/bam/test.single_end.bam"
...
test_1_fastq_gz = "${test_data_dir}/genomics/sarscov2/illumina/fastq/test_1.fastq.gz"
test_2_fastq_gz = "${test_data_dir}/genomics/sarscov2/illumina/fastq/test_2.fastq.gz"
test2_1_fastq_gz = "${test_data_dir}/genomics/sarscov2/illumina/fastq/test2_1.fastq.gz"
...
```
---
# Get to know your test data :slightly_smiling_face:
- All test data is located on the modules branch of nf-core/test-datasets
:arrow_forward: https://github.com/nf-core/test-datasets/tree/modules
- :book: Have a look at the README
- :mag_right: Search for the data you need
---
# Currently available data
- two genomes:
- sarscov2 :heavy_check_mark:
- human (chr22) :heavy_check_mark:
- standard genomics data :heavy_check_mark:
- not-so-standard genomics data :heavy_check_mark:
---
# How to use the data in your code? :thinking_face:
- Identify the file you need :female-detective:
- Find the corresponding dictionary key in `test_data.config` :heavy_check_mark:
- Use they keys to link to the file in your test code :link:
---
# I can't find the necessary data! :anguished:
- Keep calm :sunglasses:
- Upload the data yourself :muscle:
---
# How to upload new test data
- :book: Read the instructions in the test-dataset README
- :thinking_face: Think about a suitable, minimal test dataset, ideally for SARS-CoV-2
- :question: Ask the #modules channel about your new data
---
# How to upload new test data
- :arrow_heading_up: Create a PR to the modules branch of nf-core/test-datasets
- Ask for reviews in #modules and get your PR merged :first_place_medal:
- Add an entry in `test_data.config` on the nf-core/modules repo :writing_hand:
- Use your fresh test data :cocktail:
---
# Guidelines for new data
- make the data as small as possible :small_blue_diamond:
- use and adapt it to available genomes if possible :recycle:
- use standardized names :name_badge:
---
# Adding data is worth it :money_with_wings:
- adding new data takes time :hourglass_flowing_sand:
- but it benefits all of the community :dancers:
- it will benefit other communities in the future as well! :rocket:
---
## Need help?
Repository: [`nf-core/REPO`](https://github.com/nf-core/REPO)
Tutorial: [`https://nf-co.re/usage/usage_tutorials`](https://nf-co.re/usage/usage_tutorials)
Chat: [`https://nf-co.re/join`](https://nf-co.re/join) <img src="https://cdn.brandfolder.io/5H442O3W/at/pl546j-7le8zk-6gwiyo/Slack_Mark.svg" width=7.5%></img>`#channel`
<div style="margin-top:0.1em"> </div>
<p align="center">
Follow nf-core on
<a href="https://www.twitter.com/nf_core"><img src="https://openmoji.org/data/color/svg/E040.svg" width=6%></a>
<a href="https://github.com/nf-core"><img src="https://openmoji.org/data/color/svg/E045.svg" width=6%></a>
<a href="https://www.youtube.com/c/nf-core"><img src="https://openmoji.org/data/color/svg/E044.svg" width=6%></a>
</a>
</p>
<a href="https://nf-co.re/" style="color: #000000; font-family:Monaco, monospace; font-weight:bold;">https://nf-co.re/</a>
<div style="display: flex; justify-content: space-evenly; align-items:center;">
<img src="https://chanzuckerberg.com/wp-content/themes/czi/img/logo.svg" width=15%>
<div style="font-style:italic; font-size: 0.5em; color: #666;">Icons:<br><a href="https://openmoji.org">openmoji.org</a></div></div>
<section .slide: data-background-image="https://raw.githubusercontent.com/MaxUlysse/maxulysse.github.io/master/assets/img/svg/green_white_bg.svg" data-background-opacity=0.9 >
<style>
.reveal section img { background:none; border:none; box-shadow:none; }
body {
background-image: url(https://raw.githubusercontent.com/nf-core/logos/master/nf-core-logos/nf-core-logo-square.svg);
background-size: 7.5%;
background-repeat: no-repeat;
background-position: 3% 96%;
background-color: #181a1b;
}
.reveal body {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h1 {
font-family: 'Roboto', sans-serif;
font-style: bold;
font-weight: 400;
color: white;
font-size: 62px;
}
.reveal h2 {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal h3 {
font-family: 'Roboto', sans-serif;
font-style: italic;
font-weight: 300;
color: white;
}
.reveal p {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal li {
font-family: 'Roboto', sans-serif;
font-weight: 300;
color: white;
}
.reveal pre {
background-color: #272822 !important;
display: inline-block;
border-radius: 7px;
color: #aaaba9;
}
.reveal pre code {
color: #eeeeee;
background-color: #272822;
font-size: 100%;
}
.reveal code {
background-color: #272822;
font-size: 75%;
}
.reveal .progress {
color: #24B064;
}
.reveal .controls button {
color: #24B064;
}
.reveal blockquote {
display: block;
position: relative;
width: 90%;
margin: 20px auto;
padding: 5px;
background: rgba(255, 255, 255, 0.05);
box-shadow: 0px 0px 2px rgb(0 0 0 / 20%);
}
</style>
{"metaMigratedAt":"2023-06-16T02:05:41.767Z","metaMigratedFrom":"YAML","title":"nf-core/bytesize 16 modules test data","breaks":true,"contributors":"[{\"id\":\"8a7e2f4b-bae8-4838-ba0b-04044135b128\",\"add\":5022,\"del\":1654},{\"id\":\"07755eb0-739a-4628-85e2-ad4a4a341d04\",\"add\":23,\"del\":20},{\"id\":\"60ede81b-485d-4b0d-86bf-92161dcd277c\",\"add\":249,\"del\":122}]"}