Hackathon March 2021

Pipelines group

• Contributing to existing pipelines
• Creating new pipelines
• Porting pipelines to DSL2
• Pipeline maintenance: e.g. template updates

Group Members

• Maxime Garcia
• Gisela Gabernet
• Alex Peltzer
• Susanna Marquez
• Daniel Straub
• Daniel Lundin
• Emelie Nilsson
• Leon Bichmann
• Gervaise Henry
• Friederike Hanssen
• Daniel Schreyer
• Simon Heumos @subwaystation
• Anthony Underwood
• Andrea Guarracino @AndreaGuarracino
• @avantonder
• Sangram @sk-sahu
• Patrick Hüther @phue
• Alexandre Gilardet
• Sabrina Krakau
• Barry Digby
• Thanh Le Viet

Sarek 3.0 - Achievements

Monday

• Maxime: working on #358
• Many people: working on modules

Tuesday

• Maxime: still working on #358
• Maxime: about to integrate more modules

Wednesday

• Maxime: Finally merged #358
• Maxime: Starting to work on integrating the numerous modules created during the hackathon

scRNAseq - Achievements

Monday

• Sangram/Alex: Updated JSON schema + opened PR with latest template 1.13.1, Gisela reviewed it ;-)

Tuesday

• Sangram fixed container issues, PR merged to dev.
• Waiting for adding latest tools template and will make a minor release.

Wednesday

• Sangram - Taking care of PR with latest template 1.13.2, review helped by - Gisela :)
• Minor release v1.1.0 before porting into DSL2 - happening
• Adding modules for kallisto and kd (@FloWuenne, @Batool, @ggabernet)

Hlatyping - Achievements

Monday

Please chat on #hlatyping in Slack if possible. Alex will be around there and hopefully start working on it Monday afternoon :-)

• Alex: JSON Schema bugfix is in, waiting for tools fix to fix the template

Tuesday

• Alex: Started working on OptiType modules for DSLv2 conversion
  • YARA added, Optitype added to modules

Wednesday

• Alex: Added YARA Module for DSLv2 Conversion https://github.com/nf-core/modules/pull/353
• Started template for DSLv2 conversion for hlatyping pipeline
  • Still some stuff to do on that one
    Image Not Showing Possible Reasons

    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported

    Learn More →

Methylseq - Achievements

Monday

• Patrick: added missing modules to nf-core/modules that are required for the dsl2 port
  • bismark/report
  • bismark/summary
  • bismark/align
• Patrick: started to bring dsl2 draft into shape
  • removed files that are becoming obsolete with dsl2
    • Dockerfile, environment.yml, some CI workflows

Tuesday

• Patrick: preparing nf-core/methylseq for 1.6 release (hopefully the last dsl1 release)

  • template sync 1.13.1 -> https://github.com/nf-core/methylseq/pull/194
  • resolved some conda conflicts for newer dependency versions
  • resolved a small issue that has been on the v1.6 milestone for a while
  • PRs ready, but some CI tests seem to be stuck; not sure why

• Patrick: working on dsl2 port

  • trying to sync with dsl2 template from @drpatelh's branch of nf-core/tools
    • currently getting some cryptic java error with nextflow 21.03.0-edge; needs more investigation
      Unable to make field private static volatile java.util.List java.nio.file.spi.FileSystemProvider.installedProviders accessible: module java.base does not "opens java.nio.file.spi" to unnamed module @71d44a3 -> nextflow reinstalled solved it

Wednesday

• Patrick: opened 1.6 release PR -> https://github.com/nf-core/methylseq/pull/198
• Patrick: worked on dsl2 port aka 2.0dev
  • synced with latest dsl2 template
  • pushed dsl2 branch to nf-core/methylseq repo
    • in good shape overall
    • workflow summary shows NAs, don't know why
    • please test :)

Pangenome - Achievements

Monday

• @subwaystation: Pull request Important! Template update for nf-core/tools v1.13.1 and issue Resolve Template Update PR resolved via pull request https://github.com/nf-core/pangenome/pull/56.
• @subwaystation: Resolved issue Sync with latest pggb via pull request https://github.com/nf-core/pangenome/pull/57.
• @subwaystation: Bring nextflow_schema.json up to date https://github.com/nf-core/pangenome/pull/58.

Tuesday

• @subwaystation: Finished a successful automatic building of a docker image https://github.com/pangenome/pgge/pull/17 required for pgge integration.
• @AndreaGuarracino: add a new flag to compress the output files. https://github.com/nf-core/pangenome/pull/60

Wednesday

• @subwaystation: Pull request Important! Template update for nf-core/tools v1.13.2 resolved via pull request https://github.com/nf-core/pangenome/pull/61.
• @AndreaGuarracino: better graph visualization management https://github.com/nf-core/pangenome/pull/62
• @subwaystation: Add pgge as a submodule to the pipeline. FIRST DRAFT READY.

Ampliseq - Achievements

Monday

• Created new branch "DSL2" in nf-core/ampliseq to keep the DSL2 code in development more accessible and opened PR to dev that is solving transition to DSL2, version reporting, missing bibliography, data for full test, update software, uses parameter schema, classify preprocessed sequences
• Pull request forward preferentially DADA2 classification to downstream analysis to open up new possibilities for taxonomic classification and downstream analysis
• Working on flexible and user friendly usage of a mutitude of taxonomic databases.

Tuesday

• Added support for more databases, e.g. gtdb and unite, through a module
• Fixed CI tests

Wednesday

• Fixed downstream analysis, extended options with –input, and fixed linting for nf-core tools 1.13.2 in one go
• Updated the Unite and GTDB reformatting
• Parallellised CI tests for profiles
• Moved various cutadapt steps into a subworkflow and generate cutadapt summary files

Bactmap - Achievements

Monday

• @aunderwo prelim version of a pipeline with inputs and a single optional trimming step using fastp working.
• @avantonder new gubbins module
• @avantonder new snpsites module
• @avantonder new iqtree module
• @avantonder new fasttree module
• @alexandregilardet new mash_sketch module

Tuesday

next step to contribute other missing modules and stitch them together

• @avantonder new rapidnj module
• @avantonder new raxml-ng module
• @avantonder update CITATIONS.md
• @avantonder update README.md
• @thanhleviet adding a rasusa fastq sampling module
• @aunderwo building modules into pipeline. Pipeline now runs up until the point of having a sorted bam
• @alexandregilardet adding a bcf mpileup/call step

Wednesday

• @alexandregilardet added the FastTree step
• @avantonder added gubbins module to nf-core/modules and created PR
• @aunderwo made sure versions were all recorded
• @thanhleviet added genome estimation for rasusa sampling module
• @aunderwo went down a Groovy rabbit hole that he hasn't emerged from!

Bcellmagic - Achievements

Monday

• @ggabernet template update to v1.13.1

• @ssnn is working in a temp private repo on some modules for bcellmagic. Template upgrade done. Working now on DSL2 regorganization (modules, params, doc…)

Tuesday

• @ggabernet Fixing a couple of issues for DSL2 release

• @ssnn Figured out how to reorganize the code, and use custom container.

Eager - Achievements

Monday

• Added requested functionality to pmdtools process (https://github.com/nf-core/eager/issues/349).

Tuesday

Wednesday

• James merged in 1.13.2 template together with Maxime / Thiseas?
• Fixing MultiVCFAnalyzer not being triggered https://github.com/nf-core/eager/pull/711

mag - Achievements

Monday

• [Sabrina] Finished first DSL2 version, just waiting for tests (https://github.com/nf-core/mag/pull/162).

Tuesday

• [Sabrina] Finished first DSL2 version for real (https://github.com/nf-core/mag/pull/162, https://github.com/nf-core/mag/pull/171), cleaned params shown in summary, parallellised CI tests (https://github.com/nf-core/mag/issues/170).
• [Sabrina] Compressed output files (https://github.com/nf-core/mag/pull/172)
• [Sabrina] Continued work on gtdb-tk classification (https://github.com/nf-core/mag/issues/173)
• [Sabrina] Start dealing with runtime issue (when computing co-assemblies) of MetaBAT helper script (https://github.com/nf-core/mag/issues/166)

Wednesday

• [Sabrina] Fixed bug in splitting unbinned sequences, improved runtime of helper script in MetaBAT2 process (https://github.com/nf-core/mag/pull/175)
• [Sabrina] Continued work on gtdb-tk classification (https://github.com/nf-core/mag/issues/173)

DIAproteomics / MHCquant - Achievements

Monday

• [Leon] Template update merge conflicts resolved

• [Leon] Opened two PRs:

  • DIAproteomics: https://github.com/nf-core/diaproteomics/pull/103
  • MHCquant: https://github.com/nf-core/mhcquant/pull/160

• couple of bugs remain after merging

Tuesday

• [Leon] Found an issue in the 1.13.1 template (https://github.com/nf-core/tools/issues/937)

• [Kevin M and Phil E] Solved the issue

• [Leon] Two PRs still open but:

  • DIAproteomics and MHCquant tests pass
  • bugs were:
    • execution rights for scripts in bin folder
    • params.name remaining in config

• [Leon] + added an review for:

  • Metaboigniter: https://github.com/nf-core/metaboigniter/pull/10

Wednesday

• [Leon] Added another review for Metaboigniter
• [Leon] Updated github repository branch protection settings
• [Leon] Adressed reviews by Phil E
  • Added more summary docu and references to README
• If tests pass both PRs can be merged tonight

crisprquant - Achievements

Monday

• Template updated for input data
• integrated functionality to generate fasta file from library csv file
• Finished including first module

Tuesday

• Added indexing, mapping and trimming process
• working on issue with a software's input requirement

Wednesday

• fixed issue with input (Thanks Maxime)
• working on first succesful pipeline run

circrna - Achievements

Monday

• Template update v13.1
• Add summary log to main.nf
• Stopped pushing directly to head fork - made PRs from local fork :')

Tuesday

• Added parameter --bsj_reads allowing users to filter by minimum circRNA reads counts.
• Added parameter --tool_filter allowing users to take union/intersection of mutliple quantification tools.
• Update help, summary logs, input parameter checks, usage.md, schema.json for above parameters.
• Template update v13.2

Wednesday

• reworked --tool_filter to take integers instead of strings (i.e output circrnas must be called by at least n tools)

@BarryDigby

Thank you Gisela for leading this workgroup :)