{%hackmd theme-dark %}
# Hackathon March 2021
---
# Pipelines group
<!-- TODO: State the goal of your group -->
* Contributing to existing pipelines
* Creating new pipelines
* Porting pipelines to DSL2
* Pipeline maintenance: e.g. template updates
---
# Group Members
<!-- TODO: add all group members -->
- Maxime Garcia
- Gisela Gabernet
- Alex Peltzer
- Susanna Marquez
- Daniel Straub
- Daniel Lundin
- Emelie Nilsson
- Leon Bichmann
- Gervaise Henry
- Friederike Hanssen
- Daniel Schreyer
- Simon Heumos @subwaystation
- Anthony Underwood
- Andrea Guarracino @AndreaGuarracino
- @avantonder
- Sangram @sk-sahu
- Patrick Hüther @phue
- Alexandre Gilardet
- Sabrina Krakau
- Barry Digby
- Thanh Le Viet
---
# Sarek 3.0 - Achievements
#### Monday
- Maxime: working on [#358](https://github.com/nf-core/sarek/pull/358)
- Many people: working on modules
#### Tuesday
- Maxime: still working on [#358](https://github.com/nf-core/sarek/pull/358)
- Maxime: about to integrate more modules
#### Wednesday
- Maxime: Finally merged [#358](https://github.com/nf-core/sarek/pull/358)
- Maxime: Starting to work on integrating the numerous modules created during the hackathon
---
# scRNAseq - Achievements
#### Monday
- Sangram/Alex: Updated JSON schema + opened PR with latest template 1.13.1, Gisela reviewed it ;-)
#### Tuesday
- Sangram fixed container issues, PR merged to dev.
- Waiting for adding latest tools template and will make a minor release.
#### Wednesday
- Sangram - Taking care of PR with latest template 1.13.2, review helped by - Gisela :)
- Minor release v1.1.0 before porting into DSL2 - happening
- Adding modules for kallisto and kd (@FloWuenne, @Batool, @ggabernet)
---
# Hlatyping - Achievements
#### Monday
Please chat on #hlatyping in Slack if possible. Alex will be around there and hopefully start working on it Monday afternoon :-)
- Alex: JSON Schema bugfix is in, waiting for tools fix to fix the template
#### Tuesday
- Alex: Started working on OptiType modules for DSLv2 conversion
- YARA added, Optitype added to modules
#### Wednesday
- Alex: Added YARA Module for DSLv2 Conversion https://github.com/nf-core/modules/pull/353
- Started template for DSLv2 conversion for hlatyping pipeline
- Still some stuff to do on that one :frowning:
---
# Methylseq - Achievements
#### Monday
- Patrick: added missing modules to nf-core/modules that are required for the dsl2 port
- [bismark/report](https://github.com/nf-core/modules/pull/297)
- [bismark/summary](https://github.com/nf-core/modules/pull/295)
- [bismark/align](https://github.com/nf-core/modules/pull/239)
- Patrick: started to bring [dsl2 draft](https://github.com/nf-core/methylseq/pull/182) into shape
- removed files that are becoming obsolete with dsl2
- Dockerfile, environment.yml, some CI workflows
#### Tuesday
- Patrick: preparing nf-core/methylseq for 1.6 release (hopefully the last dsl1 release)
- template sync 1.13.1 -> https://github.com/nf-core/methylseq/pull/194
- resolved some conda conflicts for newer dependency versions
- resolved a small [issue](https://github.com/nf-core/methylseq/pull/194) that has been on the v1.6 milestone for a while
- PRs ready, but some CI tests seem to be stuck; not sure why
- Patrick: working on dsl2 port
- trying to sync with dsl2 template from @drpatelh's branch of nf-core/tools
- currently getting some cryptic java error with nextflow `21.03.0-edge`; needs more investigation
`Unable to make field private static volatile java.util.List java.nio.file.spi.FileSystemProvider.installedProviders accessible: module java.base does not "opens java.nio.file.spi" to unnamed module @71d44a3` -> nextflow reinstalled solved it
#### Wednesday
- Patrick: opened 1.6 release PR -> https://github.com/nf-core/methylseq/pull/198
- Patrick: worked on dsl2 port aka 2.0dev
- synced with latest dsl2 template
- pushed [dsl2 branch](https://github.com/nf-core/methylseq/tree/dsl2) to nf-core/methylseq repo
- in good shape overall
- workflow summary shows NAs, don't know why
- **please test** :)
---
# Pangenome - Achievements
<!-- State what you achieved here -->
#### Monday
- [x] @subwaystation: Pull request [Important! Template update for nf-core/tools v1.13.1](https://github.com/nf-core/pangenome/pull/51) and issue [Resolve Template Update PR](https://github.com/nf-core/pangenome/issues/54) resolved via pull request https://github.com/nf-core/pangenome/pull/56.
- [x] @subwaystation: Resolved issue [Sync with latest pggb](https://github.com/nf-core/pangenome/issues/52) via pull request https://github.com/nf-core/pangenome/pull/57.
- [x] @subwaystation: Bring `nextflow_schema.json` up to date https://github.com/nf-core/pangenome/pull/58.
#### Tuesday
- [x] @subwaystation: Finished a successful automatic building of a docker image https://github.com/pangenome/pgge/pull/17 required for pgge integration.
- [x] @AndreaGuarracino: add a new flag to compress the output files. https://github.com/nf-core/pangenome/pull/60
#### Wednesday
- [x] @subwaystation: Pull request [Important! Template update for nf-core/tools v1.13.2](https://github.com/nf-core/pangenome/pull/59) resolved via pull request https://github.com/nf-core/pangenome/pull/61.
- [x] @AndreaGuarracino: better graph visualization management https://github.com/nf-core/pangenome/pull/62
- [ ] @subwaystation: Add pgge as a submodule to the pipeline. **FIRST DRAFT READY.**
---
# Ampliseq - Achievements
#### Monday
- Created new branch "DSL2" in nf-core/ampliseq to keep the DSL2 code in development more accessible and [opened PR to dev](https://github.com/nf-core/ampliseq/pull/232) that is solving [transition to DSL2](https://github.com/nf-core/ampliseq/issues/223), [version reporting](https://github.com/nf-core/ampliseq/issues/222), [missing bibliography](https://github.com/nf-core/ampliseq/issues/221), [data for full test](https://github.com/nf-core/ampliseq/issues/215), [update software](https://github.com/nf-core/ampliseq/issues/214), [uses parameter schema](https://github.com/nf-core/ampliseq/issues/205), [classify preprocessed sequences](https://github.com/nf-core/ampliseq/issues/202)
- Pull request [forward preferentially DADA2 classification to downstream analysis](https://github.com/nf-core/ampliseq/pull/233) to open up new possibilities for [taxonomic classification and downstream analysis](https://github.com/nf-core/ampliseq/issues/231)
- Working on flexible and user friendly usage of a mutitude of taxonomic databases.
#### Tuesday
- Added support for more databases, e.g. [gtdb](https://github.com/nf-core/ampliseq/pull/234) and [unite](https://github.com/nf-core/ampliseq/pull/237), through a [module](https://github.com/nf-core/ampliseq/pull/239)
- Fixed [CI tests](https://github.com/nf-core/ampliseq/pull/235)
#### Wednesday
- Fixed downstream analysis, extended options with --input, and fixed linting for nf-core tools 1.13.2 in [one go](https://github.com/nf-core/ampliseq/pull/238)
- Updated the [Unite and GTDB reformatting](https://github.com/nf-core/ampliseq/pull/242)
- Parallellised CI tests for [profiles](https://github.com/nf-core/ampliseq/pull/241)
- Moved various [cutadapt steps into a subworkflow and generate cutadapt summary files](https://github.com/nf-core/ampliseq/pull/246)
---
# Bactmap - Achievements
#### Monday
* @aunderwo prelim version of a pipeline with inputs and a single optional trimming step using fastp working.
* @avantonder new gubbins module
* @avantonder new snpsites module
* @avantonder new iqtree module
* @avantonder new fasttree module
* @alexandregilardet new mash_sketch module
#### Tuesday
next step to contribute other missing modules and stitch them together
* @avantonder new rapidnj module
* @avantonder new raxml-ng module
* @avantonder update CITATIONS.md
* @avantonder update README.md
* @thanhleviet adding a rasusa fastq sampling module
* @aunderwo building modules into pipeline. Pipeline now runs up until the point of having a sorted bam
* @alexandregilardet adding a bcf mpileup/call step
#### Wednesday
* @alexandregilardet added the FastTree step
* @avantonder added gubbins module to nf-core/modules and created PR
* @aunderwo made sure versions were all recorded
* @thanhleviet added genome estimation for rasusa sampling module
* @aunderwo went down a Groovy rabbit hole that he hasn't emerged from!
---
# Bcellmagic - Achievements
#### Monday
* @ggabernet template update to v1.13.1
* @ssnn is working in a temp private repo on some modules for bcellmagic. Template upgrade done. Working now on DSL2 regorganization (modules, params, doc...)
#### Tuesday
* @ggabernet Fixing a couple of issues for DSL2 release
* @ssnn Figured out how to reorganize the code, and use custom container.
---
# Eager - Achievements
#### Monday
- Added requested functionality to pmdtools process (https://github.com/nf-core/eager/issues/349).
#### Tuesday
#### Wednesday
- James merged in 1.13.2 template together with Maxime / Thiseas?
- Fixing MultiVCFAnalyzer not being triggered https://github.com/nf-core/eager/pull/711
---
# mag - Achievements
#### Monday
- [Sabrina] Finished first DSL2 version, just waiting for tests (https://github.com/nf-core/mag/pull/162).
#### Tuesday
- [Sabrina] Finished first DSL2 version for real (https://github.com/nf-core/mag/pull/162, https://github.com/nf-core/mag/pull/171), cleaned params shown in summary, parallellised CI tests (https://github.com/nf-core/mag/issues/170).
- [Sabrina] Compressed output files (https://github.com/nf-core/mag/pull/172)
- [Sabrina] Continued work on gtdb-tk classification (https://github.com/nf-core/mag/issues/173)
- [Sabrina] Start dealing with runtime issue (when computing co-assemblies) of MetaBAT helper script (https://github.com/nf-core/mag/issues/166)
#### Wednesday
- [Sabrina] Fixed bug in splitting unbinned sequences, improved runtime of helper script in MetaBAT2 process (https://github.com/nf-core/mag/pull/175)
- [Sabrina] Continued work on gtdb-tk classification (https://github.com/nf-core/mag/issues/173)
---
# DIAproteomics / MHCquant - Achievements
#### Monday
- [Leon] Template update merge conflicts resolved
- [Leon] Opened two PRs:
- DIAproteomics: https://github.com/nf-core/diaproteomics/pull/103
- MHCquant: https://github.com/nf-core/mhcquant/pull/160
- couple of bugs remain after merging
#### Tuesday
- [Leon] Found an issue in the 1.13.1 template (https://github.com/nf-core/tools/issues/937)
- [Kevin M and Phil E] Solved the issue
- [Leon] Two PRs still open but:
- DIAproteomics and MHCquant tests pass
- bugs were:
- execution rights for scripts in bin folder
- params.name remaining in config
- [Leon] + added an review for:
- Metaboigniter: https://github.com/nf-core/metaboigniter/pull/10
#### Wednesday
- [Leon] Added another review for Metaboigniter
- [Leon] Updated github repository branch protection settings
- [Leon] Adressed reviews by Phil E
- Added more summary docu and references to README
- If tests pass both PRs can be merged tonight
---
# crisprquant - Achievements
#### Monday
- Template updated for input data
- integrated functionality to generate fasta file from library csv file
- Finished including first module
#### Tuesday
- Added indexing, mapping and trimming process
- working on issue with a software's input requirement
#### Wednesday
- fixed issue with input (Thanks Maxime)
- working on first succesful pipeline run
---
# circrna - Achievements
#### Monday
* Template update v13.1
* Add summary log to main.nf
* Stopped pushing directly to head fork - made PRs from local fork :')
#### Tuesday
* Added parameter `--bsj_reads` allowing users to filter by minimum circRNA reads counts.
* Added parameter `--tool_filter` allowing users to take union/intersection of mutliple quantification tools.
* Update help, summary logs, input parameter checks, usage.md, schema.json for above parameters.
* Template update v13.2
#### Wednesday
* reworked `--tool_filter` to take integers instead of strings (i.e output circrnas must be called by at least *n* tools)
@BarryDigby
### Thank you Gisela for leading this workgroup :)
---
{"metaMigratedAt":"2023-06-15T20:39:07.651Z","metaMigratedFrom":"YAML","title":"Pipelines","breaks":true,"description":"View the slide with \"Slide Mode\".","contributors":"[{\"id\":\"5aed272d-8a9a-446f-ad9d-fc39f2ce1f30\",\"add\":375,\"del\":142},{\"id\":\"098b2d6d-01be-41cd-8c08-d9c48d2dc4c2\",\"add\":610,\"del\":34},{\"id\":\"688a7413-4fd0-4c99-9ff6-727407ceed7e\",\"add\":141,\"del\":0},{\"id\":\"d16ff33f-37e2-471f-956a-ed771ba64c1e\",\"add\":995,\"del\":2},{\"id\":\"fb193497-1111-470c-a594-827d34b6f673\",\"add\":409,\"del\":38},{\"id\":\"361dff76-5a26-4d26-b3b2-56c5fdc58926\",\"add\":1160,\"del\":254},{\"id\":\"847ef72c-7712-4848-a58a-795a183c718a\",\"add\":2127,\"del\":353},{\"id\":\"86f5f2d8-206c-4433-828a-7cb2494165f5\",\"add\":1402,\"del\":237},{\"id\":\"4dce6579-d047-4955-bcfe-2cc8e6cbfffc\",\"add\":175,\"del\":0},{\"id\":\"36f3da29-b3d6-44ae-bc4e-644ef7c9d3ad\",\"add\":1665,\"del\":143},{\"id\":\"948b4be4-ca47-405b-9bfc-3dc19355cb1f\",\"add\":1096,\"del\":83},{\"id\":\"84592764-b179-4194-89a8-cfd136e52ec1\",\"add\":400,\"del\":67},{\"id\":\"59301f50-9a73-4a81-be55-0c8260ddf150\",\"add\":460,\"del\":49},{\"id\":\"07755eb0-739a-4628-85e2-ad4a4a341d04\",\"add\":838,\"del\":184},{\"id\":\"0bf05b5b-d827-4fe7-83d6-363130963052\",\"add\":685,\"del\":13},{\"id\":\"13bbc504-3a16-4a49-831b-ff3cc153a69c\",\"add\":127,\"del\":0},{\"id\":\"1a64aa5e-6005-40e3-b2d3-9fc9f48700d6\",\"add\":15,\"del\":0},{\"id\":\"53ecdebd-b399-4426-8622-f0ec77cf32c3\",\"add\":292,\"del\":1},{\"id\":\"f07c66bd-f137-4778-9af0-d2adbe7c91ef\",\"add\":231,\"del\":1},{\"id\":\"47e4517d-fe4f-4788-935e-cfa845abe7e6\",\"add\":14,\"del\":0},{\"id\":\"1ba148d8-e152-4cb4-bdc5-100993470a3e\",\"add\":132,\"del\":0},{\"id\":\"41476d21-4b95-4fee-8577-7df5bb3e16ba\",\"add\":23,\"del\":0},{\"id\":\"2aeca590-9d00-4b4f-a58a-620945456c9f\",\"add\":583,\"del\":241}]"}