owned this note changed 25 days ago
Published Linked with GitHub

2025-06-27

  1. Adam
  2. Anabella
  3. Edmund
  4. Famke
  5. Florian
  6. Igor
  7. James
  8. Jasmin
  9. Jon
  10. Joon
  11. Jonas
  12. Jose
  13. Júlia
  14. Lorena
  15. Louis
  16. Luisa
  17. Mahesh
  18. Matthias D. S.
  19. Matthias H.
  20. Maxime
  21. Nico
  22. Nicolas
  23. Pontus
  24. Ram
  25. Rike
  26. Simon
  27. Sateesh
  28. Usman (probably sleeping)

Agenda

Minutes

2025-05-30

  1. Adam
  2. Anabella
  3. Edmund
  4. Famke
  5. Florian
  6. Igor
  7. James
  8. Jasmin
  9. Jon
  10. Joon
  11. Jonas
  12. Jose
  13. Júlia
  14. Lorena
  15. Louis
  16. Luisa
  17. Mahesh
  18. Matthias D. S.
  19. Matthias H.
  20. Maxime
  21. Nico
  22. Nicolas
  23. Pontus
  24. Ram
  25. Rike
  26. Simon
  27. Sateesh
  28. Usman (probably sleeping)

Agenda

  • NO wg-taskforce-test-data takeover, but a Maxime takeover
  • nf-test migration progress report
  • @maxulysse will present the RFC proposal
  • nf-core/tools release happening next week (monday?) - includes pipeline level nf-test

Minutes

  • nf-test migration almost over
  • RFCs presentation
  • nf-core tools release
    • megatests should be fixed for good
    • discussion about nf-test pipelines tests conversion
  • bamtofastq squash-merge from dev to master
    • DON'T DO THAT :D

Action Items

  • Maxime to create a recurring zoom link for the meetings
  • Famke to write an RFC for branch merge

2025-04-25

  1. Adam
  2. Anabella
  3. Edmund
  4. Famke
  5. Florian
  6. Igor
  7. James
  8. Jasmin
  9. Jon
  10. Joon
  11. Jonas
  12. Jose
  13. Júlia
  14. Lorena
  15. Louis
  16. Luisa
  17. Mahesh
  18. Matthias D. S.
  19. Matthias H.
  20. Maxime
  21. Nico
  22. Nicolas
  23. Pontus
  24. Ram
  25. Rike
  26. Simon
  27. Sateesh
  28. Usman (probably sleeping)

Agenda

  • Welcome to new maintainers: Anabella, Louis, Pontus
  • Goodbye now alumni: Anders, Anders, Carson, Christopher, Lili, Sofia, Rob, Gisela, Harshil
  • @maxulysse: Can we set up for some pipelines a CODEOWNER, so that the actual pipelines maintainers can check up the PRs being merged, and help them actually maintain pipelines cc @emiller
  • @FriederikeHanssen Use prefix in output directive: https://github.com/nf-core/modules/issues/7792
  • @famosab: I am all for module wishlists (on Issues) I just think we need to do them differently than now - some people also responded that they completely forgot about the issue and do not need the module or it was implemented somewhere else and so on
  • @FloWuenne: I propose to include resource optimized configs for test_full profiles for all nf-core pipelines. Many orgs run these pipelines with full test profiles regularly for infrastructure tests etc. Default resource configurations are unnecessarily wasteful in most cases! I did some work to generate optimised configs for all pipelines that had succesfull runs since beginning 2024 here : https://github.com/FloWuenne/megatest-resource-optimization/tree/main/optimized_configs
  • @jfy133 comments on improvment of use of test-data?: https://github.com/nf-core/website/pull/3327
  • @jfy133: take over next meeting for wg-test-data?

Minutes

  • Welcome to new maintainers (everyone on holiday unfortunately 😅)
  • CODEOWNER
    • Some people want better information when PRs are being reviewed/merged in -> now allowing other groups coming in and reviewing amongst themselves
    • Two places: modules and pipelines
    • Opt-in? So not in templates
    • Have a fallback for core or maintainers (but we already have those rights)
    • But shouldn't protected branches help with that (you can set restrictions on pipeline repo level stuff)
    • Add to guidelines that you CAN use them if you want?
    • What if we autopropgate CODEOWNERS with maintainer tag of contribtuion in nextflow.config manifest?
  • Prefixes: this seems to be a Nextflow bug
  • Modules wishlishes -> there a TOO MANY ISSUES, to they make sense?
    • Famke's Idea: have a meta-issue to collect 'wishlist' modules for thoes who want to a module but don't want to implement themselves?
    • James: label with 'wishlist' and auto-close after 1 year if no-one takes it up (but only if no-one assigns themselves)
      • Label gets removed if someone assigns themselves (as it shows interest)
      • Can we re-add label if someone unassigns themselves
      • Can we ping assignees if they've not done the module after X amount of time
  • Florian's suggestions: James summarised (but Florian is not there)
    • Interest in automating?
    • Firstly test these optimised configs manually in their test_full
    • Maybe initially requiring in first release to add this (After the pre-release run)
  • James forced everyone to review his PR regarding test-data messiness
  • Júlia tools updates:
    • Probably a small patch release prior the bigger one to include the fixes to AWS and the latest-everything issue (likely next week ™️)

Action Items

  • Maxime: write a RFC adding codeowners to certain pipelines to test if it upsets people
    • James, Jon, Lorena, Jonas, Rike, Nicolas -> add a CODEOWNERs file with their current maintiners -> see if any pushback from any of their co-maintainers
    • Revisit in 2 months?
  • Rike to report Nextflow bug of captured input files in output patterns:
    • Ben already replied on the issue. It's a limitation of the scratch directive.
  • Famke (+ Edmund) to investigate wishlist labelling system
  • James/Lorena/Sateesh: try manually Florian's manual configs
    • Reivist in 2 months
  • James to speak to Simon about a agenda for wg-taskforce-test-data takeover of next meeting

2025-02-28

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Famke
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Lorena
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia
  • Sateesh

Agenda (Don't edit, only append!)

Minutes

  1. Tshirts
    • Will go live for a few days next week (James to announce)
  2. Spring cleaning
    • More than 500 people joining the hackathon, so we need to clean
    • The war plan was copied over from last year
    • Everyone should have a look at it
    • We will be cleaning up the teams as well
  3. Hackathon projects
    • GPU testing
    • nf-test testing
    • references
    • Ontologies?

Action Points

  • Spring cleaning (@jfy133 to add to calendar):
    • Kick off meeting Monday March 3rd 15:00 CET
    • Closing meeting Friday March 7th 15:00 CET
  • Hackathon

2025-01-31

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Famke
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Lorena
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia
  • Sateesh

Agenda (Don't edit, only append!)

  • Spring cleaning date decision
  • (final point:) Did we scare Famke off?
  • Question from Maxime: Who actively use the igenomes.config in their pipeline?
  • Proposal: Arthur Gymer for maintainers (again(?))
  • Proposal: echo commands rather than removing args (IN STUB)
  • Proposal: remove latest-everything from required tests on release PRs (still run, but not block releases)
  • Question from Florian: How can we get people to join the North American side of helpdesk?
  • Default prefixes in modules (https://github.com/nf-core/website/pull/2608)

Minutes

  • Spring cleaning date decision
    • Week 11 (03-10 -> 03 -14)
  • We haven't scared Famke yet
  • Maxime is given free rein with igenomes.config
    • He will hopefully present references soon
  • Maxime will ask Arthur about maintainer
  • STUB? args? Let's think more and raise the question later
    • We're removing args in stub due to language server
    • We don't mind echoing the args, but we don't print them, so it's kinda useless
    • Let's comment on the static types PR and see if we can do something there
  • Let's remove latest-everything from REQUIRED TESTS
  • Florian + Lorena have been almost alone in AMERICAS helpdesk
    • He worked on stickers, so it's still a win
  • Simon wants us to talk again about the default prefixes
  • Simon wants us to do pre-spring cleaning
  • Edmund informs us that seqera containers are in progress

Action Points

  • @jfy133 give @maxulysse and @FriederikeHanssen co-hosting rights
    • (not possible as on different paid 'accounts' - maybe switch to a Seqera )
  • @jfy133 Figure out how to get the Zoom meeting on the calendar invite
    • See above
  • @maxulysse asks Arthur about being a maintainer (even in Autralia)
  • DECISION FOR STUBS
    • echo args for now instead of removing
    • Capture stdout in stub test?
    • Talk with Ben and maybe Lukas and see how we can improve
    • Testing stubs with a plugin instead of writing stub tests
  • Let's do it and remove latest-everything from REQUIRED TESTS
  • Helpdesk AMERICAS
    • Change time?
    • Rebrand as Happy Hour?
    • Communicate more about the questions that can be asked
    • See with outreach team and see if we can pump on ads
  • @jfy133 @oRVLyAzyS6SABKri_BFj5w @maxulysse @emiller https://github.com/nf-core/website/pull/2608
    • jfy133
  • ALL OF US
    • Check your open PRs (@emiller
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      )

2024-11-29

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia
  • Sateesh

Agenda (Don't edit, only append!)

  1. @Jon To what degree should we adhere to Semantic versioning in pipeline releases? Specifically, what qualifies as a major release?
    • Option 1: we follow pretty closely. Any change to the interface (existing params/ inputs) should be a major release, so we should be doing a lot more of those. PR open at https://github.com/nf-core/website/pull/2842. (@Jon + ?) <<< pick this one!
    • Option 2: PhilVer - VERY loosely SemVer, but not too many major release because people would get major release fatigue (@Phil + ?)
    • Option 3: We use an LLM automation to tell us what the next release version should be so we don't argue about this (Florian + ?)
  2. @Rike Should we push for CRAM support in the genomics pipelines? What are the blockers (@Rike - in an attempt to safe polar bears)
  3. @Júlia Remove workflowCitation() function completely? or add it back to the pipelines template? https://github.com/nf-core/modules/pull/7094 (Júlia)
  4. @Florian Was contacted by a Nextflow dev at CDC about a checklist for Nextflow development they wrote: https://github.com/Ethan-Hetrick/ethans-toolbox/blob/main/nextflow/nextflow-best-practice-checklist.md#execution (Florian)
  5. (@speri) CI Changes:
    1. nf-test.yml: Runs Nextflow tests for workflows and modules across profiles using matrixed shards.
    2. gpu-tests.yml: Executes GPU-specific Nextflow tests for changes in GPU-related workflows and modules.
    3. nf-test-action.yml: Provides a reusable action to standardize and run Nextflow tests with configurable setups.
    1. detect-nf-test-changes: Include tags / Exclude tags
  6. @jfy133 Please announce when experimenting on public repos!
  7. @Simon Modules PRs are piling up again
  8. @maxulysse: /!\ Updating AWS iGenomes cf #references
  9. @maxulysse: Unify keys in igenomes / unify params name / unify fields in samplesheets / unify meta map fields (related to 1 and 7)
  10. @maxulysse: Automatic release notes (Addressed by 1)
  11. @maxulysse: Auto-generate CITATIONS for CITATIONS.md (all tools) and MultiQC (on run time) cf #regulatory

Minutes

  1. Has everyone recovered hackathon/summit? Some bitterness, some car mechanics
  2. Should we be stricter on SemVer?
    • Florian: We should try to make tooling to help developers on this
    • Jose: concerned about people misintepreting what a major release?
    • James: maybe just include in docs to tell Devs to avoid this

TANGENT: automation of releases?

  • Floran, Maxime, etc. what a lot of automation around that
  • Júlia reluctant because a lot of the AI stuff is really ugly
  • Floran says doesn't have to be LLM, but we would like more automation (maybe with tools) to also help standardisation - there are a lot of manual steps
  • GitHub action app for releases; some tooling stuff already used in tools, for example
  1. Maxime (for Rike) CRAM everywhere rather than BAM?
    • Adam/James: many tools won't work with CRAM, and then decompress/recompess procedures loses benefit
    • Maxime: we should fix those tools!
    • Adam/James: laughs in C and ancient DNA tools
    • Simon: can't influence input
    • Ultimately Rike wants bigger push
      • More awareness during reviewing?
      • Stronger wording in specifications
  2. Júlia: OK for me to remove workflowCitation()
    • Maxime: suggests replacing function within an nf-core nextflow plugin
    • No one uses/needs it, agree to remove
  3. Florian: checklists we could adopt?
    • Florian: CDC checklist from CDC is old
    • Sateesh: acually partly from when he was there
    • James: where to put them is hard question as docs structure isn't great
    • General agreement: he can make a PR if he wants, and when we do an overall we can check his checklist and use stuff we might be missing
  4. @speri CI Changes
    • TL;DR
      • nf-test sharding
      • GPU tests -> just needs a specific tag to run on a GPU runner
    • James: can we do similar GPU test but for requesting larger runners (E.g. tools that need more HDD space)?
      • Adam/Sateesh: somewhat yes, would require duplicating current action and set up a new/different queue for larger runs
    • Simon: does it help preventing PRs that run 200 tests accidently
      • Sateesh: Yes!
    • Jon/Simon (and many others) hate the new uninformative names
      • Sateesh to investigate, Maxime has a hacky implementation
    • Simon: random linting issues?
      • Sateesh: Edmun has a PR to fix this soon
  5. James: James wants Maxime to stop experimenting on Sarek and rnaseq Sateesh and Edmund exprimenting on modules ;)
  6. Simon: Sad that modules PRs are pilling
  7. Maxime: updating Igenomes, thanks Sateesh and Adam for the reviews

Action Points

  • Everyone to dump their ideas for breaking changes onto website #2842, and in 2 weeks @Jon to merge in, and then @Julia to design (release) linting rules around this
  • @Rike come up with more examples of how to push
  • @Júlia to remove the function!
  • @Florian: respond to Ethan at CDC to make a PR if he wants
  • @speri to improve naming of sharded CI tests
  • Everyone: let's close PRs in modules
  • Everyone: Announce it when you do stuff on common Repos
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
    -> fine to experiment, but be loud about it we're not in the Far West anymore, not more merge cowboy

2024-10-25

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia

Agenda (Don't edit, only append!)

  • Action points from last time

    • Maxime to invite Lorena Pantano -> FOR NEXT TIME
    • Rike to really this time restart nf-core helpdesk
  • Sign up for re-booted helpdesk: https://docs.google.com/spreadsheets/d/1c1Pz97W5azlPph9iiORAxRCLc6q4KBQRfK4M0h127KM/edit?usp=sharing

  • nf-core hackathon!

    • Who's coming?
    • Who's doing what?
      • Most people as leads already
      • Jon -> beginner training whole time
        Image Not Showing Possible Reasons
        • The image file may be corrupted
        • The server hosting the image is unavailable
        • The image path is incorrect
        • The image format is not supported
        Learn More →
        (but hackathon was planning DiffAbun improvements and references)
      • Florian -> Protein design
      • Mahesh -> genomeassembly/genomeqc
      • Matthias -> drawing corner
      • Igor -> expecting to work on CRG stuff
      • Sateesh -> nf-test plugins
      • Maxime -> Everything
      • Matthias de S -> Modules
  • Mahesh Binzer-PanchalMahesh PoC of nf-test config clean up: https://github.com/nf-core/modules/pull/6788

  • Mahesh Binzer-PanchalWhat to do about module maintainers holding up contributions? https://github.com/nf-core/modules/pull/5706

  • (sateesh) Discuss the depreciation of conf/modules.config in favor of individual configs. Example implementation - methylseq dev

    • (Mahesh): deprecation of conf/modules.conf: Makes it difficult to find settings when separate. User experience from a colleague, they like being able to see what flags are set given certain parameters. They don't want it split up.
  • [@jfy133] despite assumed knowlege, multiMap indeed does not ensure syncronised subchannels - as listed in updated docs under the new note block. We should change modules specs to have a single tuple to enure all files come intogether syncronised

  • Matthias De Smet Module guidelines points 3.3 and 3.6 seem to contradict each other. We should remove the ambiguous language and define whether or not each output file type MUST have their own output channel

  • Mahesh Binzer-Panchal Monthly update of changes in guidelines or PR reviewing. It's quite easy to fall behind and it would be nice to have monthly coverage of what's changed and go over how to review a PR. This could be a part of the office hours.

    • Can we have a GH action or something that on review request puts a checklist linking in a comment for the reviewer to check off. There would be a link also to the current guidelines for clarification of any points to be checked off.

Minutes

  • Maxime didn't invite Lorena
  • Hackathon
    • Who's doing what?
      • Most people as leads already
      • Jon -> beginner training whole time
        Image Not Showing Possible Reasons
        • The image file may be corrupted
        • The server hosting the image is unavailable
        • The image path is incorrect
        • The image format is not supported
        Learn More →
        (but hackathon was planning DiffAbun improvements and references)
      • Florian -> Protein design
      • Mahesh -> genomeassembly/genomeqc
      • Matthias -> drawing corner
      • Igor -> expecting to work on CRG stuff
      • Sateesh -> nf-test plugins
      • Maxime -> Everything
      • Matthias De S -> Modules
  • Mahesh' PoC clean up
    • Everyone loves it!
    • Only Maxime's concern: people might start doing this in a pipeline
    • Mahesh to update specifications and docs with warning!
  • Waiting for maintainers to review/approve
    • Guideline: if no (reasonable) action after 3 months, then go ahead and merge if reviewed thoroughly and tests passing
    • Maybe ask for another reviewer (so it's not steamrolling)
  • Splitting of modules.conf into one per module
    • Maxime/Sateesh reasoning: to help reduce triggering automated tests on file changes (i.e., don't trigger all module tests jsut because modules.conf)
    • Paolo/Mahesh/Jon/James/Matthias:
      • too hard to find for users to know exactly what arguments
      • not natively suppported by nextflow (a main.nf automatically imports the neighbouring nextflow.config)
      • cascading configs in rnaseq was also bad (Maxime)
      • (Mahesh) user will be confused - they find the modules modules.conf and think that's what is being used, but won't be awaremof being overridden by subworkflows.conf
      • Testing shouldn't be the priority over user/usability (Matthias) -> nextflow config should properly print out stuff, then it's OK (a user can use that to view)
      • To improve testing efficiency, we should add our own layer to detect changes
      • Igor: there are some additional subtitles to configs that won't work by passing stuff through channels
    • Vote: 6 nay for splitting vs 2 splitting
  • Modules specifications inconsistency
    • First solution: relax both 3.3/3.6
    • Matthias de S. wants to make strict again (James remembers discussion to relax this -> Mahesh also says it simplifies code a lot if we share channels for files that serve same purpose, less conditional channel combining)
    • Jon doesn't like unpredictable types
    • Mahesh/James don't like it because code becomes extremely complicated with loads of if/else statements to combine each channel back to the main file
    • James: users shouldn't modify args to add unsupported stuff that could influence this output file
  • Being informed about specification updaes
    • Versioning of docs again?
    • Better idea: more linting! Matthias says make issues for linting when you want it!

Action points

  • Maxime: blog update!
  • Maxime: invite Lorena
  • Mahesh: update nf-core modules specifications and other relevant module/nf-test documentations
  • Matthias: follow up with Simon P. if he remembers where allowing the combining of e.g. .bai,.csi came from/history

2024-09-27

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia

Agenda

  • Proposal for new maintainer: Lorena Pantano
    • Early nf-core adopter, one of the original maintainers for #smrnaseq and some other pipelines.
    • Reduced activity when she left academia to go to industry, but now she's back in academia again at Harvard.
    • She's now in the group that the bcbio workflow manager came from, and they're in the process of switching to nf-core and porting any missing functionality from bcbio into nf-core.
      • YES - Maxime will contact and ASK her
  • Coming nf-core/tools release
  • MASSIVE modules MASSIVE updates
    • GOOD progress, but still things to do
    • new modules page
      • New modules page looks AWESOME
    • Topics channel Edmund?
  • TEMPLATE update
    • Going further done the line of simplifying it
      • @maxulysse to do a speedrun to convert template to non-genomics pipeline at hackathon. Other than that people generally happy with the update: better support for non-nf-core pipelines, easier template merges for non-genomics pipelines
  • Conda lock files, where are they? (Maxime will look):
    • They will improve reproducibility
  • helpdesk is the new coffee-n-code
    • Rike to set it up. Really this time. Sorry!
  • Back to the discussion whether modules should have mandatory parameters as input (James vs. Simon)
  • pipeline level test in nf-core/rnaseq and nf-core/methylseq: https://github.com/nf-core/rnaseq/tree/master/tests / https://github.com/nf-core/rnaseq/pull/1379 and https://github.com/nf-core/methylseq/pull/414:
    • Discussion what to snapshot
    • Maxime to give a bytesize in how to do pipeline level snapshot
  • Mahesh shows how nf-test can be run without additional config files by putting the params into nf-test https://nfcore.slack.com/archives/C049MBCEW06/p1721042240473589:
    • Generally everyone happy about it. Mahesh will implement it for 1-2 modules.

2024-08-30

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia

Agenda (Don't edit, only append!)

  • (Phil) Seqera Containers
    • https://github.com/nf-core/modules/issues/6505:
      • Reviewing upcoming blog post by Edmund & Phil on Seqera Containers roll out
      • Summary: We are all happy. Lock files will be rolled out to ensure reproducibility. Container URLs will be autogenerated and hard coded into a config file.
      • Phil/Edmund will post a PR with the finalised Blog for discussion
    • (Edmund) Quick vote on Add renovate comments to environment.yml:
      • Keep bioconda declaration
  • (James) Review last meetings action points
    • Matthias to attempt again to list modules on subworkflow page
      • Postponed to next time
    • Julia to include singularity/conda tests in template (deadline: next template release).
      • Done, will come with next template update
      • Do we want the 0.9.0 nf-test CI in the template? For 3.0.0 release
    • Nicolas to update module guidelines to say all assertions in one snapshot (definitely not one per file):
      • Done
    • Mahesh to talk about params ext.args next time
      • Postponed to next time
    • Matthias to think about new ways of versioning module guidelines
      • Postponed to next time
      • Maybe annotate modules with guideline version
      • Maybe don't spend too much time on that cf Simon
  • (Rike) Office-hours continuation
    • We have volonteers to host session
    • Rebranding to (let's vote)
      • coffee-n-code
      • nf-coffee-chat
      • fireside-chat
  • (Simon)
    • Discuss allowing a default but overwritable prefix for modules (especially to convince James and Nicolas)
      • Nicolas is willing to compromise if we use task name as default. It would ensure it should work always
      • The disambiguate function should still be present because suffixes can overlap
      • the default behavior by many people is to add a suffix at the moment.
      • The output definition should in addition help with clearing up filename
    • Inviting any additional maintainers, particularly Usman Rashid
      • YES!
      • Maxime will ask
    • nf-test assertions page needs updating
      • Someone needs to do it
      • Maxime will lead and ping Nicolas/Simon/Usman
    • Any plans for meeting up at the Barcelona summit?
      • Maxime will figure out who will be around when

2024-07-28

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia

Agenda (Don't edit, only append!)

  • Last meetings action points
    • Matthias to attempt again to list modules on subworkflow page
    • Jon to update guidelines based on notes above (everyone to comment - with a poll)
    • Rike to set up a working group for pushing forward daisy-chaining
    • Test-data working group set up by Simon
    • Modules bulk update group set up by Julia
  • (Júlia) Test pipelines with singularity and conda
  • (Júlia) Bulk update of modules (https://github.com/nf-core/modules/issues/5828)
  • (Júlia) Restructuring of meta.yml (https://github.com/nf-core/modules/pull/5867)
  • (Nicolas) nf-test - a single snapshot per test vs a single snapshot per file (meaning many per test)
  • (Mahesh) use params for ext.args: https://nfcore.slack.com/archives/C049MBCEW06/p1721042240473589
  • Versioning module guidelines? Suggested by Arthur Gymer.

Minutes

  • Last meeting action points:

  • Test pipelines with singularity/conda:
    • crisprseq: only testing with docker in CI, but conda was failing! Singularity also failed horribly
    • ✅ singularity & conda also run on PR to master + workflow dispatch?
      • Make singularity required
      • Conda optional opt-out
    • Workflow dispatch?
    • Once a month? (only if new commits detected)
    • Only implement once nf-test is in pipelines
    • But what about pipelines that use custom docker containers(?) -> conda doesn't exist
    • But: nf-test in template will be quite a while
  • Bulk updates of modules:
    • Plan: update all modules with:
      • meta.yml -> channel structure
      • subworkflows ->
      • bio.tools identifiers
      • edam ontologies for all input/output files
      • removing defaults from conda channel
      • adding lock files
      • adding version topic channels
      • adding lint for stub tests for all modules
    • Version topic channels:
      • Maybe should be gradual overtime
      • Requries pipeline changes anyway
      • You can actually mix and match topic/non-topic channels
      • Hackathon task
  • subworkflow meta.yml restrucutre:
    • Only has name of the channel, being expanded to better match the channel construct (not just files)
    • What is the name of the overall channel XYZ - NMaming stuff: now FIGHT!
      • Structure -> Accepted by majority (only Mattias unhappy)
  • nf-test snapshot format:
    • One snapshot per file is bad!
    • All assertions in a snapshot
  • params for ext.args:
    • For next time
  • Versioning guidelines
    • Everyone think it's a good idea
    • Linked with too tools version (but docs independent) or date?
    • Linting against tools release, but can have a dev version of modules guidelines?
    • Problem: linting can be against tools dev, but this is a not a stable modules version guidelines specification could change a lot prior next tools release
    • Commits to website markdown file (too regular/fast)?
    • Major issue: how to render on website, without copying the entire markdown page every time etc
      • Also plans (although James found 🤢) to split current guideline pages into multiple pages

Action points

  • Matthias to attempt again to list modules on subworkflow page
  • Julia to include singulairty/conda tests in template (deadline: next template release)
  • Nicolas to update module guidelines to say all assertions in one snapshot (definitely not one per file)
  • Mahesh to talk about params ext.args next time
  • Matthias to think about new ways of versioning module guidelines

2024-06-28

Attendance

  • Adam
  • Anders J.
  • Anders S. P.
  • Carson
  • Christopher
  • Edmund
  • Florian
  • Gisela
  • Igor
  • Harshil
  • James
  • Jasmin
  • Jon
  • Joon
  • Jonas
  • Jose
  • Júlia
  • Lili
  • Luisa
  • Mahesh
  • Matthias D. S.
  • Matthias H.
  • Maxime
  • Moritz
  • Nico
  • Nicolas
  • Ram
  • Rike
  • Rob
  • Simon
  • Sofia

Agenda

  • Actions from last time:

    • Rike to ask Chris about office hours
    • James to invite new people/remove old people to team
    • preprocess_rna subworkflow: each maintainer come up with a renaming proposal for that workflow, and vote
    • Júlia to make tools dev branch much more strict (require tests passing etc.)
    • Rike to bring up expanding new pipeline approval rights to also matinainers?
    • James maintainrs minutes
    • Rike to add maintainers approval and namespace squatting timeout to core agenda
  • Jon's subworkflows name proposal:

    • Maxime: PREPARE_RNASEQ_READS_FOR_ALIGNMENT
    • Matthias: FASTQ_TRIM_QC_SUBSAMPLE_FASTQC_UMITOOLS_TRIMGALORE_SUBSAMPLE_FQ_SALMON_SORTMERNA
    • Matthias: MY_SUBWORKFLOW_FINAL_FINAL_3_FINAL
    • Jose: FASTQ_QC_TRIM_FILTER_REMOVE_STRANDEDNESS
    • Jose: FASTQ_PREPROCESS_FOR_RNASEQ
    • Jose: FASTQ_PREPROCESS_RNASEQ
    • Jon: PREPROCESS_RNASEQ
    • Jon: PREPARE_RNASEQ_READS_FOR_ALIGNMENT
  • Do we give Maxime his own pipeline (an extra one since he already has many toys)?

  • Discuss problems with test-datasets

    • Very messy outside genomics
    • Getting reviews are slow
    • Annoying to have to get two reviews each time (test-datasets, then module)
    • Suggestions to allow test-data files alongside test files in modules repo
      • But file polultion etc
      • See here
      • Non-nf-core test-data repos allowed?
      • Branch protection for the modules branch at least?
      • Linking external github data
  • How can we ensure the full size tests are run before a release?

    • We agreed in the past to check in PRs that the full size test is run (especially for first releases)
    • We recently had many released pipelines fail the full size test
    • Clearly we are very bad at remembering without some form of automated checking/blocking the release.
    • How can we do better?
  • Do we need more frequent/longer meetings, or sub-group meetings (e.g. modules/subworkflows)

Minutes

  • Jon's subworkflows name proposal:
    • Suggestions
      • Maxime: PREPARE_RNASEQ_READS_FOR_ALIGNMENT
      • Matthias: FASTQ_TRIM_QC_SUBSAMPLE_FASTQC_UMITOOLS_TRIMGALORE_SUBSAMPLE_FQ_SALMON_SORTMERNA
        • Con: if you add a new module, renaming is bad
        • Pro: you know that your tool of choice is in there
        • Con: doesn't allow switches inside subworkflow to allow different tools
      • Matthias: MY_SUBWORKFLOW_FINAL_FINAL_3_FINAL
      • Jose: FASTQ_QC_TRIM_FILTER_REMOVE_STRANDEDNESS (input, generic process description)
      • Jose: FASTQ_PREPROCESS_FOR_RNASEQ
      • Jose: FASTQ_PREPROCESS_RNASEQ
      • Jon: PREPROCESS_RNASEQ
      • Jon: PREPARE_RNASEQ_READS_FOR_ALIGNMENT
    • Thoughts
      • No strict naming, guidelines instead and approval?
      • What about renaming? Symlink to new one?
      • Noted shifting of feeling - subworkflows are too variable to standardise
      • Guidelines:
        • If short with single tool options, we suggest explicitly name the tools (approval not required)
          • Only main modules, not auxilary (e.g. index/sort)
        • If more generic subworkflow, multiple options or very long tools chains don't
        • Guideline: describe the the workflow in 1 sentence, and then condense based on main points
      • Generally: approach is finadability via keywords, but understanding by descirptive name
  • Maxime's proposal (variant part of sarek as separate pipeline - i.e., pipeline chaining)
    - Lots of customers people want this at Seqera, as do in other places
    - Should push forward
    - Where does documentation then go (if you start merging pipelines together, can default merge them)
    - Nextflow code should be relatively easy, but it's everything around it that is the issue
    - Working group?
    - Turn down for now, because e.g. docs maintaince burden is too high; as it's still an early PoC -> Still too many questions about what is a subworkflow/workflow
  • test-data
    • Fundmental issues with 'delete-me'
    • Need a working group
    • Need to remove a lot of stuff linked outside nf-core test-datasets (when not necessary)
    • README is very messy, makes it hard to find stuff too (puts people off)
      • e.g. just a simple protein sequence
    • Some nf-tests have huge setup, could be reduced by putting the reuqired input on test-dataset -> if we have a good place put them!
    • Agreement to have better docs: where is what? how can I produce small test files on the fly?
    • Render the documentaiton on the website
  • AWSMegatests:
    • Pipelines fail
      • One reason for pipelines failing is interactions with FUSION, which most users don't test during development. Are we clear enough about this and do people know they can trigger Megatests manually?
    • Improve documentation that everyone can submit this job
    • Can we run the action on open dev to master?
    • GET BACK TO MAKE THE BOTS DO ALL THE WORK: https://github.com/nf-core/tools/issues/2808
      • Bot should work on 2 passing PR reviews
      • Matthias will look into the action
  • Meeting frequency:
    • longer tough
    • more meetings also tough to schedule
    • vote on important topics
  • Modules bulk update:
    • Julia will organize working group

Action Points

  • Matthias to attempt again to list modules on subworkflow page
  • Jon to update guidelines based on notes above (everyone to comment - with a poll)
  • Rike to set up a working group for pushing forward daisy-chaining
  • Test-data working group set up by Simon
  • Modules bulk update group set up by Julia

2024-05-31

Attendance

  • Adam
  • James
  • Jon
  • Jose
  • Júlia
  • Maxime
  • Nicolas
  • Rike
  • Anders J.
  • Anders S. P.
  • Christopher
  • Daniel
  • Edmund
  • Gisela
  • Harshil
  • Jasmin
  • Jonas
  • Louisa
  • Matthias D. S.
  • Matthias H.
  • Moritz
  • Ram
  • Rob
  • Simon
  • Sofia

Agenda

  • Show gratitude to the module-meister
  • Check last times action points
    • Tell people to post request-review
    • Tell people short and quick merges
    • Spring cleaning! -> not this year (modules, configs ✅)
    • Office hours -> Rike to shout at Chris if he can finish in two week, otherwise we will go ahead and work it out
    • Maintainers hackathon day?
    • Yes to meeting discussion summary-> maxime to get a new zoom like channel fro mseqera to try treansicprt funcionality
    • Everyone who feel strongly abut subworkflows, should go throug nf-core/subworklows, and pick ones they don't like and ones they think have very good naming scehes, and have this read for next time
  • Good byes and invitations
  • Reviewing subworkflow naming homework
  • Be considerate of other when updating TEMPLATE.
  • Fix for modules meta.yml (https://github.com/nf-core/modules/issues/4983 and https://github.com/nf-core/tools/pull/2789)

Minutes

  • Proposals / goodbyes
    • Proposals: ✅ Igor - ACCEPTED, ✅ Carson - ACCEPTED, ✅ Florian Wuennemann - ACCEPTED, ✅ Nico Trummer - ACCEPTED, ✅ Thomas Danhorn - TURNEDDOWN, ✅ Mahesh - ACCEPTED, ✅ Luisa Santus, ✅ Joon Klaps - ACCEPTED, ✅ Lili - ACCEPTED
    • Goodbyes: Louisa, Alex P., (Matthieu?), Daniel Lundin
    • More ideas for women please shout?
    • Apologies people: please consider
  • Subworkflow naming:
    • Adam's idea:
      • remove tools (final suffix) OR input file
        • input + verb/noun
        • bed_scatter_bedtools -> bed_scatter
      • but what if subworkflow with same function but different tools
        • e.g. bam_deduplicate_samtools vs bam_markduplicates_picard
      • Adam's issue: long isn't bad, just more info
      • Jon: main issue is can have so many tools (so suffix gtets very long)
      • Jon: just be more specific: preprocess is bad, but rather e.g. trim, removerna, -> semantics in names is bad
  • Tools team update: nf-core version compatibility will now be tracked in pipelines
  • Ken Brewer developing a minimal nextflow template (by Monday?!) with nf-core compatibility features
    • Júlia and tools team is already starting with this from nf-core side (new TUI has turn on/off functionality for example)
    • But will take time until released, needs a lot of testing: please make a lot of pipelines!
  • Tools team: Modules meta.yml is not well formatted - Jon made proposed (made on issue 4983)
    • Jose makes points on parsing
  • Maxime: wants additional default fields into modules (bibtex as channel), he will speak to Rike/Júlia
  • New pipeline proposals:
    • Core team is slow, give Maintainers approval permissions (one core, and at least one maintainer)
    • Rike to add to core
  • Release reviews?
    • Keeping up to date so seems OK
  • namespace squatting (e.g. scflow) -> release deadline before otehrs can take over
    • Core team has already discussed, debating how long to require

Actions

  • Rike to ask Chris about office hours
  • James to invite new people/remove old people to team
  • preprocess_rna subworkflow: each maintainer come up with a renaming proposal for that workflow, and vote
  • Júlia to make tools dev branch much more strict (require tests passing etc.)
  • Rike to bring up expanding new pipeline approval rights to also matinainers?
  • James maintainrs minutes
  • Rike to add maintainers approval and namespace squatting timeout to core agenda

2024-04-26

Attendance

  • Adam
  • Edmund
  • James
  • Jasmin
  • Jon
  • Matthias H.
  • Maxime
  • Nicolas
  • Simon
  • Sofia
  • Anders J.
  • Anders S. P.
  • Christopher
  • Daniel
  • Gisela
  • Harshil
  • Jonas
  • Jose
  • Júlia
  • Louisa
  • Matthias D. S.
  • Moritz
  • Ram
  • Rike
  • Rob

Agenda

  • Spring cleaning?
    • Matthias H to lead
    • New Q from @jfy133 from last time: When? 😬
  • Maintainers team review?
    • Maxime to lead
    • New Q from @jfy133 from last time: When? 😬
  • @jfy133: I've been getting questions/comments about sometimes long review times in pipeline reviews and smaller pipelines; guidlines/suggestions/strategies to improve?
  • @jfy133: further subteams (responsibility for: configs, modules, new-pipelines etc?)
  • Core proposal: Office hours with rotating volunteers
  • Subworkflow naming guidelines

Minutes

  • PRs are mounting 😱
    • Particularly on modules
      • Check repo: 3
      • Github notficaitons: 3
      • Request-review slack: 6
      • Emails: 2
    • How to improve this?
    • #request-review keeping up to date
    • Maybe do need more ruthless with auto-closing stuff
    • Need spring cleaning
  • OFFICE HOURS
    • Everyone on board, x1 month is probably good for everyone
  • Reviewing is slow
    • Particularly large templates complicated
    • Need to make smaller
    • Maybe office hours also for tradiginv revierws
    • post-tools special office hours?
    • Better guidance on short and requick
    • Maybe do an annoucnemtn
  • Summary end of each maintaniers meeting of discussion and publish on PI
    • All agree!
  • subworkflow naming
    • Too much complexity if strictly follow subworkflow structure
    • Too many tools in some cases
    • Risk of people being too broad -> 'rnaalign'
      • Which ? All?
    • Discoverability is too low
    • But what if you add another tool? Have to change the name
    • Already problem with deprecation
    • Risk of laziness
      • Name has to at least been very specific
    • nf-core subworkflows SHOULD be as short as possible, not huge mega ones
      • You can embed subworkflows in local umbrella subworkflows
      • Don't always agree -> want to be able to copy paste between different, if it's too much embedding too complicated
    • Maybe each subwk must have a DAG/workflow grpah, if too complicated shouldn't be included
      • Popular!
      • Keywords isn't descriptive enough
      • Graphical represntation will help

Action points

  • Tell people to post request-review
  • Tell people short and quick merges
  • Spring cleaning!
  • Office hours
  • Maintainer shackathon day?
  • Yes to meeting discussion summary-> maxime to get a new zoom like channel fro mseqera to try treansicprt funcionality
  • Everyone who feel strongly abut subworkflows, should go throug hnf-core/subworklows, and pick ones they don't like and ones they think have very good naming scehes, and have this read for next time

2024-03-29

  • Cancelled due to Easter holidays for much of Europe

2024-02-23

Attendence

Agenda

Actions points from last meeting:

  • Volunteers wanted: Need group leaders for the hackathon in march
    • Maxime will ask people
  • Deprecation guidelines: https://github.com/nf-core/website/pull/2217
  • Update docs with latest ideas for file name capture or snapshots cf qualimap + @jfy133 PR
    • Nicolas to have a look
  • Agreeing on a naming convention for the tests
    • Nicolas will lead the focus group
  • (@jfy133) Spring cleaning?
    • Matthias H to lead
  • (@jfy133) Maintainers team review?
    • Maxime to lead

2024-01-26

Attendence

@jfy133, @maxulysse, @kOcBx-beQZW6WOsI5UVQcA, Ram, @emiller, @YO3oG0hdSw2Gv5IWHc0nfA, Jasmin, Jon, Jonas, @JoseEspinosa, @jmir, @mashehu, Nicolas, Simon, Sofia,

Agenda

Action Points

  • @Mahesh will start implementation pattern focus group
  • @jfy133 add to approved new pipeline applciations instructions to join #pipeline-maintainers
  • @maxulysse (with help from Simon) will start a focus group for conventions and guidelines

2023-12-15 - Cancelled

Agenda

2023-11-24

Agenda

  • nf-test:
    • modules now require nf-test (Simon Pearce) - only warning now
    • We absolutely should snapshot version
  • Handeling large scale changes, i.e. switching to environment.yml (James):
    • tools are not released yet to handle this
    • docs are not up to date
  • CODEOWNER on modules:
    • Auto-request reviews
  • Ops and teams on GitHub (Edmund):
  • POSTPONED: Improvement on the modules regarding multiple meta maps (Wait until Mahesh is a maintainer)
  • Best Practices for multiple input:
  • Refactoring blog post (Sateesh)
  • New Nextflow version not mounting $HOME:
  • Tests with differences between docker/singularity/conda (grrr bam files)
  • Modules using meta.X

Actions point

  • Run tests for all modules in spring cleaning
  • Tools to handle the environment.yml
    • Docs
  • Coordinate, announce/document and TEST changes all at once
  • Document the NXF_SINGULARITY_HOME_MOUNT issue: add https://nf-co.re/docs/usage/troubleshooting for now.
    • Volunteers needed!
  • Let's think about how to deal with bam/vcf files in nf-tests
    • Plugins?
    • Snapshot the filename
    • Check if not empty
  • Let's think about the snapshot files with nf-test and how to assess/read them
  • Last 2023 meeting: december 15?
  • common fields:
    • meta fields
    • params
    • samplesheet fields
    • emit/take || input/output fields
  • Governance:
    • what a maintainer is and entails too?
  • Think about github teams and check what can a maintainer do
  • Tests more in ops

2023-10-27

Who

  • @maxulysse
  • @nvnieuwk
  • @mashehu
  • Jon manning

Focus is hackathon follow up: nf-test migration and refactoring pipelines.
@mashehu is leading the infrastructure front
@maxulysse is leading the pipeline refactoring front
@nvnieuwk and Jonathan Manning will be coordinating and managing the general effort and making sure we stay on target.

2023-09-22

When/Where

https://zoom.us/j/93254682925?pwd=TW9GdU1YeU1EVnE2aFMvdDNkY2t1Zz09

Who

Action Points from last time

  • (Everyone) Give feedback for Rob on the guidlines conslidation
  • (Ram) will update modules docs on genomic references ideally having meta next week
  • (Rike) Will open a PR on PR etiquette docs > Not done
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

Agenda

  • Hackathon
  • Tools release is coming
    • Be ready for sync PRs
  • Ideas for refactoring:
  • Contributing guidelines + Issue/PR templates
    • Especially on nf-core/modules
      • Come and ask for review on Slack
  • Bring more subworkflows in nf-core/modules
    • Which fields should we recommend in the meta map?
    • Common params to be shared in between pipelines
    • Common fields within the samplesheet
  • Pipeline specific configs by default in TEMPLATE (cf: https://nextflow.slack.com/archives/C02TPTREJ03/p1689844381095889)
    • Need to create a file in nf-core/configs for each pipeline
  • Generally: any issues/problems/?
    • Module with diverging meta map names:
      • Consensus: No. Was cleared up with the author
    • How should we deal with multiple samplesheet options?
      • Allow multipe different input parameters?
      • Update nf-validation to handle conditional sample-sheets

Action Points

2023-06-30

When/Where

Zoom: https://eu01web.zoom.us/j/64977955782?pwd=YzRVRHArcGZ4MkN3NllZT2pRanhYQT09

Who

@jfy133
@jasmezz
@nvnieuwk
@maxulysse
@jmir
@FriederikeHanssen
@JoseEspinosa
@louperelo
@mashehu

and

Rob
Daniel
Sofia
Ram
Matthias D
Sateesh
Edmund
(and whoever else I forgot)

Agenda

  • Action points from last time
    • guidance documentation consolidation (@robsyme)
    • nf-test 'community' adoption guinea pigs: test documenation/slides for 'self-' incorporation into their pipelines (Jon/Julia/Jose/Louisa)
      • Overall going well, Sateesh will take a break now, as in a good 10 pipelines or so -> He will do a bytesize to open it to other developers to do it themselves (with a sample pipline example)
      • Adding to template? For nf-core/tools 2.10 (implementation during hackathon)?
    • Volunteers to test nf-validation's Channel.fromSamplesheet() (@jmir & Rike/Sofia/Louisa)
      • Tested/being in funcscan and sarek (some errors), but won't be in template until 2.10 (but you can ask Julia/Nicolas/Matthias if you want it earlier)
    • meta for genomic references as general recommendation (Ram)
      • Next week
    • Next meeting: discuss summer break
  • (@jfy133) How is everything going? Overloaded/Underloaded? Feeling good or bad about anything?
    • Anyone feeling uncomfortable in their role?
    • Everything is fine! Always contact James/Rike/Maxime if you have any problems and want to report privately
  • (@FriederiekeHanssen) Should we add guidelines on how to make a PR? (i.e. one feature per PR, https://hugooodias.medium.com/the-anatomy-of-a-perfect-pull-request-567382bb6067
    https://www.atlassian.com/blog/git/written-unwritten-guide-pull-requests
    https://www.freecodecamp.org/news/git-best-practices-commits-and-code-reviews/)
  • (@jfy133) Publication guidelines!
  • (@jfy133) hicar release review 😬
    • Julia, Simon, Edmund, Jose
    • Issue: people developing on fork! (e.g. HiC)

Action Points

  • (Everyone) Give feedback for Rob on the guidlines conslidation
  • (Ram) will update modules docs on genomic references ideally having meta next week
  • (Rike) Will open a PR on PR etiquette docs

Meeting 2023-05-26

When/Where

Gather! (probably or not)

Zoom: https://eu01web.zoom.us/j/67778335655?pwd=MXQwaGZKT1JMeXFJTWdiWGRpQWVsQT09

Who

@asp8200
@JoseEspinosa
@sateeshperi
@jfy133
@jmir
@ramprasadn
@nvnieuwk
@louperelo
@maxulysse
@pinin4fjords

Agenda

  • Last time
    • @robsyme to update documentation structure to consolidate all guidelines files into one place
      • Happening after this meeting
    • @sateesh to post in #pipeline-maintainers pinging main devs of nf-test PoC pipelines
      • Added to fetchngs, methylseq, rnaseq, ampliseq
      • Has to be done on a per-pipeline basis (can't do mass update) -> somewhat custom per pipeline
      • Jose Question: minimum default requirements (assertions: version, test success etc.), then pipline dev can extend as they want?
    • @mashehu and @jmir to #bytesize latest tools release
      • Will wait until next release that has more cool stuff
  • nf-validation & Channel.fromSamplesheet() @jmir
    • To be added to next release
  • Adding meta to genomic reference files
    • Reminder to everyone: when reviewing please remind people to do this!
    • It's becoming tricky with subworkflows
  • General: How's being a maintainer going?!
    • Comes in bursts
    • Maxime will get us more swag ;)
  • sc-flow discussion?
    • They sound willing, nothing ever happens

Action Points

  • nf-test 'community' adoption guinea pigs: test documenation/slides for 'self-' incorporation into their pipelines
    • This time
      • Jon: differntialabundance
      • Jose: atacseq or chipseq
      • Julia: crisprseq
    • Next time:
      • Louisa: funcscan
  • Volunteers to test Channel.fromSamplesheet()
    • Sarek: Rike
    • Funcscan: Louisia
    • Taxprofiler: Sofia
  • meta for genomic references as general recommendation
    • Ram to add to guidleines (if missing)
  • Next meeting: discuss summer break

nf-core/maintainers meeting

Meeting 2023-04-28

When/Where

2023-4-28 15:00-16:00 CEST, gather.town

Who?

@mashehu , @jfy133 , @louperelo, @ramprasadn, @pinin4fjords, @jmir, @jasmezz, @robsyme, @christopher-mohr

Agenda

  • Review last meetings TODO

    • ✅ @mashehu to update website to add all maintainer listed in last meeting
      • PROMISES TO DO IT AFTER THIS MEETING
          • Need to update list of maintainers on website (during spring cleaning)
    • ❌ Guidelines consolidation
  • new tools release upcoming

    • Today or next week
    • Main changes:
      • README restructure;
      • container registry param (default quay, but can use own) - allows more flexible selection
      • python 3.7 is dropped (as is nf-core/modules)
      • JSON schema for meta.yml -> will be linted
  • If time: review orphaned questions

    • General 'Custom' R/python etc. scripts in modules? Y/N?
      • Issues: no versioning? QA of the custom (variable) language scripts more dificult? Would require a changelog etc.; Discoverability - when to know when reused (or should be reused thus pushed to a nf-core/module!)
      • If used in more than one pipeline it should be moved from local to a custom module (@emiller will make an issue to try to collect these)
        • Local modules keywords somehow? Then can periodically screen for popular custom modules/scripts that get copy-pasted to other pipelines and could propose to make a full module
      • Difficult to find a definition of when should move from a pipeline-local to a full-module
      • GUIDANCE: Ok if it's a one off, but if getting more and more used should move to separate package.
        • So, e.g. if deseq/? grows more modules, the templated scripts should be pushed out to separate package??
  • nf-test pipeline PRs for review:

  • fetchngs

  • rnaseq

  • methylseq

  • demultiplex

  • viralrecon

  • ampliseq

  • nascent

  • viralintegration

Action Points

  • @robsyme to update documentation structure to consolidate all guidelines files into one place
  • @sateesh to post in #pipeline-maintainers pinging main devs of nf-test PoC pipelines
  • @mashehu and @jmir to #bytesize latest tools release

Meeting 2023-03-31

When/Where

2023-03-31 15:00-16:00 CET , nf-core office gather.town

Who?

Agenda

Meeting was cancelled because everyone was hackathon-tired

Meeting 2023-02-24

When/Where

2023-02-24 15:00-16:00 CET , nf-core office gather.town

Who?

@jfy133 @pinin4fjords @matthdsm @jmir @nvnieuwk, @mashehu, @louperelo, @ChrisHakk, @emiller

Agenda

  • ✅ Announcements
    • ext.args/value channel meeting on 28 February 11:00-12:00 CET
  • Review last meetings TODO (@jfy133)
    • ❌ @mashehu to update website to add all maintainer listed in last meeting
      • PROMISES TO DO IT AFTER THIS MEETING
    • ✅ Rike to add a comment and pin about unrolling github previews on #request-review
    • ✅ separate meeting to be had to hash out the two separate args issues (multi-tools, mandatory non-file inputs)
    • ❌ Guidelines consolidation
  • ✅ Review PR-to-master release project board (@jfy133)
  • ✅ All ready for spring cleaning (@jfy133)?
  • ✅ Should a stub section be required for new modules? (@matthdsm, @maxulysse)
    • No 'nays' -> (Newbie?) hackathon task!
  • ✅ JSON-schema for modules meta.yml PR
  • If time: review orphaned questions
    • General 'Custom' R/python etc. scripts in modules? Y/N?
      • Issues: no versioning? QA of the custom (variable) language scripts more dificult? Would require a changelog etc.; Discoverability - when to know when reused (or should be reused thus pushed to a nf-core/module!)
      • If used in more than one pipeline it should be moved from local to a custom module (@emiller will make an issue to try to collect these)
        • Local modules keywords somehow? Then can periodically screen for popular custom modules/scripts that get copy-pasted to other pipelines and could propose to make a full module
      • Difficult to find a definition of when should move from a pipeline-local to a full-module
  • Need to update list of maintainers on website
    • Keep a list of current plus previous maintainers
    • Stickers
      • If anyone has a concept for a hex design please send it to Chris and her will get some made
  • Automated Conda lock file creation when modules are created, or conda settings updated?
    • Jon to create feature request on tools repo

TODO

Meeting 2023-01-27

When/Where

2023-01-27 15:00-16:00 CET , nf-core office gather.town

Who?

@maxulysse, James, Jon, Jasmin, Matthias H, Louisa, Moritz, Nicolas, Gisela, Simon, Júlia, @emiller, Matthias dS, Rike, Daniel, Chris, Sofia

TODO (for next time)

  • @mashehu to update website to add all maintainer listed in last meeting
  • Rike to add a comment and pin about unrolling github previews on #request-review
  • separate meeting to be had to hash out the two separate args issues (multi-tools, mandatory non-file inputs)
  • Standardise guidelines locations
    • @jfy133 / @KuylkJ0AS0-limIJRUVsnw during spring cleaning?
    • Or someone earlier?

Agenda

  • Review previous meeting TODO

    • ✅ Rike:
      • ✅ create poll for next meeting
      • ✅ Write a message in request review about the new review emojis
    • ❓Matthias H.: Add name/affiliation to website -> CAN DO NOW
    • ✅ Everyone: add name/affiliation in list (in no particular order!)
    • ⚔️ Maxime: Remove Github PR previews from Slack
      • Can't do it, asking for another core member to help out
      • Couldn't do it on a single channel, just use markdown instead
    • ✅ Gisela: setup project board for PRs to master
  • ✅ Review spring cleaning plan from Chris/James

    • Add event to website (but don't advertise)
    • Seperate meeting notes over here
      Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →
      spring clean hackmd
  • ✅ nf-validate update from Júlia > meeting notes

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
EMERGENCY TOPIC
Bring your tomatoes
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

  • Should we allow 'named' ext.args? (ext.args.<names>) for mandatory non-file inputs OR require mandatory non-file inputs: channels
  • Should/how/where we document 'named' ext.args
args pros args cons val channels pros val channel cons

Meeting Notes

  • ❓Matthias H.: Add name/affiliation to website -> CAN DO NOW
  • ⚔️ Maxime was voted down on remove Github repo previews from Slack
    - Couldn't do it on a single channel which was the main issue (request-reviews), just use markdown instead (or manually close the preview)
  • Everyone reminded to start adding release reviews to projets board
  • Guidelines locations
    • Matthias can look into adding check boxes
    • James, Rike: Sort out markdown files during spring cleaning, but if someone beats us to it ;)
  • Un-nf-corey pipelines (prompted by https://github.com/nf-core/viralintegration/pull/52)
    • Consensus was a pre-v1.0.0 release
    • Contributor must have record of standards-compliant contributions and promise to fix it up
    • Pipeline will eventually get archived if that doen't happen

TODOs

Meeting 2022-11-25

When?

Friday, 25 November⋅13:00 – 14:00 (CET), nf-core office gather.town

Who?

Rike*, James, @maxulysse, Matthias De Smet, Jon Manning (Healx Ltd, Cambridge), Nicolas, Daniel Straub (@d4straub)*, Moritz Beber, Gisela Gabernet (@ggabernet)*, Chris Mohr, Anders Jemt, Jose Espinosa-Carrasco, Louisa Perelo (@louperelo)*, Ram, Harshil, Simon Pearce, Matthias Hörtenhuber, Anders Sune Pedersen (Danish National Genome Center), Matthieu Muffato (@muffato, Wellcome Sanger Institute)

* Quantitative Biology Center (QBiC), Eberhard Karls Universität Tübingen, Germany

TODOs (for next time):

Agenda

General Topics

  • Definition of roles/tasks
    • Members: Generally quite active across different topics (modules,subworkflows,pipelines,) > Recognize this effort & formalise it
    • Structure of group:
      • Flat structure
      • Chairs (number, who etc.: interim - Friederike, James, @maxulysse)?
      • Assign responsible persons for specific tasks (e.g. test data clean up), decide at the time
    • Suggested tasks:
      • Review PRs
      • Review release PRs (including first release)
      • Module reviews
      • Make sure the right people have access to the right repos (core)
      • On-boarding people: check #github-invitations
      • Clean up: e.g. Manage test data, stale PRs
      • Update our own dockerhub hosted containers (ensemblvep, snpeff, bclconvert, bcl2fastq: @matthdsm + @maxulysse)
      • Google calendars? or is this more an infrastructure or core tasks? (either way: @maxulysse)
  • Contribution requirements
    • From the core side: not a strict monitoring
    • Minimum Hours/Maximum Hours: i.e. how to make bosses less scared:
      • 1-2 hours week recommended but not strictly required. Breaks are also ok (e.g. sick, parental leave, vacation).
      • Not own works, but helping others
    • Joining maintainer meetings on regular basis
    • Monitor slack channels: request-review, github-invitations, help, nostupidquestions
  • What info to have on website (to formalise role):
  • How regular to meet?
    • Monthly meetings -> TODO: find a date/time
  • Who presses 'merge'?

    Edmund Miller Emphasis on keeping that it's okay to not be perfect, stuffs going to break but we'll learn from it.

    • Restrict merge rights to maintainers for common repositories. For pipeline merge rights stay open.
    • Let people merge their own PRs
  • Finalise @maintainers groups i.e., is everyone already in the groups etc! (slack/github etc.)
    • Core team todo after meeting
    • Please ping Rike if you are not in the slack group
  • PRs reviews:
    • Remove long github printing for the channel -> TODO Maxime
    • Release PRs
      • Often disappear in request-review channel and then don't get reviewed in time
      • Review buddies?:
        • How to include new people?

        Friederike Hanssen maybe not such an issue, as these are for more experienced people

      • Collect release reviews on project board?
        • Automate that e.g. with GitHub Actions workflows -> PR to add master can automatically be added to Project Board
        • Google sheet > Board view is spread sheet like enough
        • Assign yoursefl to the PRs you review for tracking
    • smaller PRs etc:
      • Image Not Showing Possible Reasons
        • The image file may be corrupted
        • The server hosting the image is unavailable
        • The image path is incorrect
        • The image format is not supported
        Learn More →
        Looking at PR
      • :checkmark: Approved PR
      • :merge: Merged PR

      James Fellows Yates get github PR icons as emojis?
      Mashehu already there

Spring Cleaning!

Organised by: James & Chris Hakkaart

When:
- week of 07th of March 2023, before the hackathon
- Kick-off meeting on the monday, distribute tasks, form smaller groups
- Spread throughout the week as time permits
- Closing meeting on Friday

Idea:

Lots of old issues, PRs, test data etc. that we need to clean up once a year as maintainers.

What should we do?

  • Archive old pipelines?
  • Close stale PRs on nf-core/modules:
    • For common repos: modules, test-data, PRs sometimes stay open for a long time
  • Close stale issues on nf-core/modules

    Edmund Miller Stale issue bots that close the issues are bothersome, but something to tag them stale might be nice.

    • For pipelines possibly not an issues
  • Standardise existing modules/subworkflows metadata tags
  • Clean up test data
    • Add meta.ymls to test data for the modules repository

      James Fellows Yates this sounds like development rather than spring cleaning?

  • Check if all installed modules are actually used inside each pipeline
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →

    Friederike Hanssen this sounds like something that could be supported with linting

  • Remove anything linking to DSL1, it won't be supported anymore
  • Go through docs and check they are up-to-date
  • Remove existing groups on github/slack
  • NEW: cleanup/standardise old subworkflows against updated guidelines
  • NEW: S3 file cleanup (deleting old work dirs etc.)
  • NEW: Clean up instution contacts on the website

Post-hackathon events:

  • Possibly second event after hackathon
  • To deal with time commitment: split into two groups, one before, one after

Topics to discuss in the upcoming months

  • General 'Custom' R/python etc. scripts in modules? Y/N?
  • Seeds in modules - leave to pipeline developer or enforce at module level
  • Specifications for a groovy-native samplesheet checking (separate meeting)
    • Current requests:
      • Order agnostic (don't care where, just have to be present)
      • Check across columns
      • Allow additional columns (that get ignored)

    Friederike Hanssen Quite some work has been done on this in the past, would be good to collect that info/difficulties there first (I think Phil was pretty involved there, I think slack channel #sample-sheet)

  • Guidelines as to what warrants/constitutes a workflow
    • minimum number of tools?
    • application?
  • How to test using nf-test
    • Specific requirements for modules, subworkflows, pipelines (workflow succes, md5sums, trace size)

    Edmund Miller Should be a badge of honor they're employing a nf-core maintainer, like Microsoft employing Guido
    Friederike Hanssen This would make for a good bytesize talk

  • remaining DSL1 pipelines need to be converted soon

Orphaned Qs

If still relevent

Select a repo