--- tags: NGFF, community-call --- # OME-NGFF community call: 2025-02-21 Please paste this into the Zoom chat as new people join: :::warning Welcome to the community call. Live notes for the session are available in https://hackmd.io/HdRQgm3oT1aCGxBXhL1f3Q Where possible, help to structure the notes for later publication rather than commenting in Zoom's chat. Thanks! ::: ## Code of conduct The OME community is open to everybody and built upon mutual respect. Please take the time to review the code of conduct below. https://github.com/ome/.github/blob/master/CODE_OF_CONDUCT.md ## Preliminary agenda (not for notes) | [discourse thread](https://forum.image.sc/t/ome-ngff-community-call-starting-2025-right/108431) | [zulip thread](https://imagesc.zulipchat.com/#narrow/channel/328251-NGFF/topic/Community.20call.20reminder.3A.20tomorrow.20Friday.2020th.20of.20February) | - Updates (30 minutes) - [slides](https://docs.google.com/presentation/d/1J0_axLXrkA32NRL3AIlBygfU9nTGybVIidpq_Dn0Hhs/edit?usp=sharing) - Zarr v3 - OME-Zarr RFCs - OME-Zarr 1.0 - Community (calls, etc.) - Open discussion (30 minutes) - ... Please list items here before the meeting 📝 ... ---- <details><summary>First session</summary> ## OME-NGFF community call (10:00 CET; 21 Feb 2025) ### "User registration" Session 1 | Name | Institute | Bluesky Handle | Mastodon Handle | GitHub Handle | |---------------|-----------|----------------|-----------------|---------------| | Copy | and | paste | me | | | Josh Moore | German BioImaging, e.V. | @joshmoore.bsky.social | | joshmoore | | Sébastien Besson | Glencoe Software | | | sbesson | | Jan Eglinger | FMI Basel | @imagejan.bsky.social | | imagejan | Norman Rzepka | scalable minds | @normanrz.com | normanrz.mastodon.social | normanrz | | Davis Bennett | independent | | | d-v-b | | Jens Wendt | NFDI4Bioimage | | | JensWendt| | Tom Boissonnet | HHU Düsseldorf | @tom-bssnnt.bsky.social | | Tom-TBT | | Koji Kyoda |RIKEN BDR|||openssbd| | Florian Aymanns | EPFL | | | | | Ken Ho | Crick | drkenho.bsky.social |@kenho@fediscience.org || | NOT HERE | :smile: | | | | ### Session 1 Notes * Presentation (10:05-10:15) * Open Discussion - Joost: - engagement e.g. on image.sc -- more clarity on how people can contribute. What's needed? Purpose for people to join - How the RFCs and the documents are published on github -- you have to be tech savvy to read them. A rendered URL. ... but specific change page. (perhaps a list). Would make it easier for "outside" contributors who have ideas what to contribute but are not as deeply familiar with the tech-stack as the folks who are already involved in the project. - Virginie: interested to here crowds opinion on how people can contribute. alternatively, interested in having these meetings be much more welcoming: beyond the technical things but here is what the community is, come and meet people. - just focusing on contributions pre-selects for a certain group of people - e.g. people who need to convert their data - Norman: agreed. these meetings could be more than specification related. More room now to use the format - +1 for increasing the frequency to quaterly or every other month. actively invite people to share. - an hour, 20 minutes to showcase work or whatever. still spec talks but they don't have to fill the meeting. - Josh: challenge was more like a working group. Similar to meshes. - Seb: showcase shows what's *possible* rather than focusing on what's missing. Pat on the back. (user story) - Norman: agreed, we need to stop talking (only) about the problems - Josh: OME-Zarr figures as a quick showcase - Koji: meeting with Nikon developers. Could ask about OME-Zarr support? - Very conservative - Timeline? - Commenting on 1.0 RFC - https://forum.image.sc/t/ome-zarr-reading-writing-with-c-libraries/89932 - Davis: tensorstore may be a big ask (also doesn't write Zarr groups) - Joost: OSS control for EM "SBEMImage" (https://github.com/SBEMimage/SBEMimage) written in Python - now support OME-TIFF. Could also support OME-Zarr through that. - Seb: Glencoe looking at private data (almost exclusively) - nice image.sc blog post about which viewers support private access ("plug your credentials") - how do we handle access control? one bucket with one password! - our tool ends up making it easy to start accessing data I shouldn't (server has credentials) - not specific to OME-Zarr but might end up as an RFC ("accessing data through a server") - is there a way to put mapped token to prove you have rights to access data? - Norman: happy to chat about it. have definitely dealt with it. - Josh: another ongoing project - https://oscars-project.eu/projects/bio-codes-enhancing-ai-readiness-bioimaging-data-content-based-identifiers - https://core.iscc.codes/ - capture a checksum (semantics+metadata+data) - Norman: Collections - hackmd to capture user stories (motivations) - does anyone have user stories? Get in touch with Norman on Zulip - next we start writing it up. - breaking change - FYI: Hackathon organization with internal projects -- https://github.com/BioImageTools/ - FYI: mental note for hackathon in Zürich in November - End: 10:50 </details> ## OME-NGFF community call (17:00 CET; 21 Feb 2025) ### "User registration" Session 2 | Name | Institute | Bluesky Handle | Mastodon Handle | GitHub Handle | |---------------|-----------|----------------|-----------------|---------------| | Copy | and | paste | me | | | Josh Moore | German BioImaging, e.V. | @joshmoore.bsky.social | | joshmoore | | Will Moore | Dundee Uni / OME | will-j-moore | | | | Ian Hunt-Isaak | earthmover | ianhi | |ianhi| | Eric Perlman | | @perlman.bsky.social | @perlman@urbanists.social | | Joel Lüthi | BioVisionCenter, UZH, CH | @joelluethi.bsky.social | | jluethi | | Dan Toloudis | Allen Institute for Cell Science | | | toloudis | | Ken Ho | The Francis Crick Institute | drkenho.sky.social| @kenho@fediscience.org| DrKenHo-crick | | Kiya Govek | The Jackson Laboratory | | | govekk | | Fernando Cervantes | The Jackson Laboratory | | | fercer | | Kabilar Gunalan | Massachusetts Institute of Technology | @kabilar.bsky.social | | kabilar | | Erick Ratamero | The Jackson Laboratory | @ratamero.microscopy.codes | idlethumbs.social/@ratamero | erickmartins | | Cameron Fraser | Allen Institute for Cell Science | | | frasercl | | John Bogovic | HHMI Janelia | @bogovicj.bsky.social | | bogovicj | | David Ackerman | HHMI Janelia | | davidackerman | Marwan Zouinkhi | HHMI Janelia | @mzouink.bsky.social | | mzouink | | Melissa Linkert | Glencoe Software | | | melissalinkert | | Rebecca Vorimo | HHMI Janelia | | | rinva| | Mark Kittisopikul | HHMI Janelia | @markkitti.bsky.social | https://fosstodon.org/@markkitti | mkitti | | Diyi Chen | HHMI Janelia | | | dchen116 | | Damir Sudar | Quantitative Imaging Systems LLC | @dsudar.bsky.social | | dsudar | ### Session 2 Notes - Erick: https://www.bioimagingnorthamerica.org/events/ome-2025-community-meeting/ - Mark: 1.0 prerequisites - Zarr v3 complaints - "Under construction" needs to go away. Yup. See https://github.com/zarr-developers/zarr-specs/pull/330 - Cross testing has posed some issues - e.g. ts writes v2 such zstd isn't readable by other implementations like zarr-python - Norman: Yes! definitely put on the list 1.0 - zstd becoming a prominent codec (zarr-python v3 default) - how do we enumerate codecs in OME-Zarr - Norman: see https://github.com/zarr-developers/zarr-extensions - Ken: if "more reliable then extensions"? - Norman: there are different goals. mine: - every version has been breaking so far - want to get to the point where we can not break for a long time - KH: :+1: for something more stable. - Dan: - 1. sample implementation that's really robust will go a long way to make it feel serious. - An "implementation" should codify all the MUSTs, SHOULDs, MAYs etc. from the spec.. - 2. metadata: official way of dealing with more metadata, even if it is just one field that says, "put it here" - or a statement that "you're on your own" - 3. thinking a lot about collections, distinction between zarr groups and sources from different filesystems. (perhaps completely required in 1.0) - David: - reference implementation in https://ome-zarr-models-py.readthedocs.io - v0.4 done; working on v0.5. Love that to be a community project. - outcome of the last hackathon - Norman: - discussed collections at the hackathon. working on stories to motivate - working towards a RFC. definitely part of 1.0 - get in touch on zulip, etc. - likely to get a WG/meeting series started soon - Luca: regarding transforms - with Wouter, contributing an implementation to ome-zarr-models-py - working other than displacements... (currently without pydantic) - please say if development needs prioritizing say the word. (April? Yes!) - Abbas - as an end user, very interested in transforms. (webknossos, etc.) - need transforms for stitching - Luca (in chat): https://github.com/multiview-stitcher/multiview-stitcher - Joel (in chat): That library is super cool, we've had a lot of fun using it - don't want to duplicate the large datasets. - top priority - John - thanks Dan + others for reviewing RFC transforms - currently working on what parts are necessary vs optional for implementation - currently working on complete set of examples - Josh - Displacement field might be something in its own page so fully optional - Joel - Community calls are helpful for people to catch up on what's going into RFCs - Meetings - Erick: need to change how people view this meeting - wasn't attending because viewed it as heavily technical spec meeting - Virginie taking over community-facing meeting series - Misc - Abbas: infrastructure based on zarr - Working on new parallelization for chunk-wise compute, but having trouble getting in touch with dask developers - Davis: DON'T set Dask chunk size to be same as Zarr chunk size. Overhead of 1 millisecond for every task. - Dask chunks should not be as small as Zarr chunks - Davis: dask forums aren't very lively, but ping people directly (e.g. Davis) on github repo - zulip is also a good option for these kinds of questions - Will: what's the Python reference implementation for working with arrays? is everyone using different things? - Josh: likely a dozen different tools... - 2023 hackathon outcomes: https://forum.image.sc/t/outcomes-of-the-next-generation-bioimage-analysis-workflows-hackathon/88733 - David: next steps need to be limited scope - started with validating metadata, before moving on to writing metadata - would be ok with adding support for reading/writing data to ome-zarr-models-py - https://ome-zarr-models-py.readthedocs.io/en/stable/ - Ken: Java for getting users to adopt... what's the latest on java development for ome-zarr? - John: progress on Zurich hackathon last year for reading/writing zarr 3 from java. can expect Fiji support for that "soon" - smaller community working on java than python, and Fiji maintenance is a big task