---
tags: NGFF, community-call
---

Please paste this into the Zoom chat as new people join:
:::warning
Welcome to the community call. Please be aware that this session may be recorded. Live notes for the session are available in https://hackmd.io/Ndb5IHRmQn2PCCNBLkG-fQ?edit Where possible, help to structure the notes for later publication rather than commenting in Zoom's chat. Thanks!
:::

# NGFF Community Call 2021-02-23

**See:** [Recordings](https://downloads.openmicroscopy.org/presentations/2021/community-call-2021-02-23/), [Connection information](https://forum.image.sc/t/connection-information-for-next-gen-call-on-feb-23rd/48907) and [Previous meeting notes](https://hackmd.io/X348vzCaTRSmIpsa3dK-Sg)

## Using this document

This document is a place where you can help drive what needs discussing on the 23rd.
Add your thoughts, needs, etc. or even new sections if need be.
If there's an idea already in place that you like, give it a :thumbsup:

If you are unclear about this document, **just add a question here** and someone will tidy it up or get in touch:

* no problems yet? Excellent! :surfer:

## Brief agenda
* Introductions `20m`
- `min(60s, 1200s/attendees)` per person
* Zarr and OME-Zarr status (Josh) `<15m`
- Zarr: funding & implementations
- OME-Zarr:
- 2020: for those who weren't at I2K
- 2021: 3D, nested storage and performance
- OME (FYI): Support in IDR and hiring
* Any outstanding community items `~25m`
- Statuses from other efforts
- Questions about statuses, goals, etc.
- _etc. **It's good to hear from you!**_
* Specifications `45min+`
- Quick run through of the specifications list (**additions?**)
- [Roadmap board][roadmap] as a place to organize?
- Writing & implementation specifications
- Compatibility
* Next & future steps `~15min`

## Specifications to be discussed

* [labels]
* [polygons]
* [meshes]
* [collections] of multiple images
* [PhysicalSize] + position
* [scales]
* [remote] links
* [metadata]
* tabular data
* ... please add more here...

## "User registration" Session 1

| Name | Institute | Twitter Handle | GitHub Handle |
|------------ |---------------------- |---------------- |--------------- |
| Copy | and | paste | me |
| Josh Moore | University of Dundee | notjustmoore | joshmoore |
| Eric Perlman | The Jackson Laboratory | perlman |perlman |
| Constantin Pape | EMBL Heidelberg | cppape | constantinpape |
| Martin Schorb | EMBL Heidelberg | - | martinschorb |
| Kyle Harrington | Max Delbrueck Center | kisharrington | kephale |
| Jean-Karim Heriche | EMBL | | jkh1 |
| Guillaume Gay | Aix-Marseille University | morpholg | glyg |
| Paolo Grisoli | KeenEye Technologies | - | kind84 |
| Nicolas Vinuesa | KeenEye Technologies | - | underscorenico |
| Jon Fuller | Basel | - | jfkotw |

## Session 1 Live Notes

* JKH: would be good to have a process to involve vendor.
- JKH: Have some kind of RFC for the format to help convince vendors
- J.F.: discuss with vendors when puchasing
* [labels], [polygons], [meshes] (10:32)
- CP: still not sure how happy they are with one big container. want to ship EM volume only when updated. Josh: [remote]
- CP: [openorganelle](https://openorganelle.janelia.org/), [neubias tutorial](https://github.com/tischi/i2k-2020-s3-zarr-workshop) Other specifications?
- JF: Discussion with Pete Bankhead. Transformations need supporting in depth. Namespace.
- DS: vector-based labelling? pixel-based could be sparse or densed.
- MS: cryo: position, orientation, some metadata. Even vector specification of each atom.
- KH: point-cloud? In some sense, yes.
- GG: looked into polygons & meshes. Difficult to have both in the same definition. In [meshes] people are only expecting to see vertices and edges. But point with URL to PDB entry would work in [polygons]. For meshes: [PLY](https://en.wikipedia.org/wiki/PLY_(file_format)) could cover most cases (esp. since it's extensible).
- JKH: point as minimal polygon. Martin alluding to having a vector associated with each point.
- https://en.wikipedia.org/wiki/VRML for meshes.
- KH: PLY can handle arbitrary dimensions, don't need edges. Can store chemical, kinetic information.
- GG: but can't store holes in PLY.
- EP: interested in multiscale meshes and fragments. getting to billions of faces. (Neuroglancer has a very custom format)
- CP: saalfeld repositories also have a custom-implementation
- EP: defining the scales, then saying how they are chunked in n-dim space.
- JKH: using linking to other representations, e.g. as table
- GG: meshio author [suggested](https://github.com/nschloe/meshio/issues/957) embedding in zarr doesn't make sense. More precisely, he said we _should not_ create a new specification.
- JF: Pete suggested [GeoJSON](https://fr.wikipedia.org/wiki/GeoJSON). Fairly practical. Several implementations. (INVOLVEMENT/VB)
- EP: would like to see useful tools for extracting byte-values from the zarr arrays. Polygons-to-bitmasks
- GG: (INVOLVEMENT/VB)
- KH: (INVOLVEMENT/VB) State of Java? Josh: ZarrReader. Mobie reads from BDV. Needs refactoring.
- DS: involvement with Scifio. Lots of issues that need fixing.
- CP: (INVOLVEMENT/PB)
- KH/DS: especially transformations. DS: overlapping labels.

* [collections]
- JNI: cf. collections - good things in Draga's format. Good to open individual images with labels, or the whole collection. HCS too heavy weight
- JM: worth getting in touch with Tischi (attending the afternoon question)
- MS: includes labels and meshes? like what CP said, modularized. collection of entities?
- JNI: nice feature of new spec is to open individual things as if they are the whole thing
- CP: link?
- DDP: arose out of a need to open a bag of images for training purposes, but maintain clear correspondence to the original set (for future editing). don't want to just always have a stack. want a hierarchy to open the stack or individual images. should definitely support more than just images. entities, point-layers, etc. all make sense.
- semi-quasi-pseudo spec, implemented as part of a napari reader/writer plugin: https://github.com/DragaDoncila/napari-compressed-labels-io
- WM: essentially what we did with HCS
- JM: balance of how much metadata to group together all images, e.g., to allow quicker clients.
- JKH: design of database and how to relate the various items. relational tables that relate indices. But zarr is needing to deal with that without the database. will need indices.
- KH: trying to understand ...
- JM: two bidirectional levels.
- JKH: like nested collections
- WM: didn't nest arbitrarily since it's hard to give an overview.

* minimal (i.e. non-microsopy)
- Juan: 5D hardcoded requirement? Still plans to remove this requirement? Yes. (Can discuss more if needed)
- Juan: related: can spec be super lightweight bottom-up, then elaborate? ie minimal *required* metadata
- DPP: nice to abstract away a lot of the metadata that's not needed. OME-Zarr is almost perfect for satellite images but just not quite. an intermediate minimal spec is needed.
- JM: tension between zarr and ome-zarr (i.e. don't want HCS)
- JF: e.g. WSI that is 2D you don't want to care about the extra stuff

* post-hoc conversations
- EP: https://forum.image.sc/t/mirroring-idr-zarr-datasets/41136/11 for example of NG metadata
- EP: downsample in xy until isotropic then in all (serial section EM)

## "User registration" Session 2

| Name | Institute | Twitter Handle | GitHub Handle |
|------------ |---------------------- |---------------- |--------------- |
| Copy | and | paste | me |
| Josh Moore | University of Dundee | notjustmoore | joshmoore |
| Davis Bennett | HHMI / Janelia | davisvbennett | d-v-b |
| Melissa Linkert | Glencoe Software | | melissalinkert |
| Volker Hilsenstein | EMBL Heidelberg | | VolkerH |
| Jackson Brown | | jmaxfieldbrown | JacksonMaxfield |
| Damir Sudar | Quantitative Imaging Systems & OHSU | | dsudar |
| Bill Katz | HHMI / Janelia | wtkatz | DocSavage |
| Christian Tischer | EMBL Heidelberg| | tischitischer | tischi |
| John Bogovic | HHMI / Janelia | BogovicJohn | bogovicj |
| Rohola Hosseini | Leiden University | | |
| Robert Haase | TU Dresden | haesleinhuepf | haesleinhuepf |
| Trevor Manz | Harvard Medical School | trevmanz | manzt |
| Talley Lambert | Harvard Medical School | TalleyJLambert | tlambert03 |
| Ken Carlile | HHMI/Janelia | |carlilek |
| Dan Toloudis | Allen Institute for Cell Science | toloudis | toloudis |
| Kevin Kozlowski | Glencoe Software | | kkoz |
| Lee Kamentsky | Kwanghun Chung Lab / MIT | LeeKamentsky | LeeKamentsky |

## Session 2 Live Notes

* HCS
- DB: different from collection? Not really. Once we have the
* Dimensions (DB)
- still hard-coded
- see [openorganelle] for one implementation
- used multiscale representation. transform with each multiscale, list of axes
- CT: each resolution has a transform? Yes. Iteration on
- Principle is: multiresolution is a list of datasets with single-resolution.
- I like that approach (being explicit) because it lets any single “level” of the pyramid be independent and consistent with the others (Bogovic)
- CA: conceptual issue is the more explicit we get the harder for people to implement.
- https://github.com/ome/ngff/issues/28
- LK: bspline warping field?
- useful since so large that you don't want to keep writing them
- transform stack
- JAM: something for later?
- JB: in favor of pushing it a little bit. too dependent on implementations (one person could be storing bspline displacements, etc.) only in the case of displacement fields (which are standard across tools) XxYx2 (2D) with a tag.
- INVOVLED(JB,)
- see "non linear transforms" link below to ngff issue
- DB: xarray support?
- JAM: tried with multiscale, didn't work. So either each scale needs to be a separate xarray or they implement multiscale.
- JB: aicsimageio 4.1 is backed by xarray, but doesn't handle multiscale. each image is a separate xarray object.
- https://github.com/pydata/xarray/issues/4118
* Samples per Pixel (JS)
- conflate samples/channels or new samples dimension. Also mosaic
- JAM: used "A" for sample since "S" was for scene.
- JS: how to make clear to someone what they're seeing. input/output.
- DT: you do want the last channel to be 3 for RGB.
- DS: is RGB/RGBA the only example of where this "hack" would be necessary?
- DB: RGBA is display, but microscope could be higher.
- CT: just treat it as 3 channels?
- DB: access pattern is important, e.g. for GPU.
- BK: kind of like column vs row store, depends on what the use case is (computation vs display)
- JS: missing indices.
* Display settings specification? (CT)
- currently under "OMERO"
- needs specification
* Label mask downsampling methods (center pixel, mode) (CT)
- was writing something for the Java implementation but opened pandora's box
- Janelia: storing frequencies at each location
- DB: mean and mode in a bare bones implementation (only for visualization)
- for *lossless* you need annotations.
* Java writer
- when available?
- using N5 library or something else?
- bioformats2raw writes ome-zarr. Also a branch with ZarrReader and ZarrWriter implementations.
* Geometry (who added? napari issue?)
- meshes / pts / etc.
- DB: there will eventually be data that won't be zarr-able.
- http://www.researchobject.org/ro-crate/
- TL: truth of napari is that we're leaning heavily on these conversations. (There's not yet a model) Community building consensus.
- VH: I have a need for polygons on 2D data (as in link polygons), not sure where meshes came from but can see this making sense for volumetric data.
- JM: need to bridge the multi-file world
- Metadata
- How/where to store image acquisition and other metadata a la OME-XML.
- DS: BINA working with a number of groups to build a set of recommendations for what should be in the metadata. Keep running in the question of what should be in the zarr-world and what should be in the pure metadata side.
- Also: you don't want metadata to be scattered acrossed a chunked filesystem. You want to be able to interpret it all. Metadata stored with a tile.
- JS: wrestled some with that with CZI. cf. mass spec field. great to have a list of things, but if it's not tightly defined it will be a nightmare to implement. (incl. catastrophic effects of resolution units)
- RH: how is HCS handling it now?
- KC: would suggest put all the metadata into one thing.
- BK: also an issue with mutability. caching problem. experimented with **log structure** for the metadata. makes sense in some situations. You get a permanent record of all the writes. (Good for provenance as well, though that's not for every use case)
- KC: or use a database?
* "remote links"
- relationships between datasets
* Label features (status?)
- Draga implemented for smallish metadata
- Josh: for large numbers likely need to look at a binary format.
- DB: datatype that you use is important. Trying to figure out when to use datatype that reflects the number of labels (1 vs 2^32)
- Saalfeld proposed for every array that represents label is to use the largest datatype and then provide a metadata mapping. Layer of complexity but preserves the full space.
- BK: experiemented a lot of that for label block annotations. For smaller sizes (8^3 minichunks inside large blocks) then the labels aren't changing very much. Great compression technique. Or is that part of the spec? This happens in neuroglancer label compression and we have enhanced version in DVID label block compression with multiple indirections. Also useful for fast relabeling since you just modify the lookup table and not the underlying data per voxel.
- DS: problem with storing labels as labelled images is that objects can't overlap. contour maps or run-length would allow overlapping. (Also no problem with maximum encoding in an image)
- LK: use a label matrix as a mapping, where they do overlap you have a list of 2 objects.
* Storage from the sysadmin side (KC)
- File metadata performance
- File count/file count per directory issues
- Object vs File - what is expected?
- Deal with Zarr & N5 that have to be managed. What exactly is expected by the formats on the storage side?)
- Filesystem can't handle a large number of files (GPFS/Lustre)
- There should be a lower limit on the size of files. Should there be upper limit on count of files?
- Saalfeld's lab has 845M files in 252 TB.
- Latency hit from that many files.
- BK: nested directories only get you so far. Trade-off between how small the files are... NG move between sharded and unsharded formats. Can potentially then use a single round-trip for several things. i.e. not just for local also blobstores.
- JAM: https://github.com/intake/fsspec-reference-maker
- BK: problems with in-place writes
- immutable is easier
* OME-Zarr as write format (JS)
- read or write format
- thinking of vendors wanting to slam things to disk, cached into local file structure
- in general, does it make sense that this is an end format but you may use an intermediate
- BK: from fly-EM side, think the primary use case for zarr/n5 was for parallel writes. quickly move data into a clustered context. (note: mutations are different from ingestion. could use a sharded system. parallelize along shards) granularity of your locking is sufficiently small.... use databases.
- LK: cameras would never implement OME-Zarr (writing 64^3 chunks)
- KC: unsuited for data dumps off instruments. getting a single stream from an instrument.
- JS: points taken but mass spec vendors didn't make an effort to implement the open standard to write. Maybe the format next.
- DB: microscopists who buy expensive cameras currently exert zero force. on the PIs & customers.
- JAM: we will try to organize a union (QUAREP-LimI is possibly just that? https://quarep.org/)
- BK: key-value storage coming. cut out POSIX. implement on the storage. Samsung has a team that is talking about this constantly. (KC: no one has implemented) https://www.snia.org/keyvalue
(There has been prototype drives but nothing popular: https://www.anandtech.com/show/14839/samsung-announces-standardscompliant-keyvalue-ssd-prototype)
- CT: one stream per camera could still be fast. then would need to rechunking for different access patterns.
- BK: seq. versus random writes.

## Links

Feel free to add links here at the bottom of the document to make referencing things above cleaner. Alphabetical by alias will make it easier to detect conflicts.

[bioformats2raw]: https://github.com/glencoesoftware/bioformats2raw

[collections]: https://forum.image.sc/t/ome-ngff-combine-multiple-images-into-one-dataset/47619/19

[labels]: https://forum.image.sc/t/multi-scale-image-labels-v0-1/43483/11

[meshes]: https://forum.image.sc/t/mesh-data-in-ome-zarr/44653/14

[metadata]: https://forum.image.sc/t/metadata-for-ome-next-generation-file-format-ngff/44373/27

[multiscales]: https://forum.image.sc/t/multiscale-arrays-v0-1/37930

[openorganelle]: https://www.openorganelle.com/

[example multiscale metadata]: https://open.quiltdata.com/b/janelia-cosem/tree/jrc_hela-4/jrc_hela-4.n5/em/fibsem-uint16/attributes.json

[PhysicalSize]: https://forum.image.sc/t/physicalsize-and-3d-ome-zarr-py/47479/5

[polygons]: https://forum.image.sc/t/polygon-and-other-roi-annotations-in-ome-zarr/47990/10

[remote]: https://github.com/ome/ngff/issues/13

[roadmap]: https://github.com/ome/ngff/projects/1

[scales]: https://github.com/ome/ngff/issues/12

[z5]: https://github.com/constantinpape/z5

[zarr-python]: https://github.com/zarr-developers/zarr-python

[xtensor-zarr]: https://github.com/xtensor-stack/xtensor-zarr

[tensorstore]: https://pypi.org/project/tensorstore

[nczarr]: https://www.unidata.ucar.edu/blogs/developer/entry/nczarr-overview

[issue 50]: https://github.com/zarr-developers/zarr-specs/issues/50

[non linear transforms]: https://github.com/ome/ngff/issues/30

[display settings]: https://github.com/ome/ngff/issues/23