Zarr

@zarr

Notes related to https://github.com/zarr-developers

Public team

Joined on Jan 26, 2022

  • Check out the website for previous meeting notes and other information: https://zarr.dev/community-calls/ Joining instructions: https://zoom.us/j/300670033 (password: 558943) GitHub repo: https://github.com/zarr-developers/community-calls Previous notes: https://j.mp/zarr-community-1 2025-04-16 Attending: Josh Moore (JMo), Eric Perlman (EP), Justus Magin (JMa), Gábor Kovács (GK) Open agenda (add here 👇🏻):
     Like 4 Bookmark
  • See: https://github.com/zarr-developers/zeps/compare/main...joshmoore:zeps:zep9-extension-naming?expand=1&short_path=1f5f2ed#diff-1f5f2ed7f4283473e6c53d0c1f3c8628be409c4ee2b0cd237679a8d48b4e0ed8 Authors: Norman Rzepka Josh Moore Status: Draft Type: Specification
     Like 1 Bookmark
  • See Second email to ZSC tl;dr -- perhaps versioning the namespace separately from the format will help if we add a few limitations to JSON-LD that it would offer one possible solution (and a well-specified one) for named resolution. I was hoping to come up with some working code that I could share with you, but ran out of time. Since I don't know when I'll be back at work in earnest, a quick email with some examples. First, a quick reminder of how JSON-LD works: Each JSON-LD file can have a "@context" key which defines how name resolution works. With that it's possible to either load a remote context field, or define namespaces and fields inline. All keys within the JSON-LD are then resolved against this context. EVERYTHING is a URI but can alternatively be referred to as "https://example.com/field", "example:field" or for the default namespace "field"
     Like  Bookmark
  • Check out the website: https://zarr.dev/zeps/meetings Joining instructions: https://openmicroscopy-org.zoom.us/j/82447735305?pwd=U3VXTnZBSk84T1BRNjZxaXFnZVQvZz09 Meeting ID: 82447735305 Password: 016623 GitHub repo: https://github.com/zarr-developers/zeps 2024-10-17 Attending: Dennis Heimbigner (DH), Sanket Verma (SV), Michael Sumner (MS), Jeremy Maitin-Shepard (JMS)
     Like 1 Bookmark
  • --> Zoom Meeting AM JK JM RA RW SV
     Like  Bookmark
  • Talk 🎙️ Proposal submitted! Check here: https://cfp.scipy.org/2023/talk/review/JRFX9N987YHCUKJLD8ZESCECQJVPC8FB Title: Preference: Zarr: community specification of large, cloud-optimized, N-dimensional, typed array storage ([name=Sanket]: Looks good!)"tensors" rather than "n-dimensional typed arrays"? Other Possible titles:
     Like 1 Bookmark
  • NumFOCUS May, 2022 2.12.0a1 A new pre-release version of Zarr is available for testing by the community. This version provides the infrastructure for reading and writing the upcoming V3 spec of the Zarr format. [name=jakirkham] Should we explain how users opt-in to using V3? Or link to that? [name=Sanket] The blog post explains this.
     Like  Bookmark
  • Hi everyone, I hope you’re doing well. After working continuously for several months, the ZEP0001 - V3 Specification is ready for review by you, the Zarr implementations! 🎉 The goal of this process is to move towards supporting the V3 specification in as many implementation as possible. There have been significant changes to the V3 since it was proposed in July 2022. We have maintained a GitHub project board to keep track of the progress. We’re pleased to confirm that all crucial issues have been marked completed after holding several ZEP meetings. Let me review how the process will work. We have created this issue to track the approvals from the ZSC, ZIC and the broader Zarr community. Specific technical feedback about the specification should be made via narrowly scoped issues on the Zarr specs repo that link to this issue.
     Like  Bookmark
  • Tasks Zarr V3 test suite using zarritaTCK (technology compatible kit) TensorstorePython wrapper to compare native stores Benchmarking/Profiling Zarr on the cloud - read Integrate ASV Tutorials: Zarr 🤝🏻 Dask/Xarray
     Like  Bookmark
  • Hi Zarr Community! 👋🏻 We're planning to have a page on our website to showcase the groups using Zarr. These can be organisations, companies, research groups and labs, academic institutions and universities etc. We're mainly thinking of adding a logo and a short blurb, but if you'd like to include more, please let us know, and we'll get that done. Please send your logo and a short blurb which we can add to our webpage. You can add the content over here in your comments or send it over to me and CC Zarr Steering Council. I'm mentioning a few adopters and POI that I'm aware of. Feel free to add/remove your names from the list below;
     Like  Bookmark
  • Creating an issue informing everyone who'll be affected Send out an email to the council Have atleast one meeting before change takes place
     Like  Bookmark
  • BY AUTHOR Fix bug that caused double counting of groups in groups() and group_keys() methods with V3 stores. By :user:Ryan Abernathey <rabernat> :issue:1228. .. _release_2.13.2: Maintenance
     Like  Bookmark
  • Time and place 2nd December, 17:00-20:00 UTC Zoom Meeting Join Zoom Meeting https://us06web.zoom.us/j/83833047249?pwd=NkJxdlFBMDZQRHY5VHBDL3RkRG4xUT09 Meeting ID: 838 3304 7249 Passcode: 369963
     Like 2 Bookmark
  • On the topic of ZEPs (and before the community call in a few hours), some news regarding ZEP0001, since there's been some understandable concern about its progress: Due to time constraints, @alimanfoo has stepped down as lead author for ZEP0001. He can say more himself, but I can't imagine it was an easy choice. For my part and probably most Zarristas, huge thanks to Alistair for all he's done to make Zarr what it is! :bouquet: Before stepping down, Alistair drafted a response for the feedback on PR149. The response document can be viewed here. @jstriebel will take over as the principal author of ZEP1. He has agreed to spearhead the ZEP0001 and resolve all pending comments on the PR along with the help of the document mentioned above. He's been tracking work on a GH project board that should be visible within the org. Moving forward (as discussed during the last ZEP meeting and summarized on the PR), we’ve decided to open separate issues for all the unresolved feedback on the ZEP0001 PR. The issues will be closed once there’s a PR submitted against it. Once all of the conversations are resolved on the ZEP0001 PR, we’d merge it, marking the completion of ZEP0001. Happy to take feedback on the plan above, and very much looking forward to everyone's continued participation! :rocket:
     Like  Bookmark
  • vEM sidenote NR: did Mathworks reach out to Unidata? WF: about NaN? NR: reached out and asked about Zarr for Matlab. They wanted a c/c++ library they could build on top of WF: not about that. Ellen Johnson is the main contact. Will add that to the list. :tada: Java and async
     Like  Bookmark
  • ZEP1 PR 🔗 https://github.com/zarr-developers/zarr-specs/pull/149 Attending: Sanket Verma (SV), Josh Moore (JM), Jonathan Striebel (JS), Alistair Miles (AM) Initial remarks: SV: no contention seen during preparation of this agenda. Several proposals and changes to language but no large blockers. AM: see this as an editor meeting. Take each point in hand and decide on resolution amongst ourselves. General feedback and comments 👇🏻
     Like  Bookmark
  •  Like  Bookmark
  • Zoom link: https://www.google.com/url?q=https://openmicroscopy-org.zoom.us/j/85829696160?pwd%3DVXZuVzZjclJxUDRyQTB4bXFVaWdpdz09%26from%3Daddon 2022-08-03 Attending: JM, MA, AB any projects? MA: maybe sparse data. None now but eventually. MA: https://github.com/pydata/sparse/issues/222
     Like  Bookmark
  • Josh editing From https://chanzuckerberg.force.com/ Zarr: a common backbone for the scalable storage of annotated tensor data (EOSS4) Due: 8/1/2022 RR-5765 Zarr: a common backbone for the scalable storage of annotated tensor data (EOSS4) Due: 10/31/2023 RR-5766 EOSS4 Interim Report #1 (RR-5765) I. Grant Overview Grantee Name NumFOCUS Inc.
     Like  Bookmark
  • Original tweet: https://twitter.com/rabernat/status/1380205129509523457 Deliverable 2: _ARRAY_DIMENSIONS convention The Xarray community has defined as specified in http://xarray.pydata.org/en/stable/internals/zarr-encoding-spec.html the Zarr metadata key “_ARRAY_DIMENSIONS” for storing additional information needed by Xarray to open a collection of Zarr arrays. An ongoing discussion with the NetCDF community (https://github.com/zarr-developers/zarr-specs/issues/73) points to the need to unify this convention. This is also critical for supporting the Xarray API from OME-Zarr, for which some steps have already been taken (https://github.com/ome/ngff/issues/48). Tasks: Submit convention to the central registry Update V2 spec as necessary based on public discussions If necessary, add support for NetCDF-specified metadata patterns in Xarray Review behavior in Xarray if the metadata is missing Test the various metadata states in ome-zarr-py as well as aicsimageio
     Like  Bookmark