--- tags: zarr, zsc --- # ZSC 2021.08.30 * Topics & Priorities * Funds: options for spending CZI money - Community manager - Ryan: Charles Stern (?) - Genevieve: 1 year dask contract - Outreachy - Caterva (C developer) - Xarray (dim names) - Marketing (website, docs) - Discussion - RA: Plea for fully remote/virtual *with* science - AM: pushing for remote first but will lose - RA: great to have someone in this role, keep things moving (spec) - AM: Unpacking the different roles (v3 as a framework is a big challenge. but important for sustainability) - RA: differentiates from tiledb / HDF5. prioritize that work. * V3 - JM: Some big questions around v3 spec, e.g., is consolidated metadata in scope? Not much more funds available for v3 dev and implementation. - RW: Recent Zarr meeting notes: https://docs.google.com/document/d/1o-xSVe1iAKNIfz2a9U0-90-z96XwwTT9jzp-ZuyOJQc/edit - RA: not enough people engaged in V3. Prioritize integration testing across all the libraries, and then check the v3 implementation to surface issues (also xarray & dask) - AM: publish & announce a version is good, but people want things that aren't in the V3 spec. Hesitant to go big, since without growth on the extensions. - JK: upload a few datasets to push things forward - AM: getting v3 implemented in zarr-python - JM: Greg Lee is doing well. E.g., introducing store base class. Need help on reviewing. netcdf community focusing on v2. How to motivate people to use v3? I spent 8 months standardising directory store versus nested directory store in v2. What would I do for consolidated metadata? Is v3 too much of a jump? - AM: focus on extension implementations to get us to the tipping point (while letting Unidata focus on V2) - JM: In grant said we'd focus on extensions. E.g., consolidated metadata. Datetime. - RA: Named dimensions. Lots of people have implemented in ad hoc ways. Is it user space metadata? Put in spec, optional but easy to include. - JM: Do it in v3 and also in v2 at the same time. We have 12 months until netcdf migrates. - AM: also a focus on defining the process that covers V2 (maintenance+appendages) & V3 (proper extensions) --> blog - AM: lingering question regarding the front-matter (e.g. URLs) for the v3 spec (something like W3C) spec. Too much overhead? How do we make it easier? Make engaging on the spec front more like engaging on the code front? - RA: trying to get OGC to approve Zarr V2. Extremely complicated process. (Press release, comment, another round, private meeting) Nightmare. But does produce yes/no vote in a finite period of time. - RA: see https://github.com/radiantearth/stac-spec/ for an example (also in terms of content --> **catalogs**) * Contractors? - RA: BeOpen, https://github.com/radiantearth/stac-spec/ - JK: Caterva? - JM: agreed, good for contracting - AM: thought blosc2 delivered the sub-chunk access - JK: worried about forking in the community - RA: not too worried about IronArray - **Let's meet.** ("Really cool. We'd love to engage." Positivity.) - AM: even though not too worried about forking. Value we have is community & extensibility. - JK: to have everything become a flavor of zarr. i.e. not producing a library but producing a spec. cf. imaging -- and the diversity. - JM: https://docs.webknossos.org/reference/data_formats * Governance - Capacity & COVID-times: status of the ZSC * RA: hiring more people, could direct people towards zarr * JK: another engineer coming on board, needs to have things explained. * AM: bandwidth for ZSC meetings and some small patches of time - Meetings: Regular get-togethers? Yes - NumFOCUS meeting: who would like to chat with Arliss? JM, JK, RW+ ## Additional paper notes & TODOs - All: ponder on the community manager - JM: write a draft role description for review - JM: write an email to ZSC finding a slot for monthly calls - JM: write to Francesc about a meeting - JM: write to Arliss about a meeting with JM, JK, RW, etc. - RA: send invite re: filecoin - JM: tabled items and points for next time are in https://hackmd.io/zRYgVxgMQZegcEqqNY3tMQ?edit