--- tags: NGFF --- # IDR-NGFF 2020-W43 ## 2020-10-23 ### Clients - Will: https://github.com/ome/ome-zarr-py/pull/57 -- initial plate viewer - stitched plate on opening based on a resolution that's 3000x3000 (not necessarily the lowest level) Make it configurable? - **Not** returning a pyramid so need to get back to that. - Today trying that plate or idr0033 with a time range, hopefully that will be lazy - Click-handler isn't in the PR. Josh: needs a standalone command. - Will: also going back to vizarr since there's a fix for viv for click-handlers. Tested by Ilan against the big plate. - https://github.com/hms-dbmi/vizarr/pull/43 - J-m: interested to look at the same tactic (lower resolution) for analysis ### Nothing - Datasets - Formats - Infrastructure ## 2020-10-22 ### Clients - Will: - Lazy loading - https://github.com/joshmoore/ome-zarr-py/pull/1 would allow arbitrarily large plates. Could zoom into each well up to full resolution. Could return a labels or polygon layer that outlines everything on the full grid. Segfaults. - Josh to review - got a simple fix back from Ilan & Trevor to load a plate at the lowest resolution. Plugin assumes that "omero" is present. - Josh to get A1 field 0 metadata in place. - need click handler. They are adding it next - can over though - looking into what we can do on click - Simon: - Store provenance of omero-cli-zarr conversion - https://github.com/ome/omero-cli-zarr/pull/33 - version+git commit into zarr metadata - i.e every commit gets its own version ### Nothing new - Datasets - Formats - Infrastructure ## 2020-10-21 ### Clients - Will: https://github.com/hms-dbmi/vizarr/pull/43 - Initially using full sized images - Testing with idr0002 without Time dimension - Having something public asap would help Trevor - JRS: having a non-grid layout would be good - above the plate is "sparse data" - Seb: anything "stitched", i.e. spatially related - Josh: thinking about that for the "plate" object level - Will: figure! - Testing? - npm install - npm run dev ### Datasets - Seb: idr0033 first plate conversion using omero-cli-zarr - using omero-cli-zarr (PRs 28 and 29) from UoD (idr2-slot3) against idr.openmicroscopy.org - w/o multiscale /uod/idr/scratch/idr2-slot3/idr0033-rohban-pathways/41744_illum_corrected/5966_pr28.zarr (~2h40) - w/ multiscale /uod/idr/scratch/idr2-slot3/idr0033-rohban-pathways/41744_illum_corrected/5966_pr29.zarr (~2h40) - happy to push these to S3 - no extra computing related to downsampling ### Formats - Dom/Will: https://github.com/ome/omero-cli-zarr/pull/28 - Will: could not use ome-zarr Scaler as it requires 5D input - Currently loading planes one at a time. Don't want to bind implementation to have 5D object into memory - Open question of whether we need Scaler API or use cv2.resize directly everywhere - Dom: looking into adding more metadata - Josh: https://github.com/ome/ome-zarr-py/pull/56 ### Infrastructure - https://minio-dev.openmicroscopy.org/idr/ is updated to latest Minio, *public* read-only (proxied to minio.openmicroscopy.org) - Seb: could see this useful for e.g. sharing development data Trevor - Simon: apply same principles as demo i.e. not permanent data - Copying the data to S3 - Seb: - https://minio.openmicroscopy.org/ is lifesci only, read-write - https://minio-slow.openmicroscopy.org is *not* updated, keep or delete? - Josh: benchmarking? Simon: doubling up on the beetle work. ## 2020-10-20 ### Clients - Will: looking at tiled images in vizarr. Starting to get something. Using initial output from omero-cli-zarr. ### Datasets - Seb: idr0033 images annotations with outline PNGs on prod90 (--> J-m). Also likely looking at omero-cli-zarr later today. Help to import into one of the Dundee servers. - **Parallelization?** Dom: IO limit? Josh: Eventually definitely. Seb: for IDR we also have multiple RO servers. - Josh: breaking (missed notes but roughly...) - moving to Nested Storage in v0.2? - Simon: part of zarr spec? Only in v3 - Hold off on nested storage? - If required, the underlying library will support it but we will need to figure out how to support v0.1 and v0.2 at the same time. - Simon: to update minio for team usage ### Formats - Dom: https://github.com/ome/omero-cli-zarr/pull/28#pullrequestreview-511941811 - Will: implementing in javascript at the moment - Josh: https://github.com/saalfeldlab/n5-zarr/pull/5 - Could be used for reading in Bio-Formats. Need switch between local and S3 ### Microservices - Simon: no progress on thumbnails service yet ### AOB - targets - I2K (Dec 2020) - demo (omero-cli-zarr against idr0033 uploaded to S3/minio, open in vizarr or napari) - NGFF (Oct 2020) - Seb: omero-cli-zarr against image + labels + upload + napari workflow - Simon: video? Could be posted to image.sc thread as well - minimal HCS workflow - Add labels? No reason it would not work. Will: labels not working in vizarr atm ## 2020-10-19 ### Clients - https://github.com/ome/omero-ms-zarr/issues/74 (Will) - Generating plate view for Zarr - Two options: zoom-in or click-on. Costs unknown - vizarr PR got merged for multi-Z&T - along with open with (client PR plus a plugin repo to migrate) - Allows on IDR "open-with" **if** converted to Zarr. - repo to be migrated. Could be deployed on idr-testing ### Datasets - Seb: will attach them in prod90. Don't intend to convert to ROIs. J-m is regenerating from the pipeline. More viable. Unknown where that stands. (Version of CP looks compatible) Waiting on PR. - Eventually have HCS + thumbnail + segmentations ### Formats - David: focusing on BF 6.6 this week - Dom: started but not much progress - Seb: decision from last Friday was to start on layout option 3 - Simon: use "A" for rows? Yes, semantic name. - Josh: Impact on the openwith-url - Seb: perhaps a catalog for looking up? - Will: any function that can be called. Class, Name & ID - Simon: annotation? or is it Zarr metadata? or both. - Josh: eventually add in metadata to match the layout - Will: question of where to store ID - Josh: move to v0.2 and differentiate between ID and path - e.g. idr/v0.2/plates/$ID.zarr - Simon: what happens when importing into another system? - Seb: assuming that's where we move to UUID - Josh: external info, but spec doesn't make it clear ### Microservices - Simon: microservice working. next step? deploying thumbnail service on IDR system ### AOB - Oct 29 Zoom - Multiple small talks (5min) rather than 1 big talk - How many? 6 at least - Seb: assuming we should have 1 at least - Private image.sc post - Evening timeslot might be tricky for people. Not mandatory. - Frances: where was this announced? First of the monthly calls - https://forum.image.sc/t/upcoming-calls-on-next-gen-bioimaging-data-tools-starting-oct-29/43489 - Nov 04 CSHL - Dec 02 I2K - Goal is to show something for HCS spec - Discussion with C. Tischer + S. Saalfeld for Java side (need a Zarr library) - ome-zarr reference - Trevor need paper. Reference - Zenodo link ## Template ### Clients - ### Datasets - ### Formats - ### Infrastructure -