---
# System prepended metadata

title: NGFF-VIV Presentation by Trevor
tags: [NGFF]

---

---
tags: NGFF
---

# NGFF-VIV Presentation by Trevor

Attending: Trevor Manzt, Frances, Josh, Will, Simon, Petr, J-m, David, Chris, Emil, Dom, Seb

- Deck.gl is a WebGL framework from Uber for customizing rendering
- viv broken into:
  - data loader components ("data not in browser")
    - getTile() function (also decodes) not called by user/client.
  - deck.gl layers calling functions to load data
- Easiest to use viv from a react project (good support for deck.gl)
  - deck.gl canvas takes layers
  - a number of declarative (rendering) properties have existing shader code
  - no way to update the properties (handled by applications)
- avivator
  - for anything available via HTTP
  - some advanced rendering
    - first 4 channels by default
    - heuristic for the settings
    - works the same for OME-TIFF and bioformats2raw zarr
  - more feature rich: 
    - linked pan & zoom
      - Will: can the channels be different for each? Don't think so. But building blocks are there in the viv library, but the applications needs to manage the state
    - picture-in-picture
    - local lens transformations (black-to-white color ramp)
  - looking to use it as a way to engage with the community about what they would like to see.
- vizarr
  - guessing is less sophisticated
  - "source" needs to point to a zarr multiscale
  - wanted to integrate viv into a notebook / remote computation location --> ImJoy
    - allows you to use a Python backend
    - ImJoy manages secure data transfer
    - loaded if in an IFRAME
    - biolerplate for setting up an ImJoy client
    - can wrap any zarr-python store for access in viv
    - jupyter kernel is functioning as a proxy to the image data
    - No additive blending between images, but up to 6 layers in one image
- Discussion
  - _looked through examples_
  - for hacking, suggest cloing imjoy_plugin.py to a forked version of the vizarr (localhost:3000)
  - recently added a versioned imjoy api (NPM packages)
  - **Seb**: support for future specs? overlayed masks or multiple images (HCS) **Trevor**: can create separate layers for multiple images, but there are other modalities. Don't yet have anything built into viv for loading masks or other types of layers. But should be a matter of using other deck.gl layers for those modalities. see https://vitessce.io Need to map JSON objects to a deck.gl component.
  - **Will**: avivator vs. vizarr seems like a lot of duplication. Implement every UI feature twice? **Trevor**: vizarr spawned as a way to quickly provide access in a Jupyter environment. Hopefully could bring all of that into the library, but don't want to proscribe the UI that someone is using. In the long-run, likely want to add ImJoy API to Avivator (demonstrates more of the features of Viv) even though it's much more tied to OME-XML (5D). **Josh:** the future OME model should be a superset of OME-XML and that would be a good target for viv and _all_ of the applications.
  - **J-m**: how difficult to deal with a plate (grid)? **Trevor:** similar to picture-in-picture but you want a special layer in the Viv library. A grid layer, an array of different loaders that get drawn. A question of what API for that layer. Several questions: same color transformations to all? Same channels? But there's definitely the posibility of having a composite layer. **Will:** you could also load the plate on the fly (new data source in Python)

- Addendum (Trevor)
  - Something I just realized that I didn't this morning but forgot who asked. The "grid" view for a plate could be achieved with viv currently (if each image is not multiscale). Viv's ImageLayer is a deck.gl layer for where the data must be non-pyramidal (e.g. just a zarr array, not list of zarr arrays), but it allows you to pass scale and translate as props as well.
  - You could create n x m separate ImageLayers and just ensure that each image is translatedon to some arbitrary grid. Deck.gl would take care of rendering
  - Here is an example of a MultiscaleImageLayer(MxIF image) with an ImageLayer(IMS data, non-pyramidal). We translated and scaled the ImageLayer to register with the layer below. http://vitessce.io/?dataset=vanderbilt&theme=dark
  - I opened this issue a while ago, but we haven't run into the use case yet to spend the time to add this feature: https://github.com/hms-dbmi/viv/issues/217