---
# System prepended metadata

title: '20XX-XX-XX: sparse on disk formats'
tags: [' governance', ' meeting', scverse]

---

---
tags: scverse, governance, meeting
---

# 20XX-XX-XX: sparse on disk formats

*Attendees:*

## Agenda

https://docs.google.com/presentation/d/1wppjz5RE_gITB9FV3uedosOzUhKTEVjPWCNZEL-_Vz0/edit#slide=id.gf0d3c72f01_0_50

* On disk format
* Logical format

## Notes

* On disk format
    * Options discussed: Zarr, HDF5, ASDF
* Requirements:
    * Dimensionality?
        * COO is easy for high dim
        * Compressed are more complicated, probably just csr, csc
    * Required formats
        * Hypersparse (where indptr is 1d sparse)
            * Similarities to CSF
        * Bitmask
            * Actually bytemask
            * Typically small arrays
            * Possibly more useful in computation than storage
            * It's a dense matrix, do we want dense storage?
    * What is the indices dtype
        * Uint64 in the spec
            * This might be bad to work with
        * Broad agreement that multiple dtypes are useful
            * 128 bit probably painful, <=32 bit more useful
    * Value data types
        * For more interesting data types there are storage format considerations
        * "struct" dtypes - in arrow
        * Or variable length dtypes
        * "Iso" value types **important**


## Follow up

* More meetings!
* Drafts
    * Spec under graph blas org
    * Draft implementations
