@web3-storage/access-api data storage

Purpose of this Document

• Explain how @web3-storage/access-api stores data today.
  • https://github.com/web3-storage/w3up/issues/684
• Enable planning of how to scale the service

access-api invocation handlers

not exhaustive. these are the recent ones that interact with data stores.

• provider/add
• access/delegate
  • asserts hasStorageProvider({ consumer: space })
  • then putMany delegations
• access/claim
  • retrieves stored delegations
    • select by delegation audience
    • may need to sort/filter by expiry

access-api operations that interact with data stores

• DelegationsStorage#find

  • DbDelegationsStorageWithR2#find

• DelegationsStorage#putMany

  • DbDelegationsStorageWithR2#putMany

• DelegationsStorage#count

  • DbDelegationsStorageWithR2#count

• ProvisionsStorage#hasStorageProvider(consumer: DID<'key'>)

  • DbProvisions#hasStorageProvider

• ProvisionsStorage#put({ invocation, space, account, provider })

  • DbProvisions#put
    • there is currently not

access-api data stores

Cloudflare D1 (sqlite-ish)

• delegations_v3
• provisions - result of provider/add

Other tables, not used by access/* or provider/*

• accounts - not needed (at all afaict)
• spaces
  • may not be needed - depends on space/info et al expectations
  • spaces registered by provider/add only return space/info { did }
    • This doesn't require spaces table
  • spaces registered by voucher/ would returns essentially this whole row.

Cloudflare R2 (s3-ish)

• DbDelegationsStorageWithR2 stores the bytes of the delegations in r2 and CIDs-only in d1
  • at key /delegations/{ucan.cid.toString(base32)}.car

opportunities to use ucan-stream

• reduce access/delegate invocations into an index that can serve access/claim

  • alternative to access/claim calling DbDelegationsStorageWithR2#find
  • write to
    • if dynamodb, should probably account for expiry sort key at same time (via @mikeal)
    • May not need dynamodb. We could group by audience and write to s3

• reduce provider/add receipts into an index that can answer whether a given space has at least one storage provider of a certain provider did

  • alternative to access/delegate calling DbProvisions#hasStorageProvider