# w3s.link

## Architecture

https://www.tldraw.com/f/IP9v8mcRMMmwIyJw5dNCF?d=v953.823.2420.2624.i54l8fYGANZo2MNop1Ij7

## Repositories

- [Reads Pipeline](https://github.com/storacha/reads/) Monorepo
- [Edge Gateway](https://github.com/storacha/reads/tree/main/packages/edge-gateway) - behind `w3s.link` and `nftstorage.link`
- Main Files
- https://github.com/storacha/w3link/blob/main/packages/edge-gateway-link/src/gateway.js
- https://github.com/storacha/reads/blob/main/packages/edge-gateway/wrangler.toml
- [w3link](https://github.com/storacha/w3link) - user facing gateway behind `w3s.link`. Its main purpose is to have an extra zone cache
- Main files
- https://github.com/storacha/w3link/blob/main/packages/edge-gateway-link/src/gateway.js#L45-L53
- https://github.com/storacha/w3link/blob/main/packages/edge-gateway-link/wrangler.toml
- [Freeway](https://github.com/storacha/freeway/tree/v2.18.3) - See version `2.18.3`
- `2.18.3` relies on Index Sources stored in Cloudflare Datacenter.
- Main files
- https://github.com/storacha/freeway/blob/v2.18.3/wrangler.toml
- https://github.com/storacha/freeway/blob/v2.18.3/src/middleware.js#L118
- Relies on 2 Buckets for indexing - DUDEWHERE AND SATNAV
- DUDEWHERE - `${rootCid}/${carCid}`
- SATNAV - `${carCid}/${carCid}.car.idx`
- Relies on 1 Bucket for CARs
- CARPARK - `${carCid}/${carCid}.car`
- From this version on, Freeway relies in other systems ([Content Claims](https://github.com/storacha/content-claims)) to know where the content is stored.
- This is built on DynamoDB, but seeing the index format may be beneficial given it was an iteration on the format above
- For instance, encode in the index the exact location of the CAR, instead of assuming there will only be one bucket

## Cloudflare details

- [Cache Size limit](https://developers.cloudflare.com/workers/platform/limits/#cache-api-limits)
- 512MB - should be possible to increase by request, but needs negotiation and potentially payment?
- [1000 Subrequests maximum by default](https://developers.cloudflare.com/workers/platform/limits/#account-plan-limits)
- it is possible to increase it contacting support, but CF makes it difficult and prefer that one architecture things differently
- D1 vs S3 for indexing
- D1 [Size limit](https://developers.cloudflare.com/d1/platform/limits/) is much bigger compared to last year
- R2 rate limiting
- at the time we implemented the gateway, it was not prefix based as S3 (i.e. `${carCid}/${carCid}.car.idx`)
- we had some Rate limit issues but things improved over time. Worth investigating these and

## Design Suggestions

- Cache Size can be overcome by performing requests with part of the tree at the time while streaming back the bytes and soring each part of the tree in the Cache
- Calculate well number of sub-requests required to read all needed indexes/external
- Consider avoiding relying on single D1/R2, but create several ones sharded to decrease likelyhood of rate limiting
- Consider R2 as Index Storage rather than D1, worth comparing costs, rate limits and general limits
- Consider [Cloudflare for SaaS](https://developers.cloudflare.com/cloudflare-for-platforms/cloudflare-for-saas/) with custom hostnames per SP. This could potentially improve Cache HIT, enable tracking egress costs per SP

## Questions?

- What parts of the Piece tree will be indexed?