---
# System prepended metadata

title: Improving CID Advertising and Routing
tags: [hashberg]

---

# Improving CID Advertising and Routing

- you are here: https://ipfs.fyi/hashberg-notes
- schedule: https://ipfs.fyi/hashberg-schedule
- session-specific [TLDRaw board](https://www.tldraw.com/f/J7FXgqo2GW9lJ-BbgiTYC)
- Facilitator(s): @BUMBLEFUDGE, mosh
- Scribe(s):
- In attendance (& quotable):
    - Bumblefudge
    - Will Scott
    - b5

## Context

- 

## Notes (scribed)

[will]:
- all of the hash table describing who has what should fit within a single rack
- plan was to have caches all over the world, this was at the time of
- What does it mean to actually get to federation:
    - baseline, just run another instance, it'll catch up, and then we point to that instance as a
    - in this "v0" iteration the federation is at the governance layer
    - There is a path where we can remove the human trust part
        - we'd have snapshots & a djstra clock of the known publishers
        - this would get us to consistent snapshots across the instances
- [riba] Can you give a common definition of federation?
- [will] Bluesky is a good metaphor: multiple full replicas run by different parties, all with their own independant failure modes
    - multiple operators, each running a full instances, and clients can go to different
    - In this world, the worst that happens here is additional latency. "bad operators" end in "well, I guess that didn't work, I have to go to another one"
- [riba] but bluesky isn't really federated. There are aggregators, and they provide different
    - [stellz] there is a global view, it's the relays
    - [will] right, and there isn't really federation of the relays
    - [stellz] but federating the relays doesn't really make sense
- [cameron] how is that namespace partitioned?
    - WS: not partitioned, permissioned writers and many replicas
- MOSH: what does adequate bus-factor coverage look like?
- b5: will record-publishers be OBLIGATED to push all records? how prevent a partially-withholding node?
    - any node having something, it gossips to others\
    - b5 what if a byzantine node holds 
    - masih: gradual federation? v1 is not byzantine tolerant (trusted parties run core servers); v2 requires quorum to be byzantine resistant
- Mosh: what's cid.contact point to in v1?
    - masih: cid points to any of the 4; v1 design was like a table held up by 4 legs; 
    - ws: v1.0-1.3
    - ws: v1.now - cid.contact proxies to whichever nodes are CAUGHT UP
        - legacy clients still supported on old routes --> v1 
        - kubo could be updated to hardcode a fanout/fallback array of replicas
        - decentralized discovery is one goal; orthogonal goal is making the federation more resilient
- daniel: why/how centralized?
    - ws: no one paid for it or evangelized decentralizing it
    - ws: this infra covers a LOT of different usecases and architectures; having a governance layer would support flexible 
        - ws: there is an open network of DHT users not paying in and not using much; we can still represent and support them without tokenizing/financializing, etc ("de minimis"?)
        - ws: optimizing for tokenland/FIL also bad for other reasons; need governance even there
        - ws: metering/load analysis needed
- cameron: what's the argument for filecoin and ipfs sharing ipni?
    - ws: CID k/v is CID k/v;
    - ws: bouncing around across boundaries kills the 20millisec roundtrip goal...
    - matt: cid.contact is a right not a privilege at present
    - masih: long tail: 90% of traffic is storacha and pinata; why do longtail users need a slow 
    - matt: why do all the freeusers go to IPNI and not DHT? why not have paying and/or power users use the centralized/federated thing and public option can be elsewhere?
- endorphic: why not train an ML on the DAGs and routes so that locality-sensitive hashing can be layered onto bare CIDs (to predict BGP route and multi-armed bandit strategy)?
- ws: paid tier/free tier seems likely given current conversation with FIL ecosys funders; plan A is a roadmap being written soon that has some kind of budgetary/treasury strategy
    - complexity of funding ops team labor
    - b5: what's it cost to run a full IPNI server for a year, ballpark?
        - 10kish hardware upfront at market price; plus price of rack near backbone
    - riba: but it doesn't work today! storacha is backlogged 5ish days, there are SPs falling down all the time
        - ws: that's an engineer-time problem, not an ops-budgeting problem
        - riba: from my perspective, this architecture cannot be optimized enough that a random writer can come in and say it works
        - masih: disagree; topology is fixable, budget has been zero for 2 years; there is a backlog of fixable bugs and upgrades!
        - masih: change in write request policy merged recently because big publishers are bursty! 2 years ago the current architecture worked fine given the write patterns from back then
        - masih: it's not fundamentally broken, it just needs some dev work to buffer and load-balance for new reality of huge-write platforms
    - riba: but we're working from the assumption that IPNI is worth fixing, because GLOBAL SEARCH is not a usecase anyone has! 
- mosh: can i call on matt? what are the consequences for you if IPNI goes down again tomorrow or for good?
    - matt: customers are PISSED and we can't make them whole; we don't run cid.contact and we can run a smaller equivalent for ourself only, and would if it goes down again; our customers think we ARE ipfs...
    - mosh: why can't you fall back to DHT?
        - matt: if we do, we would flood/break the DHT for everyone else
    - mosh: do we need another layer?
        - matt: i'd like one if it's reliable
    - ws: having all the CIDs in a public good isn't an achievable goal! 
    - riba: matt's customers are unhappy because they UPLOAD to pinata and expect to download from ipfs.io (speaking of unsustainable public goods)
    - endomorphosis: reworking architecture is worth doing; better to rework and make it more efficient than just throwing more resources at current system; if traffic blows up, right back here in a year even with 5 more replicas; 
        - ??: throttled freemium tier sounds fine?
- closing words: i am not against the idea of IPNI, i'm mostly thinking a global index of all CIDs is just a weird goal in the first place?
    - ws: readload is the issue still, volume is still entirely manageable (and the real cost center, because cacheing strategies and DDoS protection need to be finetuned over time)
        - ws: it's an economic problem, putting a price on fast, reliable CID lookuips on an SLA is good! i don't think we need to downscope or gate this to one use-case or one protocol or anything else; the more usecases there are, the lower that price; 
- héctor: in a world were this work what role is there for the DHT?
    - ws: high churn/deletes/etc (mempool-type data) better DHT'd
    - héctor: that sounds like a 90 or 180 degree turn for the DHT's purpose?
    - masih: I think the DHT stays as-is! succesful routing has fallbacks. DHT is a purpose-built system for high-volume and low-latency use-cases, period. DHT can be public option.
    - héctor but if this is free to small user... masih: it absolutely should be!
-  