Automatic discovery of content routers

(incl. MVP of federated system)

Initial Proposal

Tl;DR: Bootstrap nodes track observed indexers. client side code asks indexers and report reputation for a feedback loop.

Limitations:

• All Indexers have everything

• Is there a way to avoid the reputation loop?
  Could start by round-robining and tracking latency
  Clients are incentivized to update faster than servers

• are there ways to move logic from clients to bootstrappers?

  • filter on bootstrappers

Note: More controlled environments are easier to update.

MVP needs:

• round robin / latency tracking on clients
• protocol between bootstrappers & clients
  • 'give me known content routers'
• bootstrap library for tracking / filtering
  • can get away with a allow/deny list initially
• Announcements from indexers

V1 needs:

• persistence for clients to remember who they've seen

Additional ideas / directions

• have a federated group that can shard between them.

• could indexers advertise themselves in the DHT?

  • instead, could use a pubsub channel to listen
  • for that to be faster, you'd want active queries