Try   HackMD

Secure Kademlia Library

Summary

DAS Networking in Ethereum requires lots of small blob samples to be placed in a distributed network where arbitrary nodes can query "the network" to make sure these samples are available.

What this "network" actually looks like is still an open question, but there seems to be high level consensus on its process and design:

  1. Initial Dissemination - Builder sends rows and columns to validators via gossipsub-like protocol (unstructured network)
  2. Continued Dissemination - Validators distribute samples to rest of network via Kademlia-style DHT(structured network)
  3. Querying DHT for samples - Sampling nodes ask DHT for individual samples
  4. Reconstruction - If there's a liveness failure within the network and there are enough individual samples held amongst participants, participants send samples to at least one honest validator (or full node?) to reconstruct (and begin redistribution?) of the blob.

Motivation

This structured network, where anyone can participate, has a lot of nice properties for distributing small chunks of data across an ever-changing set of data and peers, but this thing comes with major drawbacks.

Most notably, anyone can spin up a bunch of node ids and have a disproportionately large amount of control over data and data flow within the network. These are sybil attacks.

A secure kademlia overlay can help mitigate adversaries wishing to cause harm and seems to important piece of the puzzle to making DAS Networking happen.

Within this crate, I modify libp2p's rust implementation of the kademlia protocol to create a "hardened version" of our Kademlia DHT.

 
 

High Level Roadmap

1. Fork Libp2p's implementation of a Kademlia library.

Questions:

  • Does it matter that Libp2p's DHT relies on libp2p streams?

Resources:

2. Implement S/Kad Modifications

Inspired by Dankrad's S/Kademlia document:

  • Parallel, disjoint-path key-value lookups -
    Libp2p already implemented this. See here.
    It might be worth implementing the modified version of parallel lookups, where disjoint lookups kick in if the original lookup doesn't work, as seen in Dankrad's optimizations of S/Kademlia.
  • Validator-only peer logic for routing table -
    *Might stick with a proof of work requirement for node id generation in initial creation of the library (if it's easier) to get the interface down.
  • Sibling broadcast list -
    Instead of replicated storage for a value being defined by length of the kbucket variable (really used for redundancy of overlay connectivity), we define the replicated storage of a sample by a seperate variable.

Expose same functionality as Kademlia:

  • Ping
  • Store
  • FindNode
  • FindValue

3. Integrate with discv5 (via overlay protocol)

Our secure routing table needs to follow specialized rules. Where and how should these rules be implemented?

I believe these questions fall under the umbrella of the Portal Network's Overlay Routing Table functionality.

Questions:

  • How can this library plug into overlay protocol routing tables.
  • How do sampling nodes within a validator-only routing table ask for queries?
  • Should I be creating a specialized routing table library?
    Or some sort of logical interface that plugs into arbitrary DHTs?

Resources:

Last changed by 
 
Echo
0
206

Read more

Aggregated DAS Content

This is a living document of my favorite Data Availability Sampling resources. 

Jul 5, 2023
Development Update #10

Before we get to the normal Dev Update... Over the past 4 months I dove deep into Danksharding, Ethereum Networking, and then p2p networking in rust + building out a discv5 overlay simulation. A few days back, I read a few "big-picture" Ethereum developer documents: Network Layer and Data Availability. Somewhere along the journey I lost the thread. I was building this Data Availability Sampling p2p network simulation without understanding the "why" behind it all.  

Apr 7, 2023
EPF Final Update

Tackling the DAS Networking problem requires a p2p network that has low-latency, is privacy preserving and is sybil-resistant. The idea behind Model DAS is to measure benchmarks on DAS querying via Secure Kademlia DHT Discv5 overlay network. This Secure K-DHT is a seperate validator-only overlay network that leverages validators' incentive alignment to create a robust DAS network if the primary network were to come under attack. Status Report The original vision for Model DAS didn't entail creating a new DAS simulation; the idea was just to extend the functionality of another. But seeing that I was new to gerenal networking concepts and the Rust programming language, it quickly became aparent that building a simplified version of DAS Prototype was an escential prerequisite to executing the vision behind Model DAS. Finished

Mar 1, 2023
Summary of DAS Playground (WIP)

Within this document I talk about specific concepts within DAS-Playground.   Motivation Required Reading: What is Danksharding? A TLDR on Required Reading: Ethereum's rollup-centric roadmap requires extra blob data to be placed on chain. The more blob data that can be placed on chain, the more Ethereum can scale. But... Extra data = extra cost to run a node = reduced decentralization =    : (

Feb 16, 2023
Read more from Echo
Published on HackMD