# Publish-Subscribe # P2P layer # for DAL --- _Some context:_ The Data-availibility Layer we designed during Q1 of 2022 ![DAL design](https://i.imgur.com/787LVVO.jpg "DAL Design" =600x450) --- Actors: - Slot producers - produce all shards for a given slot - Shards attesters - assigned to a set of shards ffor an epoch - Slot collectors - follows some slots (for rollups, indexers, ...) --- ### PubSub Peer-to-peer Publish-Subscribe network: - Peers can subscribe/unsubscribe to topics - Messages are annotated by topic name - Message for a given topic are only sent to subscried peers --- ### Why PubSub - Keep a permission-less network: - Any new node can join any topic - Think of topics as comm channel, p2p subnetwork - Allows to distribute data, not broadcast - Spare nodes bandwidth - Solution retained by other blockchains --- ### :arrow_right: Initial Mission - Tackling technical debt - Prepare P2P lib for PubSub 󠁝 ### :leftwards_arrow_with_hook: Deviated to new a mission Demonstrate ASAP usability of PubSub for DAL --- _People involved:_ Pietro Abate Richard Bonichon Mathias Bourgoin Julien Coolen Ilias Garnier Nohmane Oulali Vivien Pelletier Nicolas Raymond Julien Tesson --- ## The Problem to Solve - Keep backward compatibility in mind - Assess throughput and latency on **realistic** network - Ensure that attestors can be found rapidly at epoch change - Assess and Minimize the risk of topic's subnetwork split --- ### Q4 objective Demonstrate 100 Mb/s of raw data distribution in a fully connected network - 25 slot producers/consumers sends 2048 shards of 8kB every 30 sec - 400 "attesters" assigned to shards with mainnet-like distribution - Analysis of the time before-availability of shards/slots - Attesters assignement changing every 1 epoch = 10 levels. --- ### PubSub - Annotation of messages with a topic - Advertisement of peers' **topics subscription** - Maximisation of the **coverage** of **topics of interest** with relevant peers --- ### DAL Specific Design choices - Topic shape is $shard \times slot \times level$ - Levels to anticipate next peers of interest - But leads to very large topics space - Topics set representation: $level\ interval \rightarrow shard\ set \times slot\ set$ - Less precise than actual topics set, - But more compact and faster to query. --- ### PoC Performance Assessment We need to test **at scale** - Measure propagation time - Measure intra-topic connectivity - With a realistic number of connections --- ### PoC Performance Assessment - Test it **at scale** - P2p-node binary + scenarios - Large scale deployement infra - Measurements of distributed properties - Add metrics to lib_p2p and p2p-node - Insert meaningful logs - Develop tools to analyse logs from multiple nodes --- ### P2P node - A binary that can - Just run the p2p layer - Run dedicated behaviours _act as message forwarder, slot producer, attester, slot consumer, ..._ - Can be used - In large scale XP - In Tezt tests --- ### Large scale experiment infra  --- ### Large scale experiment exploitation  --- ### Large scale experiment exploitation  --- ### Large scale experiment exploitation  --- ### Results Experimental setup: - fully connected network for now - mainnet like assignement of shards to attester - 225 attesters = usual TB committee - shard assignements proportional so stake - rotation of assignements every 10 levels --- ### Results HERE draw a great curve: time to fully attest/ slots for mainnet distrib HERE draw a great curve: time to fully attest 200 slots/ slot size :dollar: --- ### Ongoing backports to octez-node's p2p - P2p metrics - P2p node for tezt tests - Some perf improvements