The current state of Waku v2 peer discovery

2021-10-15

The question of peer discovery for Waku v2 has been a focus area since the protocol was first conceptualized. Since then several different approaches to discovery had been proposed, investigated, implemented or placed on development roadmaps.

This post aims to:

• give an accurate picture of the current state of research and implementation of peer discovery mechanisms for Waku v2,
• collate the diverse requirements Waku v2 applications may have for peer discovery, and
• revisit the benefits and tradeoffs of different discovery schemes.

It is meant to be an interactive discussion, with feedback from as many Waku v2 stakeholders as possible, to get a clear picture of where we are, what the specific peer discovery requirements are for different use cases, and how to proceed from here.

Implementation status

Background

Peer discovery is the method by which nodes become aware of each other's existence. This definition should be general enough to cover the use case of discovering peers with selected capabilities, although "peer ID discovery" and "capability discovery" could be treated as separate problem domains in implementation.

Waku v2 design goals

Since the problem of discovery is (mostly) orthogonal to the messenger-oriented problems addressed by Waku v2 protocols, Waku v2 can in theory be used in conjunction with any discovery mechanism. We are, however, researching more integrated approaches that are specifically geared towards Waku v2 design goals.

Specifically, we want to maximize the following properties in selecting discovery schemes:

• scalability
• resource-efficiency
• decentralization

In practice every discovery mechanism implies a tradeoff between these three properties.

Capability discovery

Waku v2 nodes may also be interested in peers with specific capabilities, for example:

1. peers within a specific pubsub topic mesh,
2. peers with store capability,
3. store peers with x days of history for a specific content topic, etc.

There may be ways for some of these capabilities to be advertised and discoverable within any of the mechanisms described below, although this could increase the complexity of finding a node with specific capabilities within the larger network (a needle in a haystack).

Peer discovery candidates

1. DNS-based discovery

Definition

Using this method a new node can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. Our implementation is based on EIP-1459.

Benefits

• highly scalable
• resource-efficient
• existing implementations in go, nim and ts

Tradeoffs

• centralized, and therefore vulnerable to censorship/attacks
• requires multi-step, centralized updating to add new nodes to the discoverable list (i.e. retrieving the current tree, updating and signing the tree, and redeploying to a DNS provider)
• although useful for bootstrapping connection to a Waku v2 network, it must be used in conjunction with other ambient peer discovery methods for a peer to become aware of more nodes in the network.

Implementation notes

Based on EIP-1459 across all three Waku v2 implementations. Since EIP-1459 operates on EIP-778 Ethereum Node Records there is an ongoing conversation about how to encode multiaddr information for which there exists no key definition in EIP-778. These include websocket ports, DNS addresses, etc. which are required for running Waku from a browser.

IMPORTANT NOTE: RFC 25/LIBP2P-DNS-DISCOVERY proposes an alternative approach to DNS discovery not based on EIP-778 ENR. This specification is not currently implemented or on a roadmap to be implemented.

go-waku

EIP-1459 implementation done and integrated in go-waku

nim-waku

EIP-1459 implementation done and integrated in nim-waku

js-waku

WIP. Awaiting consensus on multiaddr encoding in EIP-778 ENR.

2. GossipSub peer exchange

Definition

GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism whereby a pruning peer may provide a pruned peer with a set of alternative peers where it can connect to reform its mesh. This is a very suitable mechanism to gradually discover more peers from an initial connection to a small set of bootstrap peers.

Benefits

• scales with the network
• fairly resource efficient, as it piggybacks on existing control signalling
• specified by libp2p and already implemented in many languages

Tradeoffs

• not useful as a general out-of-band discovery mechanism, as it piggybacks on in-band control signalling
• requires trusting the information provided by pruning nodes
• since this is a libp2p mechanism, not Waku v2, it relies on the respective libp2p libraries for an implementation
• requires a bootstrapping mechanism to initiate connection to the network

Implementation notes

go-waku

Implemented in go-libp2p. Enabled and integrated in go-waku.

nim-waku

Not yet implemented in nim-libp2p. Implementation is likely to form part of the medium term nim-libp2p goal of peer and connection management.

A shorter term minimal implementation is possible by adding support in nim-libp2p for:

• signed peer records
• a callback function whenever a PRUNE is received and contains peer exchange information. nim-waku could then implement this callback function, handling and connecting to the discovered peers.

js-waku

???

3. Node Discovery v5

Definition

Node Discovery Protocol v5 is a DHT-based discovery mechanism adapted to store and relay node records.

Benefits

• decentralized
• existing implementations in nim, go, etc.

Tradeoffs

• high resource-usage, specifically i.t.o. CPU and memory usage.
• "needle in a haystack": finding a peer with specific capabilities may not be feasible within the resource-restricted world of mobile clients and mobile data.
• requires a bootstrapping mechanism to initiate connection to the network

Implementation notes

go-waku

Not yet implemented.

nim-waku

POC implementation in progress. The POC proposes that discv5 can be started and stopped via API calls to limit resource usage.

js-waku

Not yet implemented.

4. Rendezvous

Definition

Rendezvous protocol is a discovery method whereby peers register and discover other registered peers at rendezvous points. Status rendezvous introduces some differences with the original specification, including using ENR for peer information.

Benefits

• suitable for resource-restricted devices
• less centralized than DNS discovery

Tradeoffs

• susceptible to adversary attacks
• requires a bootstrapping mechanism to configure/discover rendezvous points
• no existing implementation in nim. Unknown how useful the js implementation will be for js-waku, due to the differences with Status rendezvous.

Implementation notes

go-waku

Implemented and integrated in go-waku.

nim-waku

Not yet implemented.

js-waku

Not yet implemented.