# Libp2p

## What is libp2p?
Libp2p is a modular system to build peer to peer network applications. It consists of various network stack components which can interperate and selectively choosen as per the requirements.

Libp2p is more general and reliable in way that it is backward compatible to existing technologies and can easily accomodate future protocol updates.

Following are the components of libp2p:

| Component name | Description |
| -------------- | ----------- |
| Transport | It handles message(data and receipt) communication between two peers using protocols like TCP, UDP, QUIC, WebRTC(browser) depending on runtime and networking environment. |
| Identity | It handles identity management for peers using PeerId. PeerId uses public key cryptography thus providing globally unique Id and a mechanism for secure communication.|
| Security | It handle security mechanism that allows up to upgrade the existing transport component to a secured one. Eg. TLS 1.3 or NOISE |
| Peer Routing | It handles peer discovery by enquiring with its immediate connected peers. Peer discovery mechanism is DHT based and uses Kademlia routing algorithm.|
| Content Discovery | Similar to peer discovery but focuses more on content availability whitout caring much about which peer it gets value from. |
| Messaging / PubSub | It handles message communication and delivery among peer. PubSub organises messaging more by introducing topic around which peer can create group for messages of there interest. |

## Messaging / PubSub
Peer to peer systems does a lots of message passing among peers. In a p2p network there could be messages/topic a might be interested, PubSub short for Publish Subscribe mechanism makes this possible where a peer can subscribe and publish to a topic of there interest.

Libp2p has various pubsub as follows:
- floodsub
- Trade bandwidth for latency.
- Basic design, easy to implementation.
- Flooding based routing, send message to all peers.
- randomsub
- No spec, just concept.
- Send message to random set of `D` peers.
- May creates dark spot thus causing some peers to be left out.
- gossipsub
- Current Implemntation. Trade latency to bandwidth.
- Establish connection with few peers and store them in peerStore.
- Degree of peer controlled by a balancing algorithm.
- Two network design, one for messages and another for gossiping.
- episub
- New specification, upgrade over gossipsub, still under work.
- Based on Epidemic Broadcast Trees.
- Membership management using HyParView and proximity aware overlay.

## Floodsub vs Gossipsub
Simulation experiment done using gerbil simulator to demo the communication happening using both floodsub and gossipsub. Code available [here](https://github.com/vyzo/gerbil-simsub/).


The above experiment result shows that for a given topic, floodsub node communicate the message to all the other peer nodes while gossipsub instead of flooding, communicates with a few set of peer thus avoiding a lots of bandwidth consumption.

## Gossipsub Working
Before jumping into Gossipsub, discusing about peer connection and message passing. Lets look at a higher level view of how Gossipsub based network design would look like.

### Two network design
Gossipsub based networks for a given topic looks something as shown below. It has a two network design, one consisting of full-message peers and another for metadata-only peers.


- Darker link shown above are full-message peering.
- Lighter links are metedata-only peering.
- Grey are shows `topic` context.
- Full-message peering transmit full context of message.
- Metadata-only peering transmits metadat about messages.
- Peering between two connected peers can be either a full-message or metadata only.
- Peering type conversion a.k.a grafting and pruning i.e full-message <-> metadata happens based on a balancing algorithms.

### Peer Discovery
When a new peer is spawned, it needs peer to connect to and with topic it want to subscribe to. The peer discovery method is called ambient peer discovery and has following potential methods as listed below

- Distributed hash tables
- Local network broadcasts
- Exchanging peer lists with existing peers
- Centralized trackers or rendezvous points
- Lists of bootstrap peers

PubSub protocol will make sure that the new peer is floodsubor/and gossipsub based and add it the newtork. Discovered peer responds with a hello message along with the list of topic they are subscribed to.

### Subscribing and unsubscribing
Peer discovery is not enough for peer set creation, new peer need to subscribe to the topic of its interest and lets immediate peer know about it by sending a subscribe messages and same goes for unsubscription.

Un/Subscription messages is necessary, if not sent to the peers; they would end up assuming that the peer is a part or not part of topic respectively.

The un/subscribe message looks like this
> I am subscribed to:
> Topic MEV
> Topic LAYER2
> Topic DEFI

> I am unsubscribed from:
> Topic LAYER2

### Message Passing (Full-message peer)
A full-message peer does follwoing operation
- Emit message created by itself to other full-message peers.
- Forward messages received from a peer to other peers.
- Maintains a list of recently seen messages and also do message validation to avoid redundent and/or invalid message passing.

### Message Passing (Metadata-only peer)
Metadata only peer are involved in actual gossiping of the message metadata. A Peer selects 6 random metadata peers every `1sec` (a.k.a heartbeat-interval) and send them the information of the recently seen messages a.k.a `IHAVE` message.

Metadata peer gossips it with other peers. Now if a peer other than the one above notice that the message was missed out, it immediately sets up peering with that full-message peer and request for a `IWANT` message.

### Grafting and Pruning
Each peer is connected to other peers with some degree and has a peering type defined with it. If the degree of peering with full-message peers is too few or too high then it does the peering type change.

- Grafting
If the peer has too low full-message peering then the peer changes(grafts) one of its metadata peering to full-message peering by notifying the respective peer.

- Pruning
If the peer has too high full-message peering then the peer changes(prunes) one of its full-message peering to metadata only by notifying the respective peer.

### GossipSub sample javascript code

Lets look at a simple gossibsub code. The code will create two peers with gossibsub, both subscribed to a topic called `news` and a peer periodically pushing messages `Bird bird ....` to the topic.

Lets strat by importing the following required modules:

const Libp2p = require('libp2p')
const TCP = require('libp2p-tcp')
const Mplex = require('libp2p-mplex')
const { NOISE } = require('@chainsafe/libp2p-noise')
const Gossipsub = require('libp2p-gossipsub')
const { fromString: uint8ArrayFromString } = require('uint8arrays/from-string')
const { toString: uint8ArrayToString } = require('uint8arrays/to-string')

| Module | Desc |
| - | - |
| libp2p-tcp | libp2p TCP transport component |
| libp2p-gossipsub | libp2p2 gossipsub PubSub component |
| libp2p-noise | libp2p security component for secure tcp transport using NOISE protocol |
| libp2p-mplex | libp2p stream mux component |

Now lets define a function `createNode` with some custome modules configuration that will give us a peer on `createNode()` call.

const createNode = async () => {
  const node = await Libp2p.create({
    addresses: {
      listen: ['/ip4/0.0.0.0/tcp/0']
    },
    modules: {
      transport: [TCP],
      streamMuxer: [Mplex],
      connEncryption: [NOISE],
      pubsub: Gossipsub
    }
  })

  await node.start()
  return node
};

- `Libp2p.create(address, modules)`
- It create a new peer and take two arguments, address and modules.
- Address is a `multiaddress` that our peer will be listening at. Here we set it to `/ip4/0.0.0.0/tcp/0` which means use ipv4 localnetwork and tcp with a random port.
- Modules are set of libp2p components that our peer must be configured with.

- `node.start()`
- It take the newly created peer and starts it.

We are done with the code part dealing with the imports and setup. Lets now look into the main code for our example.

(async () => {
  const topic = 'news'

  const [node1, node2] = await Promise.all([
    createNode(),
    createNode()
  ])

  // Add node's 2 data to the PeerStore
  node1.peerStore.addressBook.set(node2.peerId, node2.multiaddrs)
  await node1.dial(node2.peerId)

  node1.pubsub.on(topic, (msg) => {
    console.log(`node1 received: ${uint8ArrayToString(msg.data)}`)
  })
  await node1.pubsub.subscribe(topic)

  // Will not receive own published messages by default
  node2.pubsub.on(topic, (msg) => {
    console.log(`node2 received: ${uint8ArrayToString(msg.data)}`)
  })
  await node2.pubsub.subscribe(topic)

  // node2 publishes "news" every second
  setInterval(() => {
    node2.pubsub.publish(topic, uint8ArrayFromString('Bird bird bird, bird is the word!'))
  }, 1000)
})()

The code above takes the following step:

- Declaring a topic called `news`
- Create two peer nodes, node1 and node2 using `createNode()` call.
- Node1 sets up link with node2.
- Both peers subscribe to the topic `news`
- Both peers define a callback for message received in the topic.
- Set up a code block to periodically push message from node2, to the topic `news`

Some important methods:
- `node.dial(peerID)`
It establish the communication with other peer whos peerId is passed as an argument.
- `<node>.pubsub.subscribe(topicName)`
It subscribes peer to the topicId passed as an argument.
- `<node>.pubsub.on(topicId, callback)`
It defines a callback for messages received from the topic passed in as an argument.

Result:
> node1 received: Bird bird bird, bird is the word!
> node1 received: Bird bird bird, bird is the word!
> node1 received: Bird bird bird, bird is the word!
> ......
> ......


### Peer Discovery mechanism example
The above example doesn't have any peer discovery mechanism, we had just manually create and connected two peer node. Discovery mechanism helps peers dynamically find other peers and estalish a communication with them.

Libp2p has various peer discovery mechanism as follows with there own respective easy to plug components.

- Distributed hash tables (libp2p kademlia)
- Local network broadcasts (libp2p mdns)
- Exchanging peer lists with existing peers
- Lists of bootstrap peers (libp2p bootstrap)
- Centralized trackers or rendezvous points (relay servers)

Now lets look at an example for peer discovery using `mdns` module. `mdns` help a peer discovery other peers in a localnetwork. Our example will spawn itself as a peer, listen for discovered peers and print their `peerId`

The code for each peer is as follows. Run this code using `node <file-name.js>` from multiple tab or even multiple devices on same local network and you will peers discovery themselves.

/* eslint-disable no-console */
'use strict'

const Libp2p = require('../../')
const TCP = require('libp2p-tcp')
const Mplex = require('libp2p-mplex')
const { NOISE } = require('@chainsafe/libp2p-noise')
const MulticastDNS = require('libp2p-mdns')

const createNode = async () => {
  const node = await Libp2p.create({
    addresses: {
      listen: ['/ip4/0.0.0.0/tcp/0']
    },
    modules: {
      transport: [TCP],
      streamMuxer: [Mplex],
      connEncryption: [NOISE],
      peerDiscovery: [MulticastDNS]
    },
    config: {
      peerDiscovery: {
        [MulticastDNS.tag]: {
          interval: 20e3,
          enabled: true
        }
      }
    }
  })

  return node
};

(async () => {
  const [node1] = await Promise.all([
    createNode()
  ])

  console.log("My peerId is " + node1.peerId.toB58String())

  node1.on('peer:discovery', (peerId) => console.log('Discovered:', peerId.toB58String()))

  await Promise.all([
    node1.start()
  ])
})();

This code is almost same as pervious example with few additions as follows:
- Addition of `peerDiscovery` module `MulticastDNS`
- Addition of configuration for `MulticastDNS`
- Addition of event listener for `peer:discovery` on new peer discovery.

Result:
Each peer running the code should give following output pattern:
> My peerId is QmcRydYJdAfDwF69SqKHUR56HiPL9bFHepenMcEarPNwtf
> Discovered: QmRrW2GggVMPA773PX2hENpAzwjnVunnDjUX878PS3uMa7
> ...
> ...

Running the code for 4 peer gives the following result: