# Messaging Protocol Analysis
This document highlights research conducted on multicast networking/message protocols that can be used as a potential solution for the distributed network layer of a Distributed Key Generation protocol.

Common Goals/Issues:
* Message confidentiality, integrity, and authentication
* peer-to-peer discovery and communication
* ability to multicast messages to a specific channel or instance such that only those with permission to read/write can do so

## 1. [0MQ](http://zguide.zeromq.org/)
* [zyre](https://rfc.zeromq.org/spec:36/ZRE/) / [gyre](https://github.com/zeromq/gyre) ZeroMQ Realtime Exchange Protocol - proximity-based peer-to-peer applications
* Group of peers on a network discover each other, organize into groups, and send each other events
* best [download](https://github.com/pebbe/zmq4)

### Basic Interface

ENTER fromnode headers ipaddress
    a new peer has entered the network
EXIT fromnode
    a peer has left the network
JOIN fromnode groupname
    a peer has joined a specific group
LEAVE fromnode groupname
    a peer has left a specific group
WHISPER fromnode message
    a peer has sent this node a message
SHOUT fromnode groupname message
    a peer has sent one of our groups a message

### Problems
* Mostly for local communication
* Everyone must be a publisher and a subscriber
* Channel history must be stored separately

## 2. IPFS
* There is a potential use case involving IPFS for chat message storage. Storing message state here would solve the problem of late joins in a given message channel.
* Uses libp2p is networking layer
* Probably a bit overcomplicated to implement both 0mq and IPFS

## 3. [libpp2p](https://github.com/libp2p/specs)
* p2p networking layer
* fast, robust, powerful
* Multiformats for self-description, avoiding alg lock-in, cryptographic agility
* Authentication, Transports, Stream Multiplexing, Peer Discovery, Peer Routing, Content routing NAT traversal, Relay
* Runs entirely on browser
* Allows for reconnects
* Allows for channel flooding with [Floodsub](https://github.com/libp2p/go-floodsub)

### Basic Implementation

func BasicFloodsub(network){

    // Hosts are first detected on the current network
    // Then, a NewFloodSubscriber object is created for each         host
    hosts := getNetHosts(network)
    psubs := getPubSubs(network, hosts)
    
    // hosts subscribe to channel "ChanA"
    msgs []*Subscription
    for ps := range psubs {
        subch := ps.Subscribe("ChanA")
        msgs = append(msgs, subch)
    }
    
    // double forloop connecting each host with every other host
    ConnectAllHosts()
    
    // All messages send on the network channel will be received by all hosts.
    for i :=0; i < 100; i++ {
        msg := []byte()
        owner := SendingOwnerNumber()
        
        psubs[owner].Publish("ChanA", msg)
        
        for sub := range msgs {
            got := sub.Next(network)
        }
    }
}

### Problems
* All hosts must connect to every other host
* Channel history must be stored separately
* Must self encrypt

## 4. [devp2p](https://github.com/ethereum/wiki/wiki/%C3%90%CE%9EVp2p-Wire-Protocol)
* secure netsuite for Ethereum
* set of networking protocol and peer to peer protocols
* fast/robust
* uses RLPx for Authentication, Stream multiplexing, network forming, proto0col multiplexing

### RLPx overview
* Node Discovery and Network Formation
* Encrypted handshake
* Encrypted transport
* Protocol Mux (framing)
* Flow Control
* Peer Preference Strategies
* Peer Reputation
* Security
* authenticated connectivity (ECDH+ECDHE, AES128)
* authenticated discovery protocol (ECDSA)
* encrypted transport (AES256)
* protocols sharing a connection are provided uniform bandwidth (framing)
* nodes have access to a uniform network topology
* peers can uniformly connect to network
* localised peer reputation model
* Snappy option for compression / decompression

### RLPx Basic Functions

func WriteMsg(msg) {
    // Encode msg
    encoded := rlp.EncodeToBytes(msg.Code)
    
    // Write header
    headerbuffer := writeHeader(encoded)
    checkUint24Overflow(headerbuffer)
    encheaderbuffer := encryptFirstHalf(headerbuffer)
    
    // Write header MAC
    macheaderbuffer := MAC(encheaderbuffer)
    
    // Write encrypted frame, updating the egress MAC hash with
	// the data written to conn.
	tee := cipher.StreamWriter{S: rw.enc, W:io.MultiWriter(rw.conn, rw.egressMAC)}
	if _, err := tee.Write(encoded); err != nil {
		return err
	}
	if _, err := io.Copy(tee, msg.Payload); err != nil {
		return err
	}
	if padding := fsize % 16; padding > 0 {
		if _, err := tee.Write(zero16[:16-padding]); err != nil {
			return err
		}
	}

	// Write frame MAC. egress MAC hash is up to date because
	// Frame content was written to it as well.
	fmacseed := rw.egressMAC.Sum(nil)
	mac := updateMAC(rw.egressMAC, rw.macCipher, fmacseed)
	_, err := rw.conn.Write(mac)

}

func ReadMsg(msg)
    // Read the header
    headerbuffer := readHeader(msg)
    
    // Verify MAC
    verifiedheaderbuffer := verifyMAC(headerbuffer)
    
    // Read frame content 
    var rsize = fsize // frame size rounded up to 16 byte boundary
	if padding := fsize % 16; padding > 0 {
		rsize += 16 - padding
	}
	framebuf := make([]byte, rsize)
	if _, err := io.ReadFull(rw.conn, framebuf); err != nil {
		return msg, err
	}
    
    // read and validate frame MAC. we can re-use headbuf for that.
	rw.ingressMAC.Write(framebuf)
	fmacseed := rw.ingressMAC.Sum(nil)
	if _, err := io.ReadFull(rw.conn, headbuf[:16]); err != nil {
		return msg, err
	}
	shouldMAC = updateMAC(rw.ingressMAC, rw.macCipher, fmacseed)
	if !hmac.Equal(shouldMAC, headbuf[:16]) {
		return msg, errors.New("bad frame MAC")
	}

	// decrypt frame content
	rw.dec.XORKeyStream(framebuf, framebuf)

	// decode message code
	content := bytes.NewReader(framebuf[:fsize])
	if err := rlp.Decode(content, &msg.Code); err != nil {
		return msg, err
	}
	msg.Size = uint32(content.Len())
	msg.Payload = content
    

### Problems
* DEVp2p is a separate wire protocol (we would have to implement their interface)
* When combined with DEV Node Discovery Protocol it is the [Whisper Protocol](https://github.com/ethereum/wiki/wiki/Whisper) which is a current WiP
* DEV POC 2 Wire Protocol: https://github.com/ethereum/wiki/wiki/Whisper-PoC-2-Wire-Protocol
* DEV POC 2 Whisper Protocol: https://github.com/ethereum/wiki/wiki/Whisper-PoC-2-Protocol-Spec

## 5. [Matrix](https://matrix.org)

* [Open specification](https://matrix.org/docs/spec/) exists, though it not yet stable
* [Server Reference implementation](https://github.com/matrix-org/synapse) in Python
* [Golang homeserver](https://github.com/matrix-org/dendrite) implementation in development
* [Golang client](https://github.com/matrix-org/gomatrix/blob/master/room.go) implementation in development
* [Public audit](https://www.nccgroup.trust/us/our-research/matrix-olm-cryptographic-review/) of their E2E encryption stuff
* If E2E works, then this is a messaging platform where brokers (homeservers mostly) hold encrypted data. May be a good compromise between having plaintext message brokers (most message queue systems today) vs dealing with complexities of going brokerless (some configs of ZMQ, libp2p, etc.)
* Dendrite (WIP golang server) optionally uses [Apache Kafka] for internal communications.
* "Matrix can be thought of as an eventually consistent global JSON db with an HTTP API and pubsub semantics"

Does Matrix have a binary message format?

---

## 7-27-18 Requirements:
* Room "chat" (ideally supports binary messages) functionality
* Room history functionality
* (private messages covered by tying public key to identity)
* (broadcast messages can be signed)

[Reliable multicast](https://en.wikipedia.org/wiki/Reliable_multicast)
* [zmq-pgm](http://api.zeromq.org/2-1:zmq-pgm) 0mq reliable multicast transport using PGM

[PGM](https://en.wikipedia.org/wiki/Pragmatic_General_Multicast)
* [OpenPGM](https://code.google.com/archive/p/openpgm/) google code archive
* [libpgm](https://github.com/g2p/libpgm) c library implementation
* [NORM](zeromq.org/topics:norm-protocol-transport) 0mq pgm transport layer.
* uses the concept of negative acknowledgements (NAKs).
* multicast with loss detection/retransmits similar to TCP

## Parity Secret Store Research:
* [Recent Tutorial](https://wiki.parity.io/Secret-Store-Tutorial-overview) example with dkg among SS servers
* [Parity Documentation](https://wiki.parity.io/Secret-Store.html#server-key-generation-session)

## dedis/drand:
* distributed randomness beacon that involves a set of ndes and two phases
* 1. Setup:
* each node generates a long term pub/priv keypair
* group file generate - all participants public keys and metadata (to operate beacon)
* all participants run dkg to generate collective public key + single private key share (per node)

* 2. Generation:
* any node initiates randomness round
* leader broadcasts msg (timestamp)
* msg signed by participants using threshold BLS sig scheme + priv key shares
* once threshold reached, can reconstruct full BLS signature (lagrange interpolation) = collective random value
* random beacon value can be verified by collective public key