Anton Nashatyrev

Ethereum Harmony Team

Googledocs CV link

My latets project:

JVM Libp2p

(minimal Eth 2.0 subset)

Gihub: https://github.com/libp2p/jvm-libp2p

Short Libp2p overview:

The P2P networking stack library modularized out of the IPFS (https://libp2p.io/)

By Protocol Labs

• Central Connection class which is
  • Transport agnostic : TCP, UDP, QUICK, Relayed connection
  • Secure : Secio, TLS, Noise
  • Multiplexed : many Streams over a single connection

• Modular components
• Protocol multiselect
  e.g. /multiselect/secio/1.0.0
• Endpoint multiaddress
  e.g. /ip4/1.2.3.4/tcp/1234/p2p/<peerId>

Minimal scope (Eth 2.0)

• Transport : TCP
• Security: Secio, Noise
• Multiplexer: mplex
• User protocols:
  • Gossipsub
  • Identify
  • Ping

The task sounds a bit boring

… however there was a place for

• innovations
• discoveries
• huge pain
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →

for the first time

Kotlin first impressions:

• Just improved Java
• Easy to switch (just 2 weeks to become 'native' Kotlin coder)
• Code size: Java > Go > Kotlin
• Less stupid errors
• Coroutines
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →

No reasons to use Java when you may use Kotlin

Netty-centric architecture design

Assumtions:

• Java + network == Netty
• Time-tested transports, codecs, ByteBuf, threading considerations

Exposing Netty interfaces in core Libp2p classes (src):

import io.netty.channel.Channel

class Connection(val ch: Channel) { ... }

or

import io.netty.channel.ChannelHandler

class MyCoolProtocol : ChannelHandler { ... }

looks slightly counterintuitive

However this approach has its benefits:

• Referring to well known and documented interfaces with clear contracts
• Easy reusing of existing Netty classes

Abstracting over Netty has no much sense

The pain

Image Not Showing Possible Reasons

• The image file may be corrupted
• The server hosting the image is unavailable
• The image path is incorrect
• The image format is not supported

Learn More →

Ideal Libp2p API sample (like Go libp2p)

typealias Fut<T> = CompletableFuture<T> // to fit slide 

Fut<Connection> conn = transport.dial(addr)
Fut<SecureConnection> secConn = conn
    .thenCompose { it.secure(SecurityHandler()) }
Fut<MplexedConnection> mplexConn = secConn
    .thenCompose { it.multiplex(MultiplexHandler()) }
Fut<Stream> stream = mplexConn
    .thenCompose { it.createStream() }
Fut<MyProtoCtrl> myProto = stream
    .thenCompose { it.setHandler(MyProto()) }

… but Netty has solely callback based architecture
and our attempt to deliver Future based API fails right on the first line:

Fut<Connection> conn = transport.dial(addr)
conn.thenCompose { it.secure(SecurityHandler()) }

Why?

Netty pipeline property:
dispose any event that wasn't picked up by any ChannelHandler

Fut<Connection> conn = transport.dial(addr)
sleep(1000) // let's have some rest here
conn.thenCompose { it.secure(SecurityHandler()) }

After connection established both parties send initial security packets (e.g. secio)

While we are relaxing (2) before setting up SecurityHandler Netty connects, receives intial sec packet, doesn't found any pipeline handlers and just disposes it

(3) the Connection is already completed, but it's too late

Thus the pipeline should be intialized with SecurityHandler prior or inside dial() like this

val initCallback = { SecurityHandler() }
transport.dial(addr, initCallback)

Looks less cool than original code

Another sample:

Fut<Stream> stream = connection.createStream() 
sleep(1000) // let's have some rest here
Fut<MyProtoCtrl> myProto = stream
    .thenCompose { it.setHandler(MyProto()) }

The similar situation: while sleeping (3) a new Stream is created and remote user protocol sends some initial packet which is disposed by the Netty pipeline due to absense of the terminal handler MyProto

(5) it's too late

Callback hell

This way the whole stack of protocols (Netty handlers) should be supplied beforehand either explicitly of via chain of callbacks

The pain: callback based API can not be converted to Future based API in general case

* without durty hacks or altering implementation

https://github.com/libp2p/jvm-libp2p/issues/39

… similar to like blocking API can't be converter to non-blocking

* without durty hacks

Achievement

Image Not Showing Possible Reasons

• The image file may be corrupted
• The server hosting the image is unavailable
• The image path is incorrect
• The image format is not supported

Learn More →

Netty sub-channels implemented

Netty has plain pipeline for every transport connection and has no 'sub-channel' notion. However multiplexed libp2p Streams are in essense Channels themselves but with a parent Channel

Github: io.libp2p.etc.util.netty.mux

Advantages

• Client code may work with a Stream in the usual Netty way
• Existing codecs/handlers may be used
• The same performant ByteBuf scheme
• The same concurrency model

Another interesting testing feature:

Deterministic P2P network simulator

Github: DeterministicFuzz.kt

Components:

• ScheduledExecutors factory with a single time controller (sources)
  (Executors.new*Executor())
• System clock source (System.currentTimeMillis())
• Random with predefined seed
• Simulated Netty channels: shortcut native sockets

All scheduled tasks from all Executors are executed on a single [main] thread with deterministic order

Two modes of Executor.execute():

• Immediate
• Queued

Immediate mode

public void execute(Runnable task) {
    task.run();
}

• Handy for debugging: the stack contains the whole path of a message Originating Peer -> Intermediate Peer -> Target Peer
• StackOverflow on large networks
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →

Queued mode

public void execute(Runnable task) {
    schedule(task, 0, MILLISECONDS);
}

• Still deterministic
• No StackOverflow

Gains:

• Easy protocols testing
• Reliable unit tests
• Deterministic fuzz testing
• Reproducible failings

Project was completed, timeline was met (2.5 months)

Again the formula was proven:

Time = T * 2 + 2 weeks

T - estimated time which you are 100% confident after thorough investigation of the domain area

Results

• Successfull Eth 2.0 interop with other libp2p implementations
• Artemis library adoption
• I got this T-short: