## Anton Nashatyrev
### Ethereum Harmony Team
[Googledocs CV link](https://docs.google.com/document/d/1ByraTXV7vhGSac0hGDrzRFuhLBZ--21vyWxpOv0EIM4/edit?usp=sharing)
---
##### My latets project:
# JVM Libp2p
##### (minimal Eth 2.0 subset)
Gihub: https://github.com/libp2p/jvm-libp2p
---
## Short Libp2p overview:
The P2P networking stack library modularized out of the IPFS (https://libp2p.io/)
By Protocol Labs
---
- Central `Connection` class which is
- **Transport agnostic** : TCP, UDP, QUICK, Relayed connection
- **Secure** : Secio, TLS, Noise
- **Multiplexed** : many `Streams` over a single connection
---
- Modular components
- Protocol `multiselect`
e.g. `/multiselect/secio/1.0.0`
- Endpoint `multiaddress`
e.g. `/ip4/1.2.3.4/tcp/1234/p2p/<peerId>`
---
## Minimal scope (Eth 2.0)
- Transport : **TCP**
- Security: **Secio, Noise**
- Multiplexer: **mplex**
- User protocols:
- **Gossipsub**
- **Identify**
- **Ping**
---
The task sounds a bit boring :sleeping:
---
... however there was a place for
- innovations
- discoveries
- huge pain :weary:
---
for the first time
---
Kotlin first impressions:
- Just improved Java
- Easy to switch (just 2 weeks to become 'native' Kotlin coder)
- Code size: `Java` > `Go` > `Kotlin`
- Less stupid errors
- Coroutines :heart:
---
No reasons to use Java when you may use Kotlin :sunglasses:
---
### `Netty`-centric architecture design
Assumtions:
- Java + network == Netty
- Time-tested transports, codecs, `ByteBuf`, threading considerations
---
Exposing Netty interfaces in core Libp2p classes ([src](https://github.com/libp2p/jvm-libp2p/blob/master/src/main/kotlin/io/libp2p/core/Connection.kt)):
```kotlin
import io.netty.channel.Channel
class Connection(val ch: Channel) { ... }
```
or
```kotlin
import io.netty.channel.ChannelHandler
class MyCoolProtocol : ChannelHandler { ... }
```
looks slightly counterintuitive
---
However this approach has its benefits:
- Referring to well known and documented interfaces with clear contracts
- Easy reusing of existing Netty classes
Abstracting over Netty has no much sense
---
### The pain :weary:
---
Ideal Libp2p API sample (like Go libp2p)
```kotlin=
typealias Fut<T> = CompletableFuture<T> // to fit slide
Fut<Connection> conn = transport.dial(addr)
Fut<SecureConnection> secConn = conn
.thenCompose { it.secure(SecurityHandler()) }
Fut<MplexedConnection> mplexConn = secConn
.thenCompose { it.multiplex(MultiplexHandler()) }
Fut<Stream> stream = mplexConn
.thenCompose { it.createStream() }
Fut<MyProtoCtrl> myProto = stream
.thenCompose { it.setHandler(MyProto()) }
```
---
... but Netty has solely **callback** based architecture
and our attempt to deliver **Future** based API fails right on the first line:
```kotlin=
Fut<Connection> conn = transport.dial(addr)
conn.thenCompose { it.secure(SecurityHandler()) }
```
Why?
---
Netty pipeline property:
**dispose** any event that wasn't picked up by any `ChannelHandler`
---
```kotlin=
Fut<Connection> conn = transport.dial(addr)
sleep(1000) // let's have some rest here
conn.thenCompose { it.secure(SecurityHandler()) }
```
After connection established both parties send initial security packets (e.g. `secio`)
While we are relaxing `(2)` before setting up `SecurityHandler` Netty connects, receives intial sec packet, doesn't found any pipeline handlers and just *disposes* it
`(3)` the `Connection` is already completed, but it's too late :disappointed:
---
Thus the pipeline should be intialized with `SecurityHandler` prior or inside `dial()` like this
```kotlin=
val initCallback = { SecurityHandler() }
transport.dial(addr, initCallback)
```
Looks less cool than original code
---
## Another sample:
```kotlin=
Fut<Stream> stream = connection.createStream()
sleep(1000) // let's have some rest here
Fut<MyProtoCtrl> myProto = stream
.thenCompose { it.setHandler(MyProto()) }
```
The similar situation: while sleeping `(3)` a new `Stream` is created and remote user protocol sends some initial packet which is disposed by the Netty pipeline due to absense of the terminal handler `MyProto`
`(5)` it's too late :disappointed:
---
## Callback hell
This way the whole stack of protocols (Netty handlers) should be supplied beforehand either explicitly of via chain of callbacks
---
The pain: **callback** based API can not be converted to **Future** based API in general case
###### * without durty hacks or altering implementation
https://github.com/libp2p/jvm-libp2p/issues/39
---
... similar to like blocking API can't be converter to non-blocking
###### * without durty hacks
---
## Achievement :muscle:
### Netty **sub-channels** implemented
Netty has plain pipeline for every transport connection and has no 'sub-channel' notion. However multiplexed libp2p `Stream`s are in essense `Channel`s themselves but with a parent `Channel`
Github: [io.libp2p.etc.util.netty.mux](https://github.com/libp2p/jvm-libp2p/tree/master/src/main/kotlin/io/libp2p/etc/util/netty/mux)
---
## Advantages
- Client code may work with a `Stream` in the usual Netty way
- Existing codecs/handlers may be used
- The same performant `ByteBuf` scheme
- The same concurrency model
---
#### Another interesting testing feature:
## Deterministic P2P network simulator
Github: [DeterministicFuzz.kt](https://github.com/libp2p/jvm-libp2p/blob/master/src/test/kotlin/io/libp2p/pubsub/DeterministicFuzz.kt)
---
Components:
- `ScheduledExecutor`s factory with a single time controller ([sources](https://github.com/libp2p/jvm-libp2p/tree/master/src/test/java/io/libp2p/tools/schedulers))
(<s>Executors.new*Executor()</s>)
- System clock source (<s>`System.currentTimeMillis()`</s>)
- `Random` with predefined `seed`
- Simulated Netty channels: shortcut native sockets
---
All scheduled tasks from all `Executors` are executed on a single `[main]` thread with deterministic order
Two modes of `Executor.execute()`:
- Immediate
- Queued
---
## Immediate mode
```java
public void execute(Runnable task) {
task.run();
}
```
- Handy for debugging: the stack contains the whole path of a message `Originating Peer -> Intermediate Peer -> Target Peer`
- `StackOverflow` on large networks :disappointed:
---
## Queued mode
```java
public void execute(Runnable task) {
schedule(task, 0, MILLISECONDS);
}
```
- Still deterministic
- No `StackOverflow`
---
Gains:
- Easy protocols testing
- Reliable unit tests
- Deterministic fuzz testing
- Reproducible failings
---
Project was completed, timeline was met (2.5 months) :tada:
Again the formula was proven:
```
Time = T * 2 + 2 weeks
```
`T` - estimated time which you are 100% confident after thorough investigation of the domain area
---
## Results
- Successfull Eth 2.0 interop with other libp2p implementations <!-- .element: class="fragment" data-fragment-index="1" -->
- Artemis library adoption <!-- .element: class="fragment" data-fragment-index="2" -->
- I got this T-short: <!-- .element: class="fragment" data-fragment-index="3" -->  <!-- .element: class="fragment" data-fragment-index="3" -->
{"metaMigratedAt":"2023-06-15T01:31:44.524Z","metaMigratedFrom":"YAML","title":"My project presentation","breaks":true,"slideOptions":"{\"slideNumber\":false,\"theme\":\"solarized\"}","contributors":"[{\"id\":\"811c1f96-8100-487e-8d59-64bf289c4519\",\"add\":8419,\"del\":3381}]"}