Nimbus status update – February so far

Hello everyone, and welcome to our February status update.

Before we start, we want to apologise that such a long period of time has passed since our last update. Bruno's boots have not been easy to fill.

While you can expect more regular updates from now on, we're still on the lookout for a full-time technical writer (please don't hesitate to get in touch if you're interested in this role).

tldr;

• First mobile Eth2 testnet
• Fully connected mesh using discovery v5
• BLS implementation ready
• Replacement of Nim’s default stack tracing mechanism with a lightweight alternative
• Introduction of monthly sprints

Brussels

We met in Brussels at the beginning of this month to discuss priorities, goals, and plans.

https://twitter.com/m_ratsim/status/1224769743892230151

And, as you may have seen already, we ended the week by playing with the first mobile Eth2 testnet!

https://twitter.com/VitalikButerin/status/1224934068455911426

While a step in the right direction, the testnet did not have a way to discover peers. This means it wasn't a very realistic implementation.

https://twitter.com/m_ratsim/status/1224972130200236032

Integrating a more realistic peer discovery protocol is one of our main focus points going forward.

However, we're happy to announce that in the last two weeks we've made some great progress on the peer discovery side.

Discovery v5

The mechanism by which peers discover each other in ethereum is called discovery (for more on how discovery works, see here). The latest version is v5.

The great news is that, as of last week, our nodes are able to form a fully connected mesh using discovery v5!

Note that, since the networking part of ethereum 2 was not specified until recently, peer discovery was, for a long time, the most important blocker for ethereum 2.

As it stands, eth2 uses a simplified draft protocol for interoperability testing as the mainnet protocol parameters and details are still being worked on – this includes discovery v5 as well as encryption specifics.

Since we've been working in Nim – a new language – and writing libp2p from scratch, we've fallen a bit behind on v5. Which means that, as it stands, our discovery v5 support is not enough to run on our testnet.

Up until now, we've been relying on the go daemon's built-in discovery mechanism. This was less than ideal, as it made it cumbersome to connect to other networks (one needed to specify all peers manually, instead of discovering them via a distributed hash table).

Now that we have discovery v5 support, we can start thinking about connecting to other clients' testnets.

BLS signatures

We've finished implementing BLS – a digital signature scheme that allows anyone to compress a set of n signatures into 1 signature.

BLS is being standardised by the EF, the IETF, Stanford university, and other reputable organisations. We're now onto the 5th draft of the standardisation document, so we can expect it to be pretty stable going forward.

The only thing missing is the release of the next version of the beacon chain spec (v10.2), as the EF test vectors in v10.1 have a bug on aggregate verification – fixed and pending release (see bug examples 1 and 2).

Lightweight stack traces

This month, we've replaced Nim's default stack tracing mechanism with a (fully open-source and reusable) lightweight alternative based on libbacktrace and libunwind.

This means we can now get useful debugging information without giving up on release builds, or the C compiler's tail-call optimization (an important optimization for some parts of our code base).

Additionally, since the default stack tracing mechanism inserts C preprocessor macros that expand to lines of code inside our functions, lightweight stack traces allow us to get rid of this extra code. Combined with the ability to use -d:release instead of -d:debug, this helps speed up compilation time by ≈ 25%.

Note: -d:debug and -d:release are Nim compiler flags. The compiler uses the -d:debug flag to create slow, but debugging-friendly binaries, and -d:release to create optimised binaries. The default stack tracing mechanism only works in debug builds. While the new one works in both.

Finally, the default stack tracing mechanism also has an important runtime overhead which our lightweight alternative removes. The removal of this overhead is particularly relevant for our CI runs: we can now build all the nim-beacon-chain tools and run all the tests in less than half the time.

Monthly sprints

We now have monthly sprints with milestones (decided in Brussels). Hopefully this will lead to more visibility on what we are working on.

Our main focus for the remainder of this month is the multi-client testnet MVP.

Here are our february milestones in readable format:

Networking

• Libp2p:

  • Stability
  • Validation
  • Peer lifetime

• Discovery

  • Discv5
  • DHT

Eth 2

• Naive attestation
• Block sync
• Peer pool
• Eth1 integration
• Fork choice
• v0.10.x / BLS

Testing

• Block, attestation and validator pools
• Finalization tests in CI
• Monitoring
  • Improve the debugging experience to easily pinpoint where issue came from.
• Multinet scripts
• Mainnet

Extras

• Technical debt
• Debugging
• Documentation
• Communication

See here and here for more.

🚀