HackMD
Final Dev Update
Abstract
On my EPF stay, I started work on the Lambda Ethereum Consensus Client. By adding a new consensus client, this project aims to improve the variety of implementations, at multiple levels:
- it brings a new programming language to the table, Elixir
- it brings a new programming paradigm to the game, functional
- it reduces the underrepresentation of Latin America on protocol development: most initial developers are from Argentina, Latin America
Current status
The project's still a work in progress, but we already expected this from the onset, since building a working client is a humongous task. We mostly followed the roadmap, but we failed to fully complete the "deposits" milestone, achieving only BLS signature verification but no deposit info processing and updating. Apart from that, we didn't fully complete prior BeaconAPI and EngineAPI milestone tasks. This was due to a change of plans: BeaconAPI work didn't really contribute to implementation progress, and since we didn't have a syncing node, we left it for later milestones, replacing it with simple networking metrics (with Grafana and Prometheus) for easier debugging; EngineAPI, on the other hand, was deemed unnecessary for initial syncing (see Prysm's optimistic sync for context), and left for future milestones.
Next I summarize current node functionality.
Building blocks
Done*. We are using Rust libraries (interfacing through the Rustler library) for SSZ, Snappy, and BLS. We plan to migrate them to Elixir in the future, and we are already working on migrating the SSZ one due to differences in the Rust and Elixir type systems and difficulties interfacing between them.
Networking
Done. We are able to discover peers, receive and send requests, and join gossipsub topics. Since we were using existing implementations (go-libp2p, and go-ethereum's discovery packages), only integration work was needed. In the future we plan to rewrite the used libraries in Elixir, but for now it's only in need of some fine-tuning.
Fork-choice
Done. We have implemented the fork-choice algorithm according to the spec and are able to choose between different forks.
State transitions
WIP. We are ~80% done with processing functions and should be finished with them soon. We are using the consensus-spec-tests repo to test our implementation and ensure it works in most cases.
Validator
Not started. Since validator functionality was designed to be isolated from consensus client functionality, we decided to leave this for until we have a syncing client.
APIs
Started. We have worked on some endpoints of both Beacon and Engine APIs, but they're far from finished.
Other
We have added logging and metrics for observability, and we already have tools for performance analysis.
Community
We intended to build a community around the node from the start. In accordance to this, we already have more than 100 people online on our telegram!
Next steps
Work on having a syncing node. Once we are finished with block processing (and integrating it) we need to fine-tune block downloading, since it's starting to become a bottleneck. After that, we need to start adding Beacon API endpoints and payload validation via the Engine API.
After stabilizing all of that, we'll start working on the validator functionality. In the far future, we'll work on moving networking functionality to Elixir.
Along the way, we'll also work on moving SSZ and Snappy libraries to Elixir.
Looking back
The Project
The first few weeks I spent on spec reading, and general knowledge gathering. This helped a lot on understanding the functionality we needed to implement on the client. After that, I started alternating between reading the specs and coding prototypes for the networking part. Once the networking part was ready, I started integrating it with node functionality my coworkers had worked on. With that done, I added some observability to check the current state of the networking component, also working on some other functionality on the side.
At first, we used NIFs (Native Implemented Functions) for interfacing with the Libp2p library, but after using it a bit, we found they weren't the right approach, so we rewrote it using Ports. This resulted in a setback, but we learned a lot from it.
The Fellowship
Hearing core developers' opinions was quite insightful, and seeing what the rest of the fellows were working on was both interesting and good motivation to work harder on our project. The only bad things I can think about the fellowship were the technical problems we had with Jitsi, but I understand those were already being worked on.
I'm really thankful for being able to participate on the fellowship!
