The Engine API is the interface that allows communication between the execution and the consensus layer of an Ethereum node. Since the merge, this API has become a vital piece of Ethereum's architecture. It coordinates important tasks such as block validation and block proposal. Even with its importance, it's hard to find resources beyond the Engine API spec when learning about it. Trying to understand the API just by reading the spec is a challenging task. For that reason, in this article, I'll explain the most important flows of the Engine API. I'll indicate how those flows can fail, where the spec dictates them, and how they fit together. This article is part of my EPF project. Considerations This is based on the Shanghai spec.

As the first steps of my EPF project, I want to understand how the current implementation of PBS works. And see if I can use it to propose blocks from a validator that uses a Portal Client instead of an EL client. Flashbots architecture for MEV is the most widely used PBS implementation at the moment (September 2023). So that's what this article will be about. I made this diagram to visualize the architecture: flashbots-architecture You can find this diagram in higher quality here.

To understand the Engine API flow for starting up a beacon node, I want to learn what happens behind the scenes when you launch Lighthouse with the beacon node command. I'll be focusing on the interactions with the execution layer. Including how is the EL node represented, which Engine API endpoints are called, etc. I will ignore a lot of functionality not related to the EL. You can check Jimmy Chen's article for a slightly different focus. Let's analyze what happens when you run this on your terminal: [1] lighthouse bn \ --network mainnet \

Project abstract Light clients have always been something that the Ethereum community wants. The Portal Network seems to be the most promising attempt to get light access to all data on the Ethereum network. It has been under heavy research and development for the last couple of years, and it seems to be close to delivering its purpose. For this reason, it's a good moment to figure out ways we can use this new piece of infrastructure. This project aims to enable one use-case for the portal network: replacing execution layer clients with portal clients on staking setups. To learn more about the project, you can read my project proposal. Status report I started my research by learning how the portal network and the Flashbots PBS architecture work. This was with the hope of finding a way to use the PBS infrastructure to outsource block building out of the portal clients. I quickly realized that 1) trying to make this isolated from the rest of the validator functionality (mainly, block validation) wouldn't make much sense and 2) this would be fairly simple once that other functionality is sorted out.

A map is a data structure that stores <key, value> pairs and allows two fundamental operations: PUT(key, value) to add or update a value under a specific key and GET(key) to retrieve the value associated with a particular key. This makes maps highly useful for quickly retrieving specific values. A Distributed Hash Table (DHT), on the other hand, is a more sophisticated structure that extends the functionality of maps to a distributed system. In a DHT, data is dispersed across multiple nodes (computers) in a network. To function effectively, a DHT system must meet two critical criteria: Scalability. As more nodes join the network, the DHT should distribute the data evenly across them to maintain balance and efficiency. Fault Tolerance. In case a node leaves the network or fails, the system should ensure continued access to the data that the unavailable node previously held. In DHTs, data assignment to nodes typically works like this:

This update contains my work done on weeks 15 and 16 of the EPF (2023-10-23 to 2023-11-05). tl;dr: Iterated my Engine API article. Started preparing my devconnect talks. Finalized engine API guide. Mikhail Kalinin gave me feedback on my engine API article. I solved the issues he raised, and now the article is ready!

This update contains my work done on weeks 13 and 14 of the EPF (2023-10-09 to 2023-10-22). tl;dr: Published an article explaining the Engine API. Including sequence diagrams for it's most important flows. A visual guide to the Engine API In my Update #9, I talked about how I needed to get a better understanding of the Engine API; and I mentioned that there was interest in getting some sequence diagrams done for that API. I spent my previous two weeks working on those diagrams. I compiled all of them in a single article, and added explanations for them.

This update contains my work done on weeks 11 and 12 of the EPF (2023-09-25 to 2023-10-08). tl;dr: Wrote an article explaining how Lighthouse starts a beacon node. Had a breakthrough on my understanding of how all the layers fit together. I'll be giving a talk on the ethStaker’s Staking Gathering at Devconnect! Lighthouse Beacon Node Startup On my last update, I mentioned that I went through the Lighthouse codebase trying to understand how it starts a beacon node. I learned a lot from that so I decided to write an article explaining what I learned.

I noticed that my weekly updates were getting too short, so I decided to turn them into bi-weekly updates. This update contains my work done on weeks 9 and 10 of the EPF (2023-09-04 to 2023-09-24). tl;dr: Presented my new project proposal. Dived into the Engine API. Published my Kademlia article on the portal network website. New project proposal presentation I presented my new project in week 9. You can see my slides here and the recording here (I presented last, since it was my second time).

Pretty short update this week. tl;dr: Wrote my new project proposal. My new proposal I spent this week researching, writing and discussing my new project. I opened this pull request on the EPF4 repo with my new proposal. Some discussion was carried out there and info was added to clarify some points The proposal is available here.

This research has a lot of ramifications, and I expect to adjust my course as I dive deeper into it. This makes it hard to define a stable roadmap, but I think I'll have to follow these steps: Stage 1: Get BlocksUnderstand how MEV infrastructure and other forms of PBS send blocks to validators. (I'll focus on Flashbots' MEV-Boost.) Research how can a Portal Client receive blocks from that infrastructure, and code a prototype allowing Trin to do so. Stage 2: Engine API Research how to serve the Engine API data using the blocks received in the previous stage. Research what Engine API endpoints are necessary for proposing blocks. Prototype the relevant Engine API endpoints on Trin.

There are two areas I could focus on: Enabling validators to propose blocks. Enabling validators to attest to blocks. Both options provide significant challenges: When enabling validators to propose blocks, we need to get an ExecutionPayload. There are two approaches that can be taken here: a) The portal client builds the block, or b) Someone else builds the block and the portal client validates it and broadcasts it (think about MEV-boost or any other form of PBS). When enabling validators to attest to blocks, things get trickier. To attest to a block, a normal validator would have to execute all transactions in that block... using its EL node. Including some components of reth in Trin, as libraries, could help solve this, but further investigation is needed.

tl;dr: Posted an issue on the Trin repo to discuss the structure of the documentation. Picked a new project and started working on it. Structuring the docs Two weeks ago, I published my design for Ethereum node documentation. I mentioned in my Update #5 that I wasn't sure about what to do with it. I decided to start small and just create an issue on the Trin repo. This is something that interests me, but not as much as my project. So for now, I'll just leave the issue there without doing more about it.

The Portal Network teams suggested that I change the focus of my EPF project to something else. Piper suggested that I should investigate how we could serve Engine API data from the portal clients. I thought that was a very interesting idea, so I'll evaluate if I should use this as my new project. Why do I think this is important? I have an anecdote related to this: Before the EPF, I was contributing to Lighthouse. There was an instance where I wanted to benchmark some changes to the beacon API, and I needed to run Lighthouse locally. This was troubling because I didn't have an SSD big enough for running an execution layer client alongside Lighthouse. I had to rely on an internal Sigma Prime endpoint to get access to an EL node. And that's the problem we're facing: If we want to run a consensus client, we need to also have the resources to run an execution client alongside. I was fortunate to get access to the sigp node, but running something locally would've been better.

tl;dr: Improved the experience of building and installing Trin on Arch Linux. Designed a framework for documenting nodes. Started looking for a new project. Trin and Arch Linux I've had some trouble running Trin on my laptop for the past few weeks. I used a docker container as a quick fix, but this week I decided to fix it properly. I did 3 things to fix this:

This design was originally thought for trin, but I realized that it can extend to Ethereum nodes in general. I'll base my design on docs I've found useful in the past: Reth and Lighthouse. I believe these are especially relevant as they are software similar to the Portal Clients. Additionally, I'll try to connect these designs with a more standardized approach: Diátaxis. Diátaxis Diátaxis Framework This framework is based on the idea that technical documentation can be classified into four types: Tutorials: Tutorials are lessons that guide the reader through a series of steps to complete a project. Tutorials are learning-oriented.

tl;dr: Presented my project proposal. Wrote about Trin's architecture. Project proposal presentation I started this week by finishing up my slides and getting ready to present my project proposal. Then, I presented it during the office hours call. You can see my project proposal here, my slides here, and the recording of my presentation here (I start at minute 2:05).

This document is a WIP. I expect to add more info as I learn more about trin. In this document, I try to better document how trin is built. Focusing on the history network. Let's start by learning what happens at startup. This is an outline of the functions called: We start with the main function. This is called when the program is executed, or at cargo run. main reads the config of the program (the flags included in the launch command) and calls run_trin. run_trin does a lot of things, but we'll focus on how it starts the sub-protocols. Each sub-protocol has an asociated function of the form initialize_history_network. This function starts some services (see the image above) but also returns four values. Those values are used as follows:

tl;dr Went through the suggested reading. Did some initial research on the projects suggested by mentors. Suggested reading I started by reading the suggested material shared in the repository. I was happy to realize that I already knew most of the things they discussed. This gave me the confidence to begin exploring the projects proposed by the mentors. Project ideas I created Notes on EPF project ideas to document my learnings about the various projects.

tl;dr: Wrote about Danksharding, The Portal Network, and Verkle Trees. Decided to work on The Portal Network! Overview Articles As I mentioned in last week's update, I did an overview of the upgrades coming to Ethereum. I took notes on the areas that were most interesting to me: Danksharding Portal Network