EPF Final Dev Update #16

Project Abstract

My goal of this project is to contribute to the implementation and testing of EIP-4844, and learn about the end-to-end protocol development process.

During the exploration phase, we reached out to the Lighthouse team to discuss about contributing to EIP-4844, and eventually we decided towork on the Builder API updates for EIP-4844. The reason is that it wasn't on the critical path at the time and the spec discussion hadn't been started, so we had an opportunity to learn about protocol feature development and contribute from the very beginning.

Originally we thought the scope for Builder API updates was not big enough for the duration of the program (it turned out that we were completely wrong!

), so we also to incldued a bunch of smaller tasks as part of our project.

Details of the project plan and roadmap can be found here: Project: EIP-4844 Consensus Client.

I did a short presentation at the end of the EPF, and the slides can be found here. It's a less wordy version of this update if anyone is interested.

Project Status

TL;DR: The builder API update task is still a WIP due to ongoing spec discussions over the last few months. However it's progressing well and it seems to be on the right track.

While waiting for feedback on the spec update PRs, I spent some time working on a few other areas, and I’ve grouped them into a few categories below. I didn't end up working on the Blob Explorer as I already had a few things on my plate, but @0xGabi headed towards that direction, and I'm looking forward to see his work!

Builder API Updates for Blobs (and Capella)

Issue: https://github.com/sigp/lighthouse/issues/3689
Design: https://hackmd.io/a8eyJ505Q62gXMbYr0v7_w

As mentioned above, there are still a few outstanding PRs which are listed below. The spec update PRs didn't get much attention during the early stage, as most client teams were focusing on getting the initial 4844 features implemented and getting the clients ready for devnet v3 at the time. Nevertheless, it was good to have the PRs out there early to kick start the discussion. After @0xGabi and I drafted the spec updates, I jumped onto a draft implementation in Lighthouse. At the time of writing this, the consensus spec for Deneb is pretty much finalised and I am in the process of revising the builder-specs update for Deneb, and I will be continuing on these PRs in the coming weeks.

Pull requests

PR Link	Description	PR Merged?
beacon-APIs#271	Add Capella & 4844 types to Beacon API	✅
builder-specs#60	Builder spec update for Capella	✅
builder-specs#61	Builder spec update for Deneb	🚧
ethereum-consensus#168	Rust library update: add Capella types and presets	✅
ethereum-consensus#170	Rust library update: add 4844 types and presets	🚧
mev-rs#77	Rust library update: add 4844 types	🚧
lighthouse#3808	Lighthouse draft implementation	🚧

EIP-4844 Functioanlities & Testing

The below are other 4844 tasks that I've worked on. The beacon-API spec update (beacon-APIs#282) has taken a bit longer, as we are waiting for Capella to be released first before we can merge the Deneb changes in. The blob signing PR is a recent one that I have started, but also depends on a spec change that is currently under discussion. I'll be working on these in the next couple of weeks.

PR Link	Description	PR Merged?
eip4844-interop#77	Added Lighthouse to eip4844-interop, a testing tool created by @Inphi to help with 4844 interop testing and testing some of the key features of EIP-4844.	✅
devnet-v3	docker-compose setup for devnet-v3 (Lighthouse + Geth)	✅
lighthouse#3830	Implement a Beacon API endpoint in Lighthouse to support retrieval of blobs, and endpoint for users and testing	✅
beacon-APIs#282	Add the above download blob endpoint to the standard spec	🚧
lighthouse#3979	Implement Lighthouse Blob signing APIs and VC signing functionalities	🚧

Lighthouse: Rate limiting historical backfill sync

I heard about this backfill sync issue from a meeting with the Lighthouse team and got interested into working on this problem. At the time of writing, the implementation has been completed, and I am doing some testing to finalise the configuration on number of batches to process per slot. The change is likely to be included in the next release after 3.5.0.

Some relevant links to my work:

• Notes from investigating Lighthouse code base: https://hackmd.io/RUCGCQ5HSM66uo6aeYSn8A
• Issue: https://github.com/sigp/lighthouse/issues/3212
• Benchmarking: https://hackmd.io/h9RvBwR-RiuzX1Okt-Jmnw

PR Link	Description	PR Merged?
lighthouse#3936	Rate limiting historic backfill sync	🚧

Future of the project

This project turned out to be a bunch of smaller tasks across many different areas rather than a standalone project, which was also great because I think I got a good feel for what being a core dev is like from this project. My plan is to wrap up the outstanding PRs, and continue to contribute to the protocol and ecosystem as a full time member of the core dev team.

Self evaluation

Although I didn't get to "finish" the project, I believe I've achieved all of my goals that I set out for this project. I've learnt a lot throughout the EPF program, not just the technical knowledge, collaboration approach, but also what I truely enjoy doing and helped me figure out my next steps after EPF - big thanks to everyone who helped me along the way!

Things I've learned:

• Ethereum basic knowledge, especially in EIP-4844, but there's so much more 🤩
• Improved in writing, both technical and non-technical, from writing weekly updates and doing most of the comms asynchornously on GitHub
• Basics of the Rust lanaguage. It's been fun and can't wait to dig deeper
• In terms of working on spec:
  • Spec changes can take a while and go back and forth, but it's valuable to gather inputs from various teams and do some experiments on different approaches. I've also found it valuable to start the discsussion early, either by creating an issue / a draft PR on the spec.
  • Sometimes it's worth implementing a PoC before spec is drafted or finalised. Even though it could result in an early implemention being completely re-worked or even thrown away - which doesn't seem uncommon, but it's almost never a complete waste because of the valuable learnings and potetntial issues we may discover.
• Reach out to R&D channel for help / questions, people are usually super helpful and try to help when they can.

I've really enjoyed the challenges, the learnings and working with other core devs. It's an amazing community - everyone is welcoming, collaborative, and respectful of each other, and it seems that we all share a common goal to make Ethereum better and deeply care about creating a positive impact to not just the Ethereum ecosystem but the wider society.

For me, it's been very satisfying to be able to work on this and learn about core development full time for the last couple of months. As a dev who's been in the industry for quite some time (13+ years) and have a family dependent on my income, it was a big decision for me to choose to participate in EPF full time over getting a well-paid and comfortable job - thanks to my wife for the full support! And I have to say I'm glad that I stepped out of my comfort zone and be the dumbest person in the room again, and most importantly finding something that get me up excited every morning (or at least most

)!

So it's a wrap! It's been a wonderful journey and pleasure to be part of the Ethereum Protocol Fellowship and cohort. I'm excited to continue to build Etherum and can't wait to the start of my next chapter @ Sigma Prime!

Thanks to the Lighthouse team, especially Sean & Diva, for their patience and help during EPF!

Big thanks to Mario, Josh, and EF for making all this happen!

and thank you all for reading! Bye