There's been a lot of recent debate around the bandwidth increase that comes with scaling Ethereum's blob count. This document aims to outline what I expect in the upcoming DA upgrades. Some concerns I've noticed lately: Will increasing blob count raise hardware and bandwidth requirements? Will home stakers still be able to participate in block production without outsourcing block building to MEV builders? Is maintaining current hardware and bandwidth requirements hurting Ethereum's competitiveness with other L1s like Solana? I believe researchers and client teams have been working hard to design an upgrade path that balances scaling and decentralisation. A lot of decisions and research efforts so far have been focused on keeping staking accessible. From Francesco's post From 4844 to Danksharding: a path to scaling Ethereum DA, you'll see that all the network parameters have been designed to maintain the same conditions as 4844 and what we have today on Mainnet.

A quick experiement to understand the "relative" bandwidth usage of PeerDAS supernodes and full node. To get more accurate numbers to production, we'd need a large enough network (70 - 100 nodes). Setup Local Kurtosis network Total of 10 nodes on a single machine. 1 full node with 1 validator 1 supernode with 1 validator 1 full node with 62 validatorsused to propose blocks

Using Kubernetes from Docker Desktop This guide uses k8s bundled with Docker desktop, but should work with other k8s distribution. Note that the Docker bundled k8s only supports a single worker node. Add K8s cluster to Kurtosis config Enable Kubernetes cluster in Docker Desktop and restart Docker. Open config with command: vi "$(kurtosis config path)" Add the following to the file: kurtosis-clusters: docker-k8s:

Issue: https://github.com/sigp/lighthouse/pull/2832 Finding the proposer index beacon_proposer_cache fallback to snapshot cache if not in snapshot cache, load from DB Why are we getting Cache Miss? UPDATE: @realbigsean found out we're actually accessing the proposer cache incorrectly, fixed here. That explains why we were getting cache miss so much (it only works at epoch boundaries!), the below race condition is still possible but less likely to occur, and would be easily fixed once we have tree-states.

Devnet-8 Links Request some devnet eth from one of the faucets here: https://dencun-devnet-8.ethpandaops.io/ Run a node on Devnet v8 (Lighthouse/Geth) git clone https://github.com/jimmygchen/eip4844-devnet.git cd eip4844-devnet docker-compose up -d --build Sending Blobs Nethermind's SendBlobs

Issue On Prysm nodes, we're seeing some attestations like this from Lighthouse time="2023-08-02 16:25:04" level=debug msg="Gossip message was ignored" agent="Lighthouse/v4.3.0-dfcb336/x86_64-linux" error="epoch 3 root 0x094f99ab02d5b0d71f54fe3a2309d360eeb6ccca96f91d72c7f8b350533d836b: not a checkpoint in forkchoice" gossip score=21.22931150205812 multiaddress="/ip4/144.126.197.69/tcp/59008" peer id=16Uiu2HAm9GwUAyubQR85fa4RDaTmmqp6QKniDXVHoboKrx8dRBMc prefix=sync topic="/eth2/eec15ee2/beacon_attestation_63/ssz_snappy" From @potuz: @sproul could you check if Lighthouse is struggling or in a very long fork? we have a block in 64 which is a direct descendant of 63 and Lighthouse keeps using 63 as a checkpoint but yeah, whatever this is, the processing time of these blocks shows we can't reallistically handle these networks in these computers at least. 63 took like 9 seconds to process It seems like Lighthouse is using an old root for the target checkpoint in attestations.

Builder API is an interface that enables validators to delegate block building duties. The set of endpoints are exposed by an external block builder (e.g. mev-boost), and consumed by CL clients when proposing a block. EIP-4844 introduces additional changes to beacon block body, execution payload and "blob" data, hence changes will be required on both the Beacon API & builder API to support this. Block Proposal - Builder flow End-to-end block proposer flow Beacon API & Builder API changes sequenceDiagram participant validator participant consensus participant mev_boost

Issue: https://github.com/sigp/lighthouse/issues/3212 Pull Request: https://github.com/sigp/lighthouse/pull/3936 Rate-limiting on backfill sync processing has been introduced to solve the CPU consumption issue on lower spec machines. To figure out a good default rate for processing, it is helpful to understand: the processing time for each backfill batch, so we don't schedule the processing on critical time window and impact validator performance, e.g. slot start the cpu usage improvement when rate-limit is applied to backfill processing the impact to overall time to complete backfill sync Things to consider include:

Project Abstract My goal of this project is to contribute to the implementation and testing of EIP-4844, and learn about the end-to-end protocol development process. During the exploration phase, we reached out to the Lighthouse team to discuss about contributing to EIP-4844, and eventually we decided towork on the Builder API updates for EIP-4844. The reason is that it wasn't on the critical path at the time and the spec discussion hadn't been started, so we had an opportunity to learn about protocol feature development and contribute from the very beginning. Originally we thought the scope for Builder API updates was not big enough for the duration of the program (it turned out that we were completely wrong! :sweat_smile:), so we also to incldued a bunch of smaller tasks as part of our project. Details of the project plan and roadmap can be found here: Project: EIP-4844 Consensus Client. I did a short presentation at the end of the EPF, and the slides can be found here. It's a less wordy version of this update if anyone is interested.

Project links Project: EIP-4844 Consensus Client Design: Builder API for Blobs Summary for week 16 (2023/2/13-2023/2/20) Lighthouse backfill sycn rate-limiting (lighthouse#3936): continued to work on review feedback and added some unit tests. Looked into the impact of changing the number of backfill batches processed (per slot) vs the CPU usage. I've published some analysis results here. Big shout out to @divagant-martian for the help reviewing and unselfishly sharing her knowledge and experience - I've learned a lot in just a single PR! :pray: EIP-4844: Free the blobs PR by @arnetheduck: I read through the changes in this PR, as the block / blob decoupling is quite a substantial change and will have an impact on most of my outstanding PRs on beacon-APIs, builder-specs, ethereum-consensus, lighthouse and mev-rs.Required changes in Lighthouse has been listed in this issue created by @realbigsean Worked on the blob signing endpoints and particpated in the spec discssions

Project links Project: EIP-4844 Consensus Client Design: Builder API for Blobs Summary for week 15 (2023/2/6-2023/2/13) Lighthouse backfill sycn rate-limiting (lighthouse#3936): This week I'm focusing on wrapping up the work on this, mostly addressing review comments and testing. Hopefully this change will land soon! This change will reduce the CPU load for users setting up a new Lighthouse node, by throttling the backfill rate and is configurable if user chooses not to throttle. For background on this, please see my previous update. Builder spec update for Capella finally merged in (builder-specs#60)! :ship: EIP-4844 Builder updates: there are a couple of open PRs that I need to get back to due to recent changes in spec. I had a catch up with @realbigsean earlir this week on the changes related to decoupled blobs & blocks, my plan for the next couple of days:Update the diagrams on the builder update design document to reflect the changes Outstanding PRs / work that will be impacted by this change:Add and update types for EIP-4844/Deneb builder-specs#61 Builder updates for Blobs (EIP-4844) lighthouse#3808

A lot of progress has been made in client development and especially EIP-4844 in the past week as client teams had an on-site interop event. Some important updates on EIP-4844 include: Devnet-4 went well and helped found a few bugs in clients Discussions around moving blob/block sync from coupling to decoupling (See proposal by Jacek here) More details on recent EIP-4844 related updates can be found on the last EIP-4844 Implementers' Call Notes. The decision to couple/decouple blob & blocks is likely to impact the builder API changes quite substaintially, and given that there isn't a final decision made yet (waiting until 4844 network simulation tests result is out), I'm shifting my focus temporarily to other non-builder related work. Project links Project: EIP-4844 Consensus Client

This page describes the steps for setting up a Grafana dashboard to monitor system CPU & Memory over time, using data collected from Glances. Set up Glances Grafana Dashboard Install and start InfluxDB InfluxDB is used to store timeseries data exported from Glances. brew install influxdb influxd Go to http://localhost:8086 and do the following via the InfluxDB UI:

Learning Rust has been an interesting journey so far. It's commonly known that it has a steep learning curve, and I have experienced that myself - personally I've found it more challenging than the few other langugages I picked up in the past few years (solidity, JVM & JavaScript languages). However, with the amount of good learning resources out there, I feel like I got past the initial pain period pretty quickly. Everyone approaches learning programming languages differently , some prefer books, and others prefer tutorials, examples or a combination of all. The approach the worked well for me in the past few years is 1/ do a short course, 2/ get into writing code ASAP. With time being a very scarce resource for me right now, I had to find a way that gets me into writing quality code quickly. I tried a few resources but didn't really work for me - they were either too detailed or jumping between concepts too quickly. Here are some of the resources that I personally found really useful. I think they work very well for someone with years of programming experience and limited time. Derek Banas's Rust tutorial, in just 2.5 hours (2:35:10): this is a very familiar voice for me - I'm pretty sure I picked up a lot of my programming knowledge from Derek. He explained most of the important concepts clearly, in an order that is very easy to understand. By the end of the video, I got some practical experience reading / writing Rust code, but also identified some topics to dive deeper into next, e.g. ownership, references & lifetimes. Rust Demystified 🪄 Simplifying The Toughest Parts (14:04): This video clarified most of the above topics that I wasn't familiar with, and got me more comfortable with reading Rust code. Rust concepts I wish I learned earlier: Good summary on Rust concepts.

Github issue: https://github.com/sigp/lighthouse/issues/3212 Diagrams below are based on @paulhauner's explanation in this comment. sequenceDiagram participant event_rx participant BeaconProcessor participant backfill_queue Title: Existing / Default backfill batch processing event_rx->>BeaconProcessor: new backfill batch work

My focus this week has been on Lighthouse interop testing in preparation for EIP-4844 devnet v3, which was launched a couple of days ago with just one client pair (Prysm/Geth), and we're hoping to to add Lighthouse to the mix very soon (more details in below section). We've achieved a great milestone this week to have Lighthouse integrated into the eip4844-interop repo and have passed both the blob transction tests and fee market tests! Big shoutout to @Inphi, @realbigsean and the Lighthouse team :clap: :fireworks: Areas I worked on this week: Debug Lighthouse sync issues Add Lighthouse to eip-4844-interop repo (eip4844-interop#77) Add Beacon API endpoint to download blobs by block id (lighthouse#3830)

Happy New Year everyone! :fireworks: Summary for this week: Add Beacon API endpoint to download blobs by block id. PR merged. Set up a Lighthouse/Geth node on EIP-4844 devnet-3 Builder spec changes for Capella and EIP-4844, PRs under review. Next week:

Just to add a bit of context to my updates, I'll be sticking my project links here in furture updates. Project: EIP-4844 Consensus Client Design: Builder API for Blobs Update for week 11 (2023/1/9 - 2023/1/16) Proposed a new PR to add a getBlobsSideCar to Beacon API spec (ethereum/beacon-APIs#286) Progressed further with the builder updates for EIP-4844 in Lighthouse (lighthouse/3808). More details in below section. There have been active discussions on the oustanding Beacon API & Builder spec update PRs, hopefully these will be finalised soon:Add and update types for Capella (builder-specs#60) Add and update types for EIP-4844 (builder-specs#61)

Project links Project: EIP-4844 Consensus Client Design: Builder API for Blobs Summary for week 12 (2023/1/16 - 2023/1/23) I implemented the builder updates (lighthouse#3808) in Lighthouse last week, and started looking at adding tests this week While working on the tests, I realised the TestingBuilder utility uses types from the ethereum-consensus and mev-rs libraries, and both need to be updated for Capella and Deneb, so I went down that :rabbit: :hole::Add Capella types and presets ethereum-consensus#168 Add Deneb (EIP-4844) types and presets ethereum-consensus#170 Currently working on adding Capella & Deneb types to the mev-rs library

Time really flies when you're enjoying what you do - there's only a few weeks left on the Ethereum Protocol Fellowship program, and it's a good time to reflect for a bit and think about what I'd like to achieve in the next few weeks. Working with Rust has been an awesome experience - I've learnt a lot just from getting my code to compile :sweat_smile:. The error messages and documentation are very well written and super helpful! Thanks to everyone who's helped me :pray: I'll continue to learn and get better with it while I can! :crab: Builder updates for EIP-4844: I've spent time working on both the ethereum specs (beacon-APIs, builder-specs) and the implementation (lighthouse), and I think being able to experience the end-to-end of how a feature gets implemented in Ethereum (i.e. working as a Ethereum core dev) is very rewarding and fun! I also got to experience that getting consensus on spec changes is sometimes time consuming but can be worth it, as I ended up learning a lot, and the collaboration has usually led us to a better place :smiley: In the final weeks, I'd really like to wrap up most of the work that I have started, continue to learn about Ethereum and Rust, and hopefully this would prep me for the next steps after EPF! Project links Project: EIP-4844 Consensus Client Design: Builder API for Blobs