---
tags: eth2devs
description: Notes from the regular proof of stake [Eth2] implementers call
image: https://benjaminion.xyz/f/favicon-96x96.png
---

# Consensus Implementers’ Call #101 - 2023-01-12
[Quick contemporaneous notes by Ben Edgington; fka "Eth2 Implementers' Call"]

Agenda: https://github.com/ethereum/pm/issues/702
Livestream: https://youtu.be/a-AO35BobNY

## Capella

### pre-interop testnets update

[Pari] Withdrawal devnet 2 launched yesterday. A couple of bad blocks (maybe EthereumJS), but otherwise looking good. Larger, public version coming next week. First withdrawal shadow fork went well earlier today - forked Sepolia - but only with a small subset of clients. Aim to shadow fork mainnet in a couple of weeks.

[Marius] The bad blocks on devnet 2 were due to EthereumJS - (1) warm coinbase and (2) a huge amount of init data (due to specially hand-crafted transactions). The [fork monitor](https://forkmon.withdrawaldevnet2.ethpandaops.io/) shows the bad blocks. There are four bad blocks shown by Besu nodes, but not by any other nodes - should be investigated. They are not part of the canonical chain.

### CL/EL TX and withdrawals commitment to better support light clients

[Ref](https://github.com/ethereum/pm/files/10348028/elroots.pdf) There has been substantial discussion around this on Discord/TG over the last week.

[Terence] Prysm view is that adding changes at this point is undesirable. [Andrew] For Erigon, agree that we should not change anything for this fork, but longer term switching to Gwei and SSZ for withdrawals is the best plan. [Marek] It's late to be changing things - all the testing has been going well. The tech debt we incur by inaction is not worth delaying withdrawals for. [Matt] Besu is weakly in favour of adding SSZ now and reducing tech debt, although it has come up very late.

[Danny] Do we have a full spec for the changes (both Gwei and SSZ)?

[Mikhail] If we switch EL to Gwei later rather than now then it may be more difficult as lots of people might already rely on it. [Potuz] This is not really an implementation issue. There are design decisions that remain to be made, potentially cross-layer. [Andrew] We should at least switch to Gwei now, even if we do not change to SSZ. Switching later is hacky, as per Mikhail's view. It should be very simple to do and to test on both sides.

[Danny] Is endianness an issue? The consensus side needs little-endian. [Etan] The engine API is JSON, so it's not an issue. In the block header it is RLP encoded.

[Jacek] Nimbus has [a branch](https://github.com/status-im/nimbus-eth2/pull/4487) implementing the changes. It's very small.

[Micah] We should be prioritising the long-term health of the ecosystem and clients over the short-term things today.

[Mikhail] We could try it at the interop event and see how things go, making the spec change if it all goes well.

**Conclusion:**
- Alex Stokes will spec the Gwei change by Monday for discussion at ACD next week.
- There does not seem to be sufficient consensus on changing withdrawals commitment to SSZ today, but would be good if someone wants to spec it - I think Alex also volunteered for this :-)

### Fork version gossip boundary issues

[Use Capella fork version for BLSToExecution](https://github.com/ethereum/consensus-specs/pull/3176). UX improvement for stakers.

[Potuz] Prysm will add BLS changes to pool before Capella, but the question is when to broadcast them? At the fork? Before the fork? Don't want peers to penalise us for gossiping them at a wrong time.

[Danny] The PR allows for the broadcast any time before Capella. [Potuz] Users want these changes in their pools before Capella. Easiest to implement is to ignore all gossiped messages before the fork and to broadcast one's own pool at the fork before accepting gossip.

**Action:** Potuz to create PR describing Prysm's approach. A full spec release candidate due mid/late tomorrow.

[Mikhail] Hsiao-Wei suggested always using Genesis fork id for this. [Danny] But that would make key management in the case of forks very difficult (replay protection). Prefer to change only the network spec at this stage rather than the consensus spec. [Hsiao-Wei] Using genesis fork version is better UX. [Potuz] It's a minor change to implement.

**Decision:** Use genesis fork version for signing BLS credential change messages. Hsiao-Wei to open a PR for the change.

[Potuz] We expect to see hacked versions of clients broadcasting BLS change messages before the fork. Can we add to the spec a condition that they are ignored? You will receive them again at some point. [Danny] This proposal does make the playing field more even.

### engine api

#### [Engine API: unify failure mode for mismatched structure versions](https://github.com/ethereum/execution-apis/pull/337)

Informational: this has been merged into the spec.

#### [Engine API: a bunch of cleanups](https://github.com/ethereum/execution-apis/pull/338)

Informational: this has been merged into the spec.

#### [Engine API: define payload bodies requests](https://github.com/ethereum/execution-apis/pull/352)

This is newly proposed for Shanghai. How do EL client teams feel about it?

Nethermind already has this. Also in Geth. Erigon does not have it, but it's easy to do. Ditto EthereumJS. Besu has not implemented this, but it does not look too difficult.

**Decision:** Merge it into Shaghai spec. Will remove it later if any issues occur.

## 4844

### Quick client status updates

With respect to the latest spec tests (there is one test with a bug):

- Lighthouse: passing all but the one test.
- Prysm: similar. Working on end-to-end tests. Using Mofi's blob utility. Ready for interop.
- Lodestar also passing the tests. Ready for Devnet 4 and interop.
- Besu has some pieces implemented, but not a full implementation. Could do with having a document with all the testing resources that are available. **Action:** Proto to make a list.
- Nimbus has tests passing, and other bits and pieces implemented. Quite a bit of UX work to do yet (how long to store blobs, how big they should be, how to prune well, ...).
- Teku: implementing the state logic. Block production flows are in good shape. Passing most if not all the tests.

[Danny] Are these production-grade or PoC grade? Teku: production quality but some work to do. Lighthouse: ditto. Lodestar: sounds like similar.

### d-star name

[Ref](https://github.com/ethereum/pm/issues/702#issuecomment-1371791032). [Hsiao-Wei] Deneb seems popular. Are we good with that?

No objections. **Decision** The next CL fork after Capella will be Deneb.

## Research, spec, etc

[Zsolt] These two topics are extensions of the light client beacon API. It is desirable to be able to implement a light client via the REST APIs. Geth could use these right now for its light client.

Just informational, no decisions required today.

### [Add block/state multiproof request/subscription feature (WIP)](https://github.com/ethereum/beacon-APIs/pull/267)

Need access to Merkle proofs about the consensus layer, at least the execution payload header, but ideally other things too. Lodestar has such an endpoint already, but my be impractical for other clients to implement due to differences in the implementations of the internal beacon state.

[Etan] Note that [this](https://github.com/ethereum/consensus-specs/pull/3151) might help. [Zsolt] A general purpose API would nonetheless be very useful.

### [Add /eth/v0/beacon/light_client/instant_update](https://github.com/ethereum/beacon-APIs/pull/270)

Less complex. Allows faster access to information about the head of the chain by providing light client committee aggregates as early as possible.

## Open Discussion/Closing Remarks

### [Shanghai/Capella Community Call](https://github.com/ethereum/pm/issues/708)

[Tim] Next Friday at 1500 UTC. Q&A about withdrawals etc. Can post questions on the [agenda](https://github.com/ethereum/pm/issues/708) in advance.

* * *

# Chat highlights

From danny to Everyone 02:04 PM
: https://github.com/ethereum/pm/issues/702

From Trent to Everyone 02:05 PM
: woo!

From MariusVanDerWijden to Everyone 02:05 PM
: Great work Pari!

From Gajinder to Everyone 02:05 PM
: 👏

From MariusVanDerWijden to Everyone 02:06 PM
: https://forkmon.withdrawaldevnet2.ethpandaops.io/

From Justin Florentine to Everyone 02:07 PM
: Copy that

From Etan (Nimbus) to Everyone 02:08 PM
: https://github.com/ethereum/pm/files/10348028/elroots.pdf

From Potuz to Everyone 02:10 PM
: the minimal reasonable one would need changes on both layers because of the Gwei/Wei difference.

From Micah Zoltu to Everyone 02:11 PM
: If we change only withdraw_root to SSZ, the CL "block header" and the EL "block header" won't match still right? To get them to match we need to also change transactions/receipt roots as well for them to match?

From Etan (Nimbus) to Everyone 02:12 PM
: txs yes, receipts no

From Micah Zoltu to Everyone 02:12 PM
: 👍

From Etan (Nimbus) to Everyone 02:12 PM
: but we could also just treat txs as an exception and put the hex root into the CL executionpayload

From Micah Zoltu to Everyone 02:12 PM
: CL doesn't compute it, it can just pull it from the EL?

From Etan (Nimbus) to Everyone 02:13 PM
: it is not sent through engine api

From Potuz to Everyone 02:18 PM
: switching to Gwei from our side is indeed trivial. Not sure on the EL

From Mikhail Kalinin to Everyone 02:18 PM
: WEI on CL side doesn’t sound bad, just inconsistent with other amounts, likewise, GWEI on EL side

From Etan (Nimbus) to Everyone 02:18 PM
: endianness is not relevant for engine api

From Potuz to Everyone 02:19 PM
: endianness is not an issue cause the root will mismatch anyway

From lightclient to Everyone 02:19 PM
: yeah

From Mikhail Kalinin to Everyone 02:19 PM
: QUANTITY is encoded big-endian for all fields, amount isn’t special case here

From Potuz to Everyone 02:19 PM
: we can switch to Gwei and keep endianness in a two-liner

From lightclient to Everyone 02:19 PM
: what is the win for changing from wei -> gwei ?

From Tim Beiko to Everyone 02:20 PM
: never having to change from wei -> gwei on-chan

From lightclient to Everyone 02:20 PM
: when would one do that? wouldn’t you verify against the EL block hash?

From Mikhail Kalinin to Everyone 02:20 PM
: @lightclient hash_tree_root(withdrawal) differs depending on wei/gwei usage

From Etan (Nimbus) to Everyone 02:20 PM
: https://github.com/status-im/nimbus-eth2/pull/4487
the branch

From Potuz to Everyone 02:21 PM
: @mkalinin, anyway regardless we will have different roots because of ssz/rlp

From Mikhail Kalinin to Everyone 02:21 PM
: no, roots are the same

From Potuz to Everyone 02:21 PM
: nope, ssz hashes in a different way

From Andrew Ashikhmin to Everyone 02:21 PM
: With Gwei we can later (Cancun?) just replace the algo to SSZ to ensure the same root w/o having to change the units as well.

From arnetheduck to Everyone 02:21 PM
: +1 for Mikhails idea

From Martin Holst Swende to Everyone 02:22 PM
: I don't see how replicating an ssz root on the EL side, to be calculated same as already done on CL side would add two months

From arnetheduck to Everyone 02:22 PM
: +1 Martin too

From Mikhail Kalinin to Everyone 02:22 PM
: @Potuz they will use RLP to encode but SSZ to calculate commitment

From lightclient to Everyone 02:23 PM
: one idea from a while ago was to switch the EL networking stack to SSZ first before changing consensus objects

From protolambda to Everyone 02:23 PM
: +1 on interop best-effort attempt, but cut it if anyone can't handle a late but small change

From lightclient to Everyone 02:23 PM
: to vet out things in a lower risk way

From Micah Zoltu to Everyone 02:23 PM
: I didn't mean to suggest this will take 2 months (I don't think it will), but I was simply saying that we should be willing to delay withdraws by 2 months to reduce long term tech debt.

From Mikhail Kalinin to Everyone 02:23 PM
: +1 to Andrew

From Potuz to Everyone 02:23 PM
: @mkalinin, but on that we do not have consensus, I believe if we *only* change wei/gwei then we'll get consensus today, but I don't believe we will agree on requiring ssz to compute the htr

From Tim Beiko to Everyone 02:23 PM
: Are there any -1 to Andrew from the EL side?

From Ruben to Everyone 02:24 PM
: makes sense to switching to gwei now not to overcomplicate the code later

From Tim Beiko to Everyone 02:24 PM
: Because we can perhaps make those two decisions today: eventually SSZ, and gwei today

From Micah Zoltu to Everyone 02:24 PM
: I think there is notable consensus on gwei (haven't heard anyone disagree at least).

From Matt Nelson to Everyone 02:24 PM
: +0 from Besu , still weakly in favor.
in agreement on reducing tech debt accumulation
@Tim

From arnetheduck to Everyone 02:24 PM
: There's also "withdrawals-only" SSZ because that is the major tech-debt-introducing point

From lightclient to Everyone 02:25 PM
: wdym major tech debt introducing point?

From stokes to Everyone 02:25 PM
: I can make a pass

From MariusVanDerWijden to Everyone 02:25 PM
: i'm also +0.5, tech debt and because it is a very small change, don't suspect it delaying at all

From lightclient to Everyone 02:25 PM
: and so the main reason for wei -> gwei is to avoid changing this later if/when we change to ssz?

From stokes to Everyone 02:25 PM
: yes

From arnetheduck to Everyone 02:25 PM
: > wdym major tech debt introducing point?

with-roots don't exist right now on EL side in this capacity, so it's new stuff

From Potuz to Everyone 02:25 PM
: We need to change the engine API spec as well, I dare to speak for Prysm that we are ok with this change

From Martin Holst Swende to Everyone 02:25 PM
: @Marius +0.5 to which suggestion?

From lightclient to Everyone 02:26 PM
: @arnetheduck true, but the code for MPT is very easily available and tested, so i think the debt is extremely minimal

From stokes to Everyone 02:26 PM
: Just withdrawals!

From arnetheduck to Everyone 02:26 PM
: > so i think the debt is extremely minimal

all clients have to maintain it forever to run the one fork

From lightclient to Everyone 02:27 PM
: not if you don’t sync from genesis
/ validate

From arnetheduck to Everyone 02:27 PM
: well, the values end up in th eEL

From danny to Everyone 02:27 PM
: https://github.com/ethereum/consensus-specs/pull/3176

From arnetheduck to Everyone 02:27 PM
: which means they end up in "random places" all over

From Mario Vega to Everyone 02:27 PM
: I'll work on updating the hive engine api tests once the spec changes are done

From MariusVanDerWijden to Everyone 02:27 PM
: wei-> gwei AND SSZ for withdrawals

From stokes to Everyone 02:28 PM
: I’d personally wait for a bit more confirmation @Mario

From MariusVanDerWijden to Everyone 02:28 PM
: If no ssz for withdrawals, then I'm against wei->gwei

From stokes to Everyone 02:28 PM
: Why marius?
Its basically just doing part of the work now, rather than all later

From MariusVanDerWijden to Everyone 02:28 PM
: No point in it if we have two different roots

From Etan (Nimbus) to Everyone 02:29 PM
: the wei->gwei part is the only portion that affects engine API. SSZ vs RLP doesn't affect engine

From Etan (Nimbus) to Everyone 02:29 PM
: wei -> gwei also reduces complexity by not having to check that is divisible by 10^9 when converting back on reading

From MariusVanDerWijden to Everyone 02:30 PM
: That check is just moved to the el then, no?

From Etan (Nimbus) to Everyone 02:31 PM
: no, if withdrawal is gwei globally, you only need to multiply by 1b when crediting beneficiary. you never need to divide

From Mikhail Kalinin to Everyone 02:34 PM
: So, we do want IGNORE instead of REJECT in the first line of the change that this PR proposes?

From Potuz to Everyone 02:35 PM
: I think that's right Misha

From Mikhail Kalinin to Everyone 02:35 PM
: I am fine with either solution that suites implementers

From arnetheduck to Everyone 02:39 PM
: fine with whatever gets specified :)

From Potuz to Everyone 02:40 PM
: I think it's safe to assume that Capella releases will have Capella fork version already included :)

From seananderson to Everyone 02:43 PM
: Genesis fork version sounds pretty good to me

From terence to Everyone 02:43 PM
: +1

From Potuz to Everyone 02:44 PM
: We do not oppose, it'll change a few unit tests though.
+1

From Mikhail Kalinin to Everyone 02:45 PM
: GENESIS_FORK_VERSIONS also implies the change to the beacon chain spec as the BLSToExecution sig verification part
https://github.com/ethereum/execution-apis/pull/337

From MariusVanDerWijden to Everyone 02:50 PM
: Implemented in geth, ready to ship
*ready to review

From Mario Vega to Everyone 02:51 PM
: I'll work on hive tests updates for all three PRs

From Mikhail Kalinin to Everyone 02:55 PM
: Yeah, my question was about the cache

From Etan (Nimbus) to Everyone 03:08 PM
: If access to ExecutionPayloadHeader already unblocks you, this LC gossip extension may help you: https://github.com/ethereum/consensus-specs/pull/3151 — it's less generic than what you describe though, and not stateful with subscriptions