# All Core Devs - Testing (ACDT) #62, December 1, 2025

Barnabas Busa facilitated the call

## Happy 5th Birthday, Beacon Chain!
Barnabas congratulated everyone on the 5th anniv of beacon chain launch. A big milestone for Ethereum’s consensus journey.

## Fusaka

- **msf-1** concluded last week, was a success. No obvious bugs were found.
- **Mainnet fork on Wednesday**: Feeling safe to be moving on the fork.

## Glamsterdam Devnet Updates
### BAL devnet 0 update
Stefan
- Strong progress on consensus and execution testing
- Clients successfully passing execution spec tests
- Devnet launch expected soon
- Kurtosis framework testing in active development

#### Clients update
**Nethermind**(Marc)
- Overall progress going well
- Client is passing key spec tests

**Nimbus** (Dustin)
- Passing 5 or 7 checks consistently

**Besu** (Felipe) shared in chat
- 100% passing tests

**Reth** (Dragan Rakita)
- Review needed on withdrawal flow tests

Spencer
- Running static, niche-case tests may be valuable
- Use Ethereum test suite for edge-case coverage on BALs

### ePBS devnet 0 update
Justin Traglia:
- ePBS progress is steady, and changes are being merged into trunk
- [Breakout call](https://github.com/ethereum/pm/issues/1801) scheduled this Friday
- Justin will review refactoring
- Decision on trustless payments expected this Friday or next All Core Dev meeting

**Lodestar** (in chat)
- Steady progress on state transition
- Now passing v1.6.1 spec

## Gas & Network Capacity
Barnabas
- Ethereum **mainnet reached 60M gas per block last week**; the next milestone target is 80M gas.
- Jochem Brouwer raised a concern related to 80M gas receipts size.
- Also shared the EIP PFId for Glamsterdam addressing receipts size concerns: [EIP‑7975](https://eips.ethereum.org/EIPS/eip-7975#rationale).
- Kamil Chodoła suggested: move first to 75M gas, then ship the EIP implementation to unlock 100M gas later.
- Barnabas asked from the goup on async adoption of the EIP.
- Ben Adams noted that partial adoption of EIP is not a disaster if done safely.
- Łukasz Rozmej shared that increases are possible once client implementations align and are ready.
- Lukasz updated that Carlos from EF is working on state tests.
- Kamil: Some state/receipt tests remain challenging but are expected to be edge cases, not blockers. The plan is to roll out Fusaka first, then continue gas expansion work toward 80M–100M.
- Testing teams expressed support and interest in spinning up a performance test network.

#### Next Steps Agreed
- Barnabas will reach out to client teams to check readiness.
- Performance network (“perfnet”) to test 75M gas and receipts: proposed by Kamil → agreed by Jochem.
- Will be discussed around the end of January.

## Block building
- Marcin Sobczak is building a common interface to trigger Execution Layer client (Nethermind) block creation [here](https://github.com/marcindsobczak/execution-apis/blob/main/src/testing/testing_buildBlockV1.md).
- It would be great for other client teams to review this early version — it still needs small updates, but it will significantly improve the testability of Ethereum block building and enable better test generation.
- Suggesting the Ethereum DevOps Team could review implementation needs.
- EIP‑7951 interactions can be part of broader testing coverage.
- Kamil also suggested to follow up next week so teams can measure progress and align releases.

## ssz vs protobuff vs grpc

Teams want to discuss improving Ethereum’s data format and serialization in future network upgrades, especially for the Glamsterdam upgrade.

- Barnabas: Proposed these API changes after multiple [discussions](https://discord.com/channels/595666850260713488/1443978877440626809/1443980019947929783) and wants feedback from the Execution Layer (EL) teams.
- Alexey: Says this will help the EL move to the more efficient SSZ format faster and that serialization needs to be optimized for performance.
- Lukasz: Notes that changes should be considered during Glamsterdam but warns that in Nethermind, current libraries are missing and it requires heavy effort.
- EL teams generally see SSZ support as an important future goal, but right now, it’s still on the wishlist, not finalized.

#### Next Steps
- The BAL and ePBS breakout calls will transition to ACDT on Jan 5 onward.
- The finalized decisions on API changes will be reviewed again in early 2025.

# All Core Devs - Testing (ACDT) #61 (Nov 10, 2025)

Mario Vega lead the call

#### Quick Summary
- Nov 17 & 24 ACD-T calls canceled
- Next call: Dec 1
- Fusaka looking very healthy
- ePBS devnet-0 by mid-Jan
- Prysm release dropping today (per Kasey)

## Fusaka:
### Hoodi BPO Activation updates
Barnabas
- No issues observed in BPO-1, but discovered a release bug in N/M.
- Mentioned a halting issue with Nimbus on Fusaka-devnet-3 — currently under investigation.
- Nearly all Nimbus nodes appear to be affected; ongoing coordination with the Nimbus team.

Marcin (N/M)
- Implemented several bug fixes and released a new Candidate Release (CR).
- Planning another release to address rare edge-case bugs.
- Snap sync is currently broken, but will be resolved soon.
- Targeting v0.2 as “good enough” for the mainnet release.

Pari
- Reported a release bug in Reth.
- BAL-related changes accidentally impacted mainnet client releases.
- Another minor (non-consensus-critical) issue identified in Lighthouse; fix to follow.
- Shared issue link: [sigp/lighthouse#6837](https://github.com/sigp/lighthouse/issues/6837)

Mario
- Summarized:
“Most things look good. The only pending task is investigating the strange RPC requests.”

Enrico (Besu)
- Nodes are detecting data columns via RPC calls (noticed within the past 10–15 minutes).
- Raised a question about potential buggy client behavior in RPC requests.

#### Devnet-3 status updates
- Several nodes ran out of resources around the 20 blocks per blob configuration.
- Additional debugging needed.

#### Mainnet Client Release Status

Jen (Reth)
- Released Fusaka patch: [reth v1.9.1](https://github.com/paradigmxyz/reth/releases/tag/v1.9.1)

Kasey (Prysm)
- Preston is preparing a new release, expected later today.
- Update: expect the release today.

## 60M gas limit on mainnet updates:
### State test updates
Benchmark Testing

Marcin
- Conducted initial benchmark testing with EIP-7904; results are working as expected.
- Next steps:
- Develop a dashboard to reprice N/M and Geth, then compare results.
- Generate new payloads to extend the benchmark analysis.

Ameziane Hamlat
- Suggested including TPS (transactions per second) as an additional metric, since Mgas/s alone may not fully capture performance.
- Highlighted TPS as a potentially more insightful performance indicator.

### Gas limit testing update
- Completed two rounds of testing.
- No further Gas Repricing Breakout sessions are currently planned.

## Glamsterdam Testing Updates:
### BALer updates - bal-devnet-0
Pari
- Planning to launch a new BAL devnet soon.

Stefan
- Asked whether BAL tests are fixed.
- Inquired if client teams have executed the EVM fuzzing scenarios.

Spencer
- Confirmed that previous releases focused on BAL-specific tests.
- Shared release reference: https://github.com/ethereum/execution-spec-tests/releases/tag/bal%40v1.4.0
- With the EIP-7702-related release, a patch will be issued.
- Working on fixing BPO-related configuration.
- Next release expected by Tuesday/Wednesday, alongside Hive updates.
- Shared Hive dashboard: hive.ethpandaops.io/#/group/bal

Notes
- Nethermind failure reported: [Discord message](https://discord.com/channels/1359927674746835211/1424818910846386357/1435999369919725659)

TL;DR — Mario
- Check Hive for latest BAL results.
- Chime in with updates from your respective clients.
- Coordinate with Felipe, Toni, and Raxhvl for follow-ups.

### ePBS updates
Justin Traglia
Justin Tr
- Breakout call held last week — main focus: execution payment value
- Devnet-0 for ePBS expected mid-January at the earliest
- Next breakout scheduled for Dec 5

### Other Topics:

#### Upcoming ACD-T Calls
- Next week = in-person meetings, so no call
- Nov 24 call also canceled
- Back on Dec 1

#### Fusaka Readiness - Confidence Check
Saulius G.
- Asked how comfortable we feel with Fusaka progress given recent bugs

Barnabas (Chat)
- Most bugs are not Fusaka-related.
- Described it as “the smoothest upgrade so far.”

Pari
- “Smoother than the Merge 😛 — all testnets failed during Merge testing”

Ben (N/M)
- Ambitious, but releases are lining up well

Pari
- Offered testing bandwidth for any client teams needing extra validation.

Next steps:
- Wait on nimbus to triage issue in devnet 3
- Then coordinate another non-finality test

#### Post-PeerDAS Metrics

Łukasz Rozmej (Chat)
- Asked if mempool stats post-PeerDAS are available.

Pari
- Sam is investigating
- Sam investigating; Sunnyside Labs dashboards will be shared soon.

#### Client-Side Concerns
Kasey asked about handling:
- Private blobs, and
- Re-org heavy scenarios

Pari: Ref: https://github.com/flashbots/mev-boost/pull/855

# All Core Devs - Testing (ACDT) #60 (Nov 03, 2025)

**Next Steps:**

* Client teams to complete **release candidates by Nov 5** .
* **Testing complexity assessment** to start on prioritized EIPs for Glamsterdam.
* **Mainnet upgrade target remains Dec 3, 2025.**

BarnabasBusa facilitated the call.

## Fusaka:

### Sepolia BPO fork events/updates
- Last week’s BPO 2 went smoothly overall, except for an issue involving Prysm.

Manu (Prysm)
- [Summary TL;DR](https://hackmd.io/JLER6rPgTP6Uk0wd0yCpPA?view)
1. Starting from Hoodi/Fusaka, a few attester nodes are struggling — producing attestations where target.epoch - source.epoch is inconsistent.
2. These problematic nodes may or may not be Prysm.
3. When a Prysm node receives such an attestation, it can reject it, downscore the peer, and eventually disconnect the gossiping node.

The Prysm team has identified this as an attestation node issue and is working on a fix, expected later this week.

#### Other Client Reports – Hoodi

nflaig (Lodestar):
- Currently working on the Lodestar zero proofs bug.

Tbenr / Enrico (Teku):
- Discovered some bad blob sidecars circulating in the network with incorrect KZG commitments.
- The mainnet release addressing this is in progress and expected to be out shortly.

Manu (Prysm):
- Also observed Lodestar peers passing invalid KZG proofs.
- In such cases, Prysm will not verify these proofs since validation depends on the Execution Layer (EL).
~~
Tbenr (Teku):
- Confirmed that Teku behaves similarly to Prys~~m in this regard — if the issue originates from the EL, it could involve the Engine API.

nflaig (Lodestar):
- The bug fix is underway.
- Believes the “zero proof” issue stems from an internal buffer problem.
- Noted that major refactoring done a month ago might explain why it wasn’t detected earlier.

Manu (Prysm):
- Added that sometimes the problem may come from other clients, not necessarily Lodestar.
- Doesn’t appear to be critical, but worth noting since sidecar handling can occasionally trigger this issue.

#### Hoodi BPO is expected this week

## 60M gas limit on mainnet updates:

### State Test Updates

Kamil Chodola:
- Focused on stabilizing the data and testing environment — conditions are now more consistent and results are steady.
- The feedback loop has improved, enabling faster iteration.
- Currently adding a new EEST compute scenario; facing some issues generating genesis files and collaborating with Louis to resolve them.
- Working with Mario on sum repricing, aiming to identify potential repricing candidates for the Glamsterdam upgrade.

From the stateful testing perspective:
- The machine running other stateful tests needs a refresh — it’s currently down due to technical issues.
- Overall progress is solid, but metrics collection for repricing scenarios needs improvement going forward.

### Gas limit testing update

Pari:
- Noted that starting this week, the 60M gas limit is set as default.
- As the network progresses, gas limit increases should start to appear in testing results.

## Glamsterdam Testing Updates:

### BALer updates - bal-devnet-0

Stefan:
- The devnet is ready.
- Reth and Nethermind have signaled readiness; awaiting confirmation from Geth and Besu before launch.

Raxhvl:
- Shared latest BAL test results (v1.3.0):
- Overall: 98%
- Besu – 100%
- Reth – 100%
- Geth – 99%
- Nethermind – 91%
- A new release is coming soon, adding ~25 tests to cover coinbase changes.
- Results will be updated following the new release.

Stefan: Once all clients confirm readiness, the BALer devnet will go live.

Toni:
-Announced that the EIP-7928 breakout will take place on Wednesday at 14:00 UTC.

### ePBS Updates

Mario Vega:
- Shared that Justin T. provided updates on recent progress.
- There have been two new spec releases, each accompanied by a comprehensive README for better clarity and implementation guidance.
- [v1.6.0-beta.2](https://github.com/ethereum/consensus-specs/releases/tag/v1.6.0-beta.2)
- [v1.6.0](https://github.com/ethereum/consensus-specs/releases/tag/v1.6.0)

Barnabas:
- Noted that the focus will be on completing the BALer devnet first, with the ePBS devnet targeted for late November or early December.

## Other Topics:

### Mainnet Client Releases – Check Client Readiness

Nethermind (Kamil):
- Working on a new release.
- A minimized version will be released today, with a hotfix expected in the coming days.

Erigon (Andrew):
- Fusaka mainnet release is out today with the 60M gas limit enabled.

Teku (Enrico Del Fante / tbenr):
- Release in progress, expected within a few hours.
- The 60M gas limit was already set as default in a previous Teku release.

Reth (draganrakita):
- Planning to release on Wednesday.

Lodestar (Phil Ngo):
- Working on a release but currently blocked by the zero-proofs bug.
- A release is possible today if the fix lands; otherwise, it would be incomplete.

Nethermind (Kamil Chodoła):
- Asked whether there’s a deadline for client releases for coordination or blog inclusion.
- Noted that the release might slip to tomorrow (Nov 4) in a worst-case scenario.

Barnabas:
- Deadline for release candidates (RCs) is Wednesday, Nov 5.
- The EF blog post will go out by Thursday’s ACD, confirmed by Alex S.

Prysm (Manu):
- Tentative release this week; uncertain if it will meet the Nov 5 timeline.
- If the bug fix lands today, an RC build could be ready by Wednesday.

Lighthouse:
- No new updates shared.

Lodestar (Matthew):
- Identified a bug that only appears under larger network conditions.
- Currently soak-testing, and if stable, Lodestar’s release will also be ready by Nov 5.

#### Summary:
- Barnabas: Stay on track with the Dec 3 mainnet upgrade timeline.
- Aim for all **Client Releases by Wednesday, Nov 5**.
- If any client isn’t ready by Thursday, the mainnet timeline will be revisited.

### Testing Complexity Assessment Checklist

Mario:
- Currently gathering information for PFI Glamsterdam.
- [Tracking progress](https://github.com/ethsteel/pm/issues/7).
- The team will follow the testing complexity checklist for the Execution Layer (EL).
- Invited everyone to chime in with EIPs they believe should be prioritized.
- The testing team will prioritize assessing complexity and begin implementation for key proposals.
- If there are Consensus Layer (CL) EIPs needing prioritization, those can also be assessed subsequently.

Raúl Kripalani (Chat):
- Offered support from the networking team — happy to assist with reviews, testing, and implementation.

Mario:
- Emphasized the goal is to get a sentiment check for Glamsterdam EIP priorities.
- Based on impact and community feedback, the team will plan assessment and implementation work.

Ansgar:
- Mentioned that initial decisions on prioritized EIPs and DFIs may come during this week’s ACDE call.

Wolovim:
- Announced that the Forkcast rank page is back online for client team scoping: https://forkcast.org/rank

lightclient:
- Requested more clarity on prioritization, noting there are nearly 10 gas fee EIPs.

Marius:
- Suggested it’s best to quickly identify which EIPs to kick or deprioritize first.
- Barnabas agreed with this approach.

Mario Vega:
- Added that some proposals are under-specified, and it makes sense to exclude those first.

Barnabas:
- Asked for a Nethermind-maintained list to help other teams coordinate priorities.
- Marc confirmed this will be prepared.

lightclient:
- Reiterated that repricing EIPs are a high priority.
- Requested stronger testing team signaling to guide client focus.

Ansgar Dietrichs:
- Noted that Maria shared a document outlining bundling options for repricing proposals.
- The core EIPs in that document should be treated as highest priority within the repricing group.

Fredrik:
- Mentioned the Mainnet Upgrade & Incident Response Plan, similar to the Pectra model: [Fusaka Mainnet Plan](https://github.com/ralexstokes/pm/blob/fusaka-pm-docs/Fusaka/fusaka-mainnet-plan.md)
- Encouraged client teams to start identifying coordinators and backup coordinators.

#### Summary
Barnabas:
- Echoed the request for client teams to make PRs to the plan file with coordinator details.
- Expressed hope that by Wednesday/Thursday, the plan and coordination list will be complete.

# All Core Devs - Testing (ACDT) #59 October 27, 2025

Mario Vega facilitated the call

## Fusaka:
### Sepolia BPO fork events/updates
Barnabas
- BPO 1 fork happened last Monday.
* BPO 1 completed successfully
* BPO 2 expected in ~9 hours
* Everything running smoothly; no major client issues reported

Other updates
* Devnet 3 remains online — expected to continue for about a month
* Validators: please update your nodes and stay online as much as possible

### Holesky deprecation
- Holesky has been deprecated
- Migration recommended to Sepolia and Hoodi for continued testing
- Holesky stopped finalizing as of Oct 27 (10:32 AM UTC)

Minhyuk Kim
- Suggested Hoodi fork before Sepolia sunset to support L2 project testing

Barnabas
- Emphasized balancing between home staker and L2 operator needs

Marius van der Wijden
- Encouraged everyone to use Sepolia for L2 testing

Mario Vega
- Advised moving to Hoodi
- Confirmed fork sequence: Sepolia first, then Hoodi

## Gas Limit Testing Update
- 60M gas is now set as the default value across all clients.

### State Test Updates

Kamil
- All compute tests are running fine.
- Some scenarios are merging faster — currently under review.
- Major milestone: discovered a way to overcome a key stateful testing challenge.
- Implemented better result comparison methods.
- Shared the latest compute test updates.
- Found one issue in n/m, which has been fixed.
- Working on creating a custom buffer file for improved test handling.

## Glamsterdam Testing Updates:

### BALs

Raxhvl
- A new release is coming up with 20 additional test cases.
- Identified a bug during BAL testing.
- Shared a presentation detailing findings.
- Two bugs were missed earlier — one in Geth, another in Nethermind.
- Shared a small presnetation with suggestions.

Stefan
- Devnet is in progress.
- Invited CL client teams to reach out and join the devnet implementation efforts.

### ePBS
Justin T.
- Everything looks good so far; waiting for devnets to start.
- Next [ePBS breakout](https://github.com/ethereum/pm/issues/1783) is scheduled for Friday.
-Requested all client teams to join the upcoming meeting.

## Other Topics

### RPC Updates

Lukasz
- Two weeks ago, RPC enhancement discussions were initiated.
- Proposed changing the reverted error code to be free.
- Suggested a larger discussion may be needed around blob fields and related tests.

Keri
- Recommended opening an issue on the Execution API GitHub for further review by the team.

### EIP-7610 Discussion

Marius
- Provided overview that this EIP defines a specific scenario.
- Mentioned there are 28 such instances on mainnet that cannot be easily removed.
- Explained a theoretical example of deploying a contract to illustrate the challenge.

### Weld updates (EELS+EEST)
Dan provided a summary
- Weld focuses on migrating code from the execution-spec-tests repository to the execution-specs repository.
- This integration will improve alignment between the spec and testing processes.
- The team has been working intensively over the past few weeks, and an update has been posted — though it is still a work in progress (WIP).
- As a result, the old repositories are being phased out and `execution-spec-tests` will now contain the test fixtures, and test fixture releases, but no code.
- For Osaka, contributors can open pull requests directly in the new repository.
- Amsterdam contributors are advised to hold off temporarily until further notice.
- Regarding benchmarking, Louis has a large PR in progress to restructure workflows — expect significant upcoming changes.
- Client teams do not need to take any action at this stage.
- Updated testing documentation will be published on the official website once available (currently WIP).