Pectra Retrospective

This post is an overview of the Teku team's Pectra retrospective. It exists to (try to) consolidate the individual and shared opinions of team members into a somewhat structured document.

Summary

• It is impressive to see how much we have evolved in our testing capabilities, largely due to the support from EthPandaOps and the tooling they have developed. We cannot stop praising them for all their work and how much more productive we are because of their help.
• It is really good to see many teams working together and collaborating towards making the protocol better. Interop events and Eth R&D workshops have been super valuable.
• Scope creep has impacted our productivity and workflow and made us take longer to deliver value. We should try to have a better understanding of the features going into a fork earlier in the cycle, and converging into a stable scope sooner.
• Ideally, every fork should have a clear goal (associated with the Ethereum roadmap). This can help core-devs to understand where to focus and also help us during the process of scoping and prioritizing features.
• We need to rethink the way we make decisions about EIP inclusion in a fork. Ideally, we want more mature EIPs being scheduled, after teams have had time to understand their value, impact and implementation details.
• The separation between ACDE and ACDC seems artificial and creates more harm than good. We believe that we should move towards having a unified call, to talk about scope, priorities and governance related to the protocol. Separate “technical” calls can be held between Execution teams and Consensus teams to discuss anything relevant just to one of the layers.

Points of Reflection

Longer version of some thoughts directly related to Tim's prompts on Pectra Retrospective (EthMagicians).

Over the past year, what went worse than expected in Pectra’s development and/or ACD generally? How could we have prevented this?

• Pectra scoping: No deadline to freeze the EIPs inclusion. Even after splitting the fork to deliver it faster, we kept adding features and delaying the shipping date. The Pectra scope should have been frozen earlier.
• Pectra did not have a clear goal based on the global Ethereum roadmap and ended up including too many features. We should have a clear mapping between the scheduled fork and the roadmap.
• Spent time on some EIPs that didn’t make it to the fork in the end: Inclusion Lists, Uncoupling blob limits per block across CL and EL, Stable Containers… We should avoid early over-committing and give time for EIPs to be more discussed and estimated.
• Underestimated the complexity of EIP7549 (Moving committee index outside Attestation) which even required some additional changes as a next step. This could have been avoided by allocating more time to discussions and some initial prototyping.
• The “we need more blobs“ movement came too late and further disrupted the fork schedule.

Over the past year, what went better than expected in Pectra’s development and/or ACD generally? What caused this?

• Kurtosis and EF Devops support significantly improved testing efficiency and allowed for quick, reproducible scenarios. Devnets were set up quickly, and monitoring tools provided clear feedback on protocol performance.
• A lot of long-wished features are coming into the protocol thanks to great work and good spec coming out of R&D.
• Interop saw a lot of progress for Pectra: Good developer representation and many teams were very ready for devnets.
  Looking forward, what should we start doing, stop doing and/or do differently?

We should start

• Keep the scope of forks small to ship them faster and rapidly react to changes when needed. It is hard to say how short we want the interval between forks to be. Too fast will likely be an issue for other stakeholders like institutional staking providers and L2s to keep up. However, we have observed that having a fork every 1 or 1.5 years is not enough to keep up with demand. We believe that somewhere between 6 to 9 months would be ideal.
• Freeze the next fork scope when the current fork is delivered as suggested by Tim.
• Start Featursnets / EIPnets asap to challenge the EIPs designs and better estimate the required workload to deliver them. The SIG (special interest group) concept, suggested by ileuthwehfoi, is interesting and could bring some structure around different areas of the protocol.
• Stick to a "few forks" global roadmap: This could help us make the scoping process easier and give ACD and the community a clear direction. Vitalik's roadmap is a good starting point, we should make sure the planned fork fits into it and see where.

We should stop

• Estimating features 'on a hunch' (eg. attestations refactoring being two lines of code change).
• Long presentations and technical in-depth discussions during ACD calls. The ACD calls should be the place to rapidly pitch ideas and make quick, final decisions to show the direction.

We should do differently

• The ACD calls agenda should be shared and filled more in advance. Usually, topics to discuss and the agenda summary are added/updated a few days/minutes before the call. It is hard for members of the core devs community living in a timezone that is not compatible with core dev calls to engage and share their opinions in a timely manner if we do not have time between a proposal or idea being added to the agenda and a decision being made on that same cycle.

What should we not compromise on, even if there is pressure to do so?

• Keep the cooperation and communication among teams even though the number of clients is getting bigger.
• We should not make any change to the fork scope once frozen unless the protocol is at risk.

What parts of ACD are “legacy process debt”, which we should consider rethinking from first principles?

• Revisiting ACD calls to focus on decision-making and separating technical discussions and CFI/SFI processes to be handled asynchronously on Ethereum Magicians or Discord could streamline the process and enhance efficiency.
• Remove the separation between ACDE and ACDC and have a unified governance/scoping call and separate technical calls (Special Interest Groups could help to achieve this).