EIP-4444 Implementation Plan: History Expiry in Ethereum

Overview

This document outlines the agreed-upon implementation plan for historical data expiry in Ethereum execution clients, as discussed and agreed upon during the L1 Workshop in Bangkok, November 2024. This represents the first concrete step toward shipping EIP-4444.

Timeline

Phase 1: Pre-Merge Data Expiry

  • Timeline: May 1, 2025
  • Scope: Allow for dropping of block bodies and receipts from pre-merge blocks by other clients.
  • Status: Agreed upon by all major client teams (Besu EthereumJS, Erigon, Geth, Nethermind, Nimbus, Reth)

Phase 2: Rolling window & Header Retention

  • Timeline: Not set
  • Scope: Allow for dropping of block bodies and receipts as they pass beyond a fixed rolling window near the head of the chain
  • Rolling window implementation
    • Needs to be clearly demonstrated that we have a multi layered approach to preserving this data.
    • Archival formats for blocks closer to the HEAD are being generated and standardized format (such as ERA).
    • P2P solution for post-merge is agreed upon and supported by clients.
  • Dropping headers
    • Not required for rolling window.
    • Headers represent a small portion of the data and so dropping them is of lower value.
    • Clients likely need time to operate under the new paradigm before they can explore whether the engineering work to also drop headers is worth the savings.

Agreed-Upon Changes

What Will Change

  1. Execution clients may drop all pre-merge block bodies and receipts from their local storage
  2. Clients are no longer required to respond to devp2p or JSON-RPC requests for pre-merge block bodies and receipts
  3. Clients may return errors for requests seeking pre-merge historical data
  4. Clients may continue serving pre-merge data if they choose not to implement 4444s

What Remains Unchanged

  1. Pre-merge block headers must still be served over DevP2P
  2. Post-merge historical data requirements remain unchanged

Client Implementation Plans

Geth

  • Solution: Portal Network
  • Status: Planning stage, implementation beginning shortly

Reth

  • Short-term: Relying upon era files / torrents from a server
  • Long-term: Possible Portal Network implementation
  • Concerns: Sync performance for Reth clients, who will only be able to fetch pre-merge history data from other Reth clients on the network, significantly slowing the sync process.

Erigon

  • Solution: EIP-7801 & native torrents
  • Status: Planning phase for 7801, torrents may already be live.

Besu

  • Short-term: Look into EIP-7801
  • Long-term: Portal Network
  • Status: Planning phase

Nethermind

  • Dual approach: Portal Network and EIP-7801
  • Status: Implementation phase

Nimbus

  • Solution: Portal Network
  • Status: Implementation phase

EthereumJS

  • Solution: Portal Network
  • Status: Implementation phase

Technical Solutions

Portal Network

  • Current status: Live and operational
  • Specification: Portal Network Specs
  • Reliability: 100% for pre-merge data via Glados monitoring tool.
  • Data Retrieval Patterns:
    • Block hash → header lookups
    • Block hash → body (transactions, uncles, withdrawals)
    • Block hash → receipts
    • Block number → header lookups
  • Fixed vs. Variable data radius
    • In the Portal Network, the data radius defines what chunk of data a client is responsible for storing
    • Clients will need to assess what is their ideal strategy for managing db usage
      • Fixed-Size DB & variable data radius
        • Pro: Fixed sized disk usage.
        • Con: Implementation complexity for pruning data from local database.
      • Fixed data radius & variable-size db (simplest to implement)
        • Pro: Simplest to implement
        • Con: Database size will grow slowly over time.
  • Future developments:
    • Block range queries (in development) to support full sync
    • State network / Beacon light client network integration w/ EL clients

EIP-7801

  • Purpose: Standardize historical data serving, by nodes advertising the range of block data they store
  • Updated specification allows blocks to record the current & future ranges being stored by their peers
  • Link to spec: EIP-7801

ERA Format

  • Current usage: Pre-merge data export/import
  • Not intended as primary storage format
  • Provides proving capabilities
  • ERA1 files will be stored on multiple servers to act as a fallback strategy for recovery in case of data loss

Erigon Torrents

  • Clients may choose to leverage Erigon's torrent strategy to source this data

Implementation Considerations

  1. Sync Performance

    • Clients must evaluate impact on sync strategies
    • Potential degradation of network performance during transition if clients don't choose to implement 4444s
    • Need for fallback mechanisms that prohibit permanent data loss

  2. Data Availability

    • Portal Network serves as an method for granular on demand retrieval of data.
    • ERA files & torrents serve as a method for bulk retrieval of data.
    • Confidence in data loss primarily through ERA files being redundantly generated and stored on multiple servers.

  3. Networking Stack

    • Portal (Discv5/Udp) & 7801 (DevP2P/TCP) are based on different networking stacks. What are the tradeoffs involved between these two?

Path to Rolling Window Implementation

Current Limitations

  1. Security considerations for post-merge data recovery
  2. Consensus client requirements for deposit contract logs
  3. Storage format standardization
  4. Proving format standardization

Requirements for Future Agreement

  1. Secure & tested recovery mechanisms for post-merge data
  2. Agreement on window size
    • 1 million blocks, enables EL & CL to share a db
    • simply agree on a new threshold every year
      • simple to implement, but requires annual maintenance
  3. Solution for deposit contract log accessibility

Next Steps

  1. Client teams to implement and test 4444s solutions
  2. As clients implement the pre-merge solution, they should test & evaluate various storage/proving strategies and how they can be applied to the future rolling window target.

Regular Review and Updates

This document will be updated as implementation progresses and new agreements are reached. Major changes will be tracked and dated.

References

Select a repo