EIP-4444 Implementation Plan: History Expiry in Ethereum

# EIP-4444 Implementation Plan: History Expiry in Ethereum ## Overview This document outlines the agreed-upon implementation plan for historical data expiry in Ethereum execution clients, as discussed and agreed upon during the L1 Workshop in Bangkok, November 2024. This represents the first concrete step toward shipping [EIP-4444](https://eips.ethereum.org/EIPS/eip-4444). ## Timeline ### Phase 1: Pre-Merge Data Expiry * **Timeline**: May 1, 2025 * **Scope**: Allow for dropping of block bodies and receipts from pre-merge blocks by other clients. * **Status**: Agreed upon by all major client teams (Besu EthereumJS, Erigon, Geth, Nethermind, Nimbus, Reth) ### Phase 2: Rolling window & Header Retention * **Timeline**: Not set * **Scope**: Allow for dropping of block bodies and receipts as they pass beyond a fixed rolling window near the head of the chain * Rolling window implementation * Needs to be clearly demonstrated that we have a multi layered approach to preserving this data. * Archival formats for blocks closer to the HEAD are being generated and standardized format (such as ERA). * P2P solution for post-merge is agreed upon and supported by clients. * Dropping headers * Not required for rolling window. * Headers represent a small portion of the data and so dropping them is of lower value. * Clients likely need time to operate under the new paradigm before they can explore whether the engineering work to also drop headers is worth the savings. ## Agreed-Upon Changes ### What Will Change 1. Execution clients may drop all pre-merge block bodies and receipts from their local storage 2. Clients are no longer required to respond to devp2p or JSON-RPC requests for pre-merge block bodies and receipts 3. Clients may return errors for requests seeking pre-merge historical data 4. Clients may continue serving pre-merge data if they choose not to implement 4444s ### What Remains Unchanged 1. Pre-merge block headers must still be served over DevP2P 2. Post-merge historical data requirements remain unchanged ## Client Implementation Plans ### Geth * Solution: Portal Network * Status: Planning stage, implementation beginning shortly ### Reth * Short-term: Relying upon era files / torrents from a server * Long-term: Possible Portal Network implementation * Concerns: Sync performance for Reth clients, who will only be able to fetch pre-merge history data from other Reth clients on the network, significantly slowing the sync process. ### Erigon * Solution: EIP-7801 & native torrents * Status: Planning phase for 7801, torrents may already be live. ### Besu * Short-term: Look into EIP-7801 * Long-term: Portal Network * Status: Planning phase ### Nethermind * Dual approach: Portal Network and EIP-7801 * Status: Implementation phase ### Nimbus * Solution: Portal Network * Status: Implementation phase ### EthereumJS * Solution: Portal Network * Status: Implementation phase ## Technical Solutions ### Portal Network * Current status: Live and operational * Specification: [Portal Network Specs](https://github.com/ethereum/portal-network-specs) * Reliability: 100% for pre-merge data via [Glados monitoring tool.](https://glados.ethdevops.io/) * Data Retrieval Patterns: * Block hash → header lookups * Block hash → body (transactions, uncles, withdrawals) * Block hash → receipts * Block number → header lookups * Fixed vs. Variable data radius * In the Portal Network, the data radius defines what chunk of data a client is responsible for storing * Clients will need to assess what is their ideal strategy for managing db usage * Fixed-Size DB & variable data radius * Pro: Fixed sized disk usage. * Con: Implementation complexity for pruning data from local database. * Fixed data radius & variable-size db (simplest to implement) * Pro: Simplest to implement * Con: Database size will grow slowly over time. * Future developments: * Block range queries (in development) to support full sync * State network / Beacon light client network integration w/ EL clients ### EIP-7801 * Purpose: Standardize historical data serving, by nodes advertising the range of block data they store * Updated specification allows blocks to record the current & future ranges being stored by their peers * Link to spec: [EIP-7801](https://eips.ethereum.org/EIPS/eip-7801) ### ERA Format * Current usage: Pre-merge data export/import * Not intended as primary storage format * Provides proving capabilities * ERA1 files will be stored on multiple servers to act as a fallback strategy for recovery in case of data loss ### Erigon Torrents * Clients may choose to leverage Erigon's torrent strategy to source this data ## Implementation Considerations 1. Sync Performance * Clients must evaluate impact on sync strategies * Potential degradation of network performance during transition if clients don't choose to implement 4444s * Need for fallback mechanisms that prohibit permanent data loss 2. Data Availability * Portal Network serves as an method for granular on demand retrieval of data. * ERA files & torrents serve as a method for bulk retrieval of data. * Confidence in data loss primarily through ERA files being redundantly generated and stored on multiple servers. 3. Networking Stack * Portal (Discv5/Udp) & 7801 (DevP2P/TCP) are based on different networking stacks. What are the tradeoffs involved between these two? ## Path to Rolling Window Implementation ### Current Limitations 1. Security considerations for post-merge data recovery 2. Consensus client requirements for deposit contract logs 3. Storage format standardization 4. Proving format standardization ### Requirements for Future Agreement 1. Secure & tested recovery mechanisms for post-merge data 2. Agreement on window size - 1 million blocks, enables EL & CL to share a db - simply agree on a new threshold every year - simple to implement, but requires annual maintenance 3. Solution for deposit contract log accessibility ### Next Steps 1. Client teams to implement and test 4444s solutions 2. As clients implement the pre-merge solution, they should test & evaluate various storage/proving strategies and how they can be applied to the future rolling window target. ## Regular Review and Updates This document will be updated as implementation progresses and new agreements are reached. Major changes will be tracked and dated. ## References * [EIP-4444](https://eips.ethereum.org/EIPS/eip-4444) * [EIP-7801](https://eips.ethereum.org/EIPS/eip-7801) * [Portal Network Specifications](https://github.com/ethereum/portal-network-specs) * [Portal Hive Testing Framework](https://github.com/ethereum/hive/tree/master/simulators/portal) * [L1 Workshop Notes (November 2024)](https://notes.ethereum.org/_XVO7jmXTGOwZmhR5-3T9Q)