# Ethereum consensus upgrade [application-layer perspective] **Warning:** This document is under active development. [toc] The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). ## Glossary - application-layer, application node, PoW node -- former eth1 client - consensus-layer, consensus node, PoS node -- beacon node, former eth2 client - application block -- block in application-layer, containing transactions and being executed - consensus block -- block in consensus-layer (in beacon chain), also wrapping an application block ## External consensus protocol and fork choice rule This section describes the changes required for the current Ethereum network to utilize a set of adjunct block validity and fork choice rules with the intention to replace the current PoW consensus algorithm with that of the beacon chain. The word "adjunct" implies that there is a consensus-layer (beacon chain) that is running side-by-side with core of today's Ethereum network, called the "application-layer" further in the doc. The new consensus-layer takes on full responsibility of block sealing, seal verification, tracking the head of the chain, and cryptoeconomic incentives contributing to this set of responsibilities. The changes described below do not affect the Ethereum state transition function, validity rules, and transaction pool management. ### Interaction between consensus and application layers The interaction is driven by the consensus-layer and is as follows. Consensus-layer sends a message with a subset of data down to the application-layer which takes corresponding action and responds accordingly. There are four main messages. They are listed below with the corresponding actions expected to be taken by the application-layer. - **AssembleBlock.** Creates new application-layer block on top of the given parent by packing a set of transactions from the transaction pool. Application-layer block fields are returned in the response. - **NewBlock.** Inserts the provided application-layer block into an application chain and updates the application state. Response is the block validity flag. Before inserting the block application MUST do all the necessary checks: - Verify that the block has been assembled correctly by the producer; - Verify that the transactions are executed correctly; - Verify that the state root matches the expected value; - Verify that the receipts root hashes are as expected. - **NewHead.** Notifies about the new head of the application-layer chain. Block/state storage reorgs to the block specified in the message. - **FinaliseBlock.** Specified block becomes finalised signaling to the application-layer that it will never be reverted in the future. Application-layer may do garbage collection of its chain data and take any other meaningful actions at this point. ### New block format #### Constants - **`DIFFICULTY = 1`** - **`UNCLES_HASH = keccak256(RLP([]))`** The application block format remains unchanged from eth1 except for the following list of fields. - **`ommersHash`**. Set to `UNCLES_HASH` -- with adjunct consensus rules, the uncle mechanism loses its meaning. - **`difficulty`**. Set to `DIFFICULTY`. - **`extraData`**. Set to an empty byte string. - **`mixHash`**. Set to a string of zero bytes. - **`nonce`**. Set to `0`. To avoid the redundancy in storing these fields, the consensus-layer may store stripped down version of the application block. In this case application-layer must verify that the block has been correctly assembled by matching the expected block hash given by consensus-layer with the actual one. ### Changes to block processing This section describes the changes that are introduced to block processing. Most processing and validations remain stable and are not explicitly documented here. For instance, the EVM execution component of block processing with state and receipts root validity checks remains unchanged. #### Consensus and application block trees In general, there is a one-to-one relation between consensus and application blocks (there could be a rare case when many consensus blocks relate to one application block). Thus, the application block tree reflects the one that is maintained by the consensus-layer. The consensus-layer is responsible for ensuring that the block tree is consistent. Data passed to the application-layer must always have its prerequisites already satisfied. For example, the case when **`NewBlock`** message has a reference to an unknown parent block is an exceptional (fatal) case and MUST fail accordingly. This implies that consensus messages may be (and most often *are*) causally dependent and MUST be processed sequentially. #### Block format/hash validity The consensus-layer works with stripped down version of the application block (i.e. no references to stubbed fields like `omersHash` or `difficulty`). Thus the application-layer must verify its consistency with regard to the hash of the block passed in by the consensus-layer. Namely, the application-layer must assemble a block header with the data passed from the consensus, filling absent fields with constants described in the [block format section](#New-block-format). Then validate that given block hash equals to `keccak256(RLP(BlockHeader))`. #### Timestamp **`timestamp`** field of a block is determined by the consensus-layer and effectively is set to the beginning of a slot which the block is being produced. Data subset of **`AssembleBlock`** and **`NewBlock`** messages explicitly contain the **`timestamp`** field value. Any validation of the **`timestamp`** field should be removed from the block processing as it's pre-validated by the consensus-layer. #### Difficulty and nonce Difficulty field in a block is verified by matching it to the `DIFFICULTY` constant which is implicitly done by the block format validity rule. The same is applied to the `nonce`, `extraData`, and `mixHash` values. #### Legacy rewards Ommers validation and block rewards are deprecated. The application layer no longer issues ETH. *Note*: Transaction Fees are still processed as per the rules of the application-layer, with non-burnt fees going to the address specified in the **`beneficiary`** field of a block. ### External fork choice rule The fork choice rule relying on total difficulty as a function of weight is replaced with external fork choice where the head of the chain is only changed upon the two following events: - `NewHead` message is received from the consensus-layer. The block specified by the message becomes a new head of the chain. - `NewBlock` message block is built on top of current head of the chain. The block becomes a new head if it's been processed and considered as valid. `NewHead` may point to the same block that has already been set by `NewBlock`. In this case the redundant `NewHead` should be ignored. The other approach that clients may take in maintaining chain head is to change the head upon receiving `NewHead` message only, i.e. do not trigger the fork choice on `NewBlock`. ## Network ### Block gossip Block gossip is deprecated on the application-layer. The reason is inability to verify block seal and propagate a block securely. Consensus network has its own overlay for block gossip, bits of application block data are passed down to the application-layer via `NewBlock` messages. Consensus-layer takes responsibility of keeping the application chain up-to-date upon turning the client into "online" mode by keeping up with the head and passing all the blocks required to build the chain down to the application-layer. ### State sync proposal State storage management, state format and sync strategy remain unchanged. Which means that state downloader may use `fast`, `snap`, or any other available network protocol in state downloader. Instead of being evaluated with the data received from the network, the head of the chain is passed directly to the state downloader via `NewBlock` and `NewHead` messages received from consensus-layer. State downloader bootstrap process may look as follows: 1. Application-layer starts with empty chain and state storages. 2. **`NewBlock(blockHash=X)`** is received from the consensus layer. 3. **`NewHead(blockHash=X)`** is received after the previous message. 4. State downloader starts with **`Block(X).stateRoot`**. While being in the sync mode, the application-layer may process all the messages coming down from the consensus-layer except for **`AssembleBlock`** which MUST be responded with error code. The other exception is the **`NewBlock`** message. Transaction execution and post-execution verifications must be postponed until state gets fully downloaded. While the application-layer may keep constructing its chain and track its head if this is required by state sync implementation. Chain data received from the consensus-layer should be enough to keep state downloader up-to-date with the state of the network. Namely, updating it with the **`stateRoot`** referred by the head of the chain. Headers, receipts and block bodies may start being downloaded once state sync gets bootstrapped. Ethash verification of blocks produced by PoW may be omitted without loss in security. Once the state gets fully downloaded and the application-layer becomes capable of executing blocks, it should notify consensus-layer accordingly. It is proposed to enhance validity flag in the response to **`NewBlock`** message to the status code with the following values: - `valid/invalid` to address block validity - `in_sync` to highlight the sync mode Application-layer is considered to be fully synced once the consensus-layer receives the first `valid` response to the **`NewBlock`** message. *Note:* Historical headers, blocks and receipts are not required for block execution and should not affect `in_sync` status of the application-layer, exception is the most recent `256` headers. ### Block sync proposal Though, forward sync strategy becomes unavailable without consensus-layer involvement into the process, the underlying network protocol may be re-used in a different approach to the block sync. Block sync may look as follows: 1. Application-layer starts with empty chain and state storages. 2. **`NewBlock(blockHash=X)`** is received from the consensus layer. 3. **`NewHead(blockHash=X)`** is received after the previous message. 4. Header downloader retrieves headers in reversed order starting from received block. 5. New blocks that are keep coming from the consensus-layer are kept in the application block tree. 6. Blocks start being downloaded and processed once the genesis block is reached. 7. Block sync is considered to be finished once the application-layer becomes capable of executing a block sent with subsequent **`NewBlock`** message. Once that happens consensus-layer gets notified accordingly. When the sync processed is finished, application-layer stops downloading blocks and keeps building its chain with blocks received from consensus-layer with **`NewBlock`** messages. Ethash verification of blocks produced by PoW may be omitted without loss in security. *Note:* This strategy may have significant time increase comparing to the usual forward sync. ## Transition process Transition process (a.k.a. docking procedure) includes producing and finalising the first PoS block on top of the last PoW block. Application-layer switches between three subsequent operation modes during the transition. 1. **PoW mode.** Application-layer operates as normal. The chain is secured by PoW, thus, all PoW validity conditions are maintained and fork choice rule remains unchanged. 2. **Transition mode.** PoW blocks are received from the wire, verified and processed as usual. It implies missing ancestor blocks are being requested from other peers. Total difficulty rule is replaced by the [external fork choice rule](#External-fork-choice-rule). 3. **PoS mode.** Block gossip is disabled implying that no PoW blocks are processed and propagated accross the network. Consensus-layer messages becomes the only source of application blocks. Switching between modes is a one-way action which is triggered by corresponding consensus-layer message. - **PoW -> Transition.** First **`NewHead`** message received. The block specified by the message is set as a new head of the application chain. All subsequent head updates must be controlled by the external fork choice rule. - **Transition -> PoS.** First **`FinaliseBlock`** message received signalling the end of transition period. Block propagation must be disabled. Transition conditions are checked by consensus-layer and are transparent to the application-layer. Consensus-layer messages may have different meaning depending on the mode. - **PoW mode** - **`AssembleBlock`**. Transition conditions are met and the first PoS block is being produced. - **`NewBlock`**. Transition conditions are met, the first PoS block has been produced and this block is being processed now. If ancestors of the block are missed then they must be requested from the network and processed accordingly before processing the block itself. - **`NewHead`**. The first PoS block has been set as the head of the chain. Application-layer must change the head accordingly and turn into Transition mode. - **`FinaliseBlock`** is not expected at this point. - **Transition mode** - **`AssembleBlock`**, **`NewBlock`**, **`NewHead`** has no special meaning other than the one that was described above. - **`FinaliseBlock`**. The first PoS block has been finalised, signals the end of transition period. Application-layer must turn into PoS mode. - **PoS mode** - No special cases for any of the messages. Application-layer operates under PoS consensus.