With the introduction of new modules, the Oracle third phase report might not fit into a single transaction. This document presents an approach for implementing a multi-transactional third phase.
The third phase in the Accounting Oracle plays a crucial role in providing updated counts of stuck and exited validators for each node operator within the Lido protocol. This data required for distributing rewards and allocating new stake among node operators. Currently, the third phase is restricted to 200 Node Operators updates due to node operators limit in NOR contract and Ethereum block size limitations, which is sufficient for the Curated Module and Simple DVT modules.
However, with the introduction of the Community Staking Module, which don't impose any limits on node operator counts, updates for all node operators cannot be accommodated within a single transaction due to block gas limitation.
The impending launch of new staking modules within Lido necessitates a scalable solution for accommodating an increasing number of node operators.
The Accounting Oracle report can require an update for all Node Operators in Lido. A very large transaction could potentially take longer to process or even stall, as validators might delay including it due to its size and the commitment of a large portion of a block's gas limit.
The third phase of the Accounting report has two main parts:
The finalization hook is out of scope for this document. Detailed improvements for it are described in the Reward distribution document.
As for Node Operator states updates, the estimates for the CSM module indicate that the report for 1000 Node Operators consumes approximately 37 million of gas. However, this estimate appears outdated. After reviewing the current code, 7.8 million gas for 1000 Node Operator updates seems like a more reasonable estimation.
Per Operator Cost:
Total per operator cost: 7800 gas.
For 1000 Operators:
24,700 (base cost) + 1000 * 7800 (per operator cost)
24,700 + 7,800,000 = 7,824,700
In the case of a growing CSM module and adding new SimpleDVT modules, we might be at risk that accounting oracle report will not fit into the single transaction.
To address these challenges, a multi-transactional approach is proposed for the third phase of the Accounting Oracle report. The second-phase report is assumed to contain the hash of the first third-phase transaction, while each third-phase transaction will include the hash of the subsequent third-phase transaction. The last transaction will containZERO_HASH
.
Proposed design ensures that each transaction of the third phase has been confirmed through the hash consensus of oracles during the first phase. It is not possible to make changes to any transaction without altering the report hash, thereby maintaining the integrity of the report. The process for verifying each transaction is described in the Implementation Details section.
The current third phase single transaction data format is as follows:
| 3 bytes | 2 bytes | X bytes | ...
| itemIndex | itemType | itemPayload | ...
The new version will include the nextHash
:
| 32 bytes | array of items
| nextHash | ...
Example
| 32 bytes | 3 bytes | 2 bytes | X bytes | ...
| nextHash | itemIndex | itemType | itemPayload | ...
Once the delivery of updates is complete, the Accounting Oracle triggers the Finalization hook (call Staking RouteronValidatorsCountsByNodeOperatorReportingFinished
). This signalizes that all Node Operator updates have been successfully delivered and the Accounting Oracle report is finalized.
Currently, in the Curated Module and Simple DVT modules, the finalization hook initiates reward distribution.
No changes are required in the case when the third phase report is empty.
The second-phase report will contain ZERO_HASH
The AccountingOracle's submitReportExtraDataEmpty
is called, which validates the report state and calls the finalization hook (Staking Router onValidatorsCountsByNodeOperatorReportingFinished
).
In the second phase, accounting oracle contract save in storage the hash of the first transaction's from the series of third-phase transactions.
During the third phase, we then verify that the hash of the report matches the expected hash previously stored in the state. Upon confirming a match, we proceed to process the items and update the storage with the next expected hash, as indicated by the report's nextHash
field.
After processing the transaction portion of the report data (items), the ExtraDataSubmitted
event will be emitted.
event ExtraDataSubmitted(uint256 indexed refSlot, uint256 itemsProcessed, uint256 itemsCount);
Currently the third phase report data is split into items based on composite keys. These keys include the itemType, moduleId, and the first node operator ID in each item.
The proposed design for multi-transaction processing assumes that dividing the report into items will maintain the integrity and sequence of the report data. Report data must not be sent in an unsorted order, and each transaction must contain sorted items. The sorting order is maintained across all third phase transactions.
For more details, check the “Sanity checks” section below.
Each transaction consists of items, with each item representing a portion of the report data. A new parameter, maxItemsPerExtraDataTransaction
, is suggested to define the maximum count of items in a single transaction.
// Maximum count of items in a third phase transaction
maxItemsPerExtraDataTransaction
The exact item count will be based on the following fundamental assumptions:
The gas-based calculations provided below are intended to illustrate our method for determining the specific value for the maximum item count. The Oracle will not store any gas limits; they are used solely for the purpose of calculating the maximum item count in transactions.
// More accurate values will be determined
// after a precise calculation of gas consumption for each module update.
// Maximum gas cost for processing a third phase transaction
maxExtraDataTransactionGasCost = 8_000_000;
// Maximum gas cost for processing a single item
maxExtraDataTransactionItemGasCost = 1_000_000;
// Maximum item count in a third phase transaction
maxItemsPerExtraDataTransaction =
maxExtraDataTransactionGasCost / maxExtraDataTransactionItemGasCost;
The proposed design does not restrict the format of the data that can be sent in an item's payload. However, as mentioned in the previous section, the processing of each item must not exceed the maximum gas cost allocated for an item.
Each item type may implement its own sanity checks to ensure data integrity and adherence to gas limitations.
Currently, for all modules (Curated, SimpleDVT, and CSM), the oracle's third phase supports only two types of items:
Due to the similarity between updating exited and stuck validators count, a single maxNodeOperatorsPerExtraDataItem
can be used to ensure that the data processing for an item does not exceed the gas limit for an item.
// Maximum node operator updates per item
maxNodeOperatorsPerExtraDataItem
The gas-based calculations provided below are intended to illustrate our method for determining the specific value for the maximum node operator updates per item. The Oracle will not store any gas limits; they are used solely for the purpose of calculating the maximum node operator updates per item.
// Maximum node operator updates per item
maxNodeOperatorsPerExtraDataItem =
maxExtraDataTransactionItemGasCost / maxExtraDataTransactionNodeOperatorUpdateGasCost
Where maxExtraDataTransactionNodeOperatorUpdateGasCost
is defined as the maximum gas cost for updating the count of stuck or exited validators for a single node operator across all modules.
Let's assume that M1, M2, and M3 represent the gas consumption for updating a single node operator (either stuck or exited validators count) in the respective modules with IDs 1, 2, and 3. The maximum gas consumption across these modules for node operator count updates (stuck and exited) is then calculated as follows:
maxExtraDataTransactionNodeOperatorUpdateGasCost = max(
M1_nodeOperatorUpdateStuckValidatorsCountGasCost,
M1_nodeOperatorUpdateExitedValidatorsCountGasCost,
M2_nodeOperatorUpdateStuckValidatorsCountGasCost,
M2_nodeOperatorUpdateExitedValidatorsCountGasCost,
M3_nodeOperatorUpdateStuckValidatorsCountGasCost,
M3_nodeOperatorUpdateExitedValidatorsCountGasCost
)
The concrete value of maximum operator update gas cost will be determined after a precise calculation of the gas consumption.
The proposed multi-transaction design specifies the maximum item count for each third-phase transaction without dictating the method by which the oracle will pack items within these transactions.
Suppose the maximum item count per transaction is 3, and the oracle's third-phase report contains 4 items that need to be packed into 2 transactions.
// maximum items count per transaction
maxItemsPerExtraDataTransaction = 3
// third phase report contais 4 items
totalItemsCount = 4
Total: 4 items, will be split on 2 transactions
The most advisable strategies are:
The first strategy is the easiest to implement, but the large transactions may take longer to be included in a block.
We might consider the even distribution approach because, in this scenario, the processing time for each transaction should be equal. However, implementation could be challenging because the algorithm must take into account both the number of items and the actual payload of these items.
Currently, the churnValidatorsPerDayLimit
parameter from the OracleReportSanityChecker
is used to validate both newly exited validators checkExitedValidatorsRatePerDay
and newly deposited validators _checkAppearedValidatorsChurnLimit
. This threshold is set at 20,000 validators, a suitable number for scenarios that involve adding a large volume of new validators. However, this threshold proves excessively high for the validator exit process, where the Ethereum churn limit should be taken into account.
We propose dividing this into two distinct parameters, each with more precise values tailored to the respective cases.
On the second phase the checkExitedValidatorsRatePerDay
sanity check will use new exitedValidatorsPerDayChurnLimit
. And the _checkAppearedValidatorsChurnLimit
check will use the value from appearedValidatorsPerDayLimit
.
After the third phase for each module we check that total count of exited validators is equal to the exited validators count which we got on the second phase. (StakingRouter line 567)
// https://docs.lido.fi/guides/verify-lido-v2-upgrade-manual#depositsecuritymodule
// The maximum number of validators that Lido can deposit per day.
/**
* const BLOCKS_PER_DAY = (24 * 60 * 60) / 12 = 7200
* const DSM_MAX_DEPOSITS_PER_BLOCK = 150
* const DSM_MIN_DEPOSIT_BLOCK_DISTANCE = 25
*
* const appearedValidatorsPerDayLimit = BLOCKS_PER_DAY / DSM_MIN_DEPOSIT_BLOCK_DISTANCE * DSM_MAX_DEPOSITS_PER_BLOCK // 43200
*/
appearedValidatorsPerDayLimit: 43200
/**
* const currentValidatorsNumber = 1385929
* const ACTIVATION_CHURN_LIMIT = 8
* const VALIDATORS_PER_DAY = (BLOCKS_PER_DAY / 32) * ACTIVATION_CHURN_LIMIT
* const VALIDATORS_AFTER_TWO_YEARS = VALIDATORS_PER_DAY * 365 * 2 + currentValidatorsNumber // 1385929 + (1800 * 365 * 2)
*
* const INITIAL_CHURN_LIMIT = 7
* const CHURN_LIMIT_QUOTIENT = 65_536
* const VALIDATORS_WITH_INITIAL_CHURN_LIMIT = 500_000
*
* const exitedValidatorsPerDayChurnLimit =
* Math.floor((VALIDATORS_AFTER_TWO_YEARS - VALIDATORS_WITH_INITIAL_CHURN_LIMIT) / CHURN_LIMIT_QUOTIENT) +
* INITIAL_CHURN_LIMIT // 40
*
* const maxExitedValidatorsPerDay = exitedValidatorsPerDayChurnLimit * (BLOCKS_PER_DAY / 32) // 9000
*/
exitedValidatorsPerDayChurnLimit: 9000
Above is an example calculation and recommended configuration. In our calculations we rely on the DSM limits. This number can be specified much lower, but its calculation will be dynamic, where the limits can vary greatly due to the number of rewards, withdrawals. We therefore recommend the use of static limits based on data from the DSM.
Will be used as a value in the check: _checkAppearedValidatorsChurnLimit
In the calculation of this parameter, we calculate the growth of CHURN_LIMIT two years ahead. This parameter will be valid until the implementation of MEB.
Will be used as a value in the check: checkExitedValidatorsRatePerDay
Currently, during the third phase of report processing, the Accounting Oracle validates that items are sorted according to composite keys. These keys include the itemType, moduleId, and the first node operator ID in each item.
// | 2 bytes | 19 bytes | 3 bytes | 8 bytes |
// | itemType | 00000000 | moduleId | firstNodeOpId |
uint256 sortingKey
The Accounting Oracle stores the lastSortingKey
in ExtraDataIterState
and updates it during processing the third phase transaction.
The proposed design for multi-transaction processing assumes that dividing the report into items will maintain the integrity and sequence of the report data. The lastSortingKey
will be reused to validate the order of items within multiple transactions.
Currently the OracleReportSanityChecker
contains parameters to establish the maximum number of node operators that can be updated during the third phase of the report.
Here is the current configuration for the on-chain sanity check, designed to accommodate two modules: Curated and SimpleDVT. It is configured to handle the delivery of up to four data lists. Each list represents either stuck or exited validators for node operators within a specific module, with no single list exceeding 50 node operators. In total, this allows for up to 200 updates concerning node operators.
"maxAccountingExtraDataListItemsCount": 4,
"maxNodeOperatorsPerExtraDataItemCount": 50
interface IOracleReportSanityChecker {
function checkAccountingExtraDataListItemCount(uint256 _extraDataListItemsCount) external view;
function checkNodeOperatorsPerExtraDataItemCount(uint256 _itemIndex, uint256 _nodeOperatorsCount) external view;
}
These parameters and related sanity checks will be changed as the multi-transaction approach no longer imposes limits on the entire report size. Rather than validating the entire report size, we will independently validate each transaction during the third phase.
"maxItemsPerExtraDataTransaction": 8
"maxNodeOperatorsPerExtraDataItem": 24
interface IOracleReportSanityChecker {
function checkExtraDataItemsCountPerTransaction(uint256 _extraDataListItemsCount) external view;
function checkNodeOperatorsPerExtraDataItemCount(uint256 _itemIndex, uint256 _nodeOperatorsCount) external view;
}
The gas consumption for updating a single node operator:
The processing of each item should not exceed the specified gas limit of 1_000_000 gas. Each transaction can contain up to 8 items, each consuming 1_000_000 gas (totaling 8_000_000 gas).
Using the higher value: CSM: 41_150 gas
This results in: 24 operators per item (1_000_000 / 41_150 = 24.3)
By setting these values, we ensure that each transaction stays within the 8_000_000 gas limit.