---
title: Lido Oracle Specification
tags: Oracle, withdrawals
status: Draft
author: Raman Siamionau
created: 2022-10-21
updated: 2023-01-14
---

# Lido Oracle Specification (Outdated)

Oracle daemon (further Oracle) for Lido stacking protocol collects, make some decisions and reports Consensus and Execution Layer data to Lido Oracle smart contract. Depends in which mode Oracle is running it reports validator's balances on Consensus Layer, skimmed rewards, withdrawal requests and validators to eject.
Validators ejection works separately from main oracle report. It could happen more often, depends on Oracle contract setup and withdrawals activity.

New Oracle will contain 3 modules:
- **Accounting** - Makes decisions about protocol day-to-day operations.
- How much balance on Consensus Layer.
- How much Lido validators running and exited.
- How much ETH do we have to fulfill withdrawal requests.
- How much ETH we should block on deposit contract for withdrawals.
- **Ejector** - Decides how many and which validators should exit to make sure we will be able to fulfill withdrawal requests as soon as possible. Can change reserve amount.
- **Pool balancer** - Balance pool ETH/stETH is the difference is more than 5%.

:::warning
1. We use quorums to avoid scamming. That's why all reports from Oracle holders should be same and should be built for same slot and block number.
2. We should avoid double accounting to avoid loosing TVL or community trust because of wrong numbers.
:::

## Tech zen

1. **Stateless. All oracle instance in different point of time should build same report for specific epoch.**
2. **Better Oracle be tweaked on chain rather than with variables. Fewer settings in variables, more on chain variables.**
3. **First readability, next performance.**

## Frames

Oracle will build the report using first non-missed slot in last **finalized** epoch, that multiple of “epoch per frame”.

if last_finalized_epoch % epoch_per_frame == 0:
    slot_number = get_first_non_missed_slot_number_in_epoch(last_finalized_epoch)
    
    build_report(slot_number)


## Two phase Oracle

Oracle smart contract could be in 3 states:
*(this also applies to exit bus smart contract)*

- Ready for report
Oracle is ready to receive report's hash from the participants of committee.
***handleCommitteeMemberReport(epochId, reportHash)*** method.

- Ready to finalize
Oracle got enough hashes and the report is ready to be finalized.
One of the committee should send report data (hash of which was used above).
***handleReportData(reportData)*** method.

:::warning
Behavior could be changed in this part. Sending report's data can take place in several stages.
:::

- Reported
All calculations are done and Oracle is inactive until new epoch in next frame will be finalized.

## Bunker mode

Oracle could work in two modes:
- Turbo mode - The default oracle mode. All Oracle's features are enabled.
- Bunker mode - Activates when Lido slashed a lot. In this mode, all withdrawals are disabled.

Details about bunker mode are [**here**](https://hackmd.io/@lido/Bk9xMAVso).

## Predictions

There are quite a few places in the Oracle where we try to predict the income to vault for a certain number of blocks.
To do this we need to calculate income for equal periods of time before the period of time for which we want to know the income and calculate the median of these values.

:::info
For example, we want to predict income to EL vault for 7200 blocks (1 day). We will check income for past 10 days day by day. And the median of that values will be our assumption income for next day.
:::

## Oracle Update

### Oracle v2 -> v3

Update path:
1. Startup Oracle Daemon v3 near Old Oracle v2.
2. As soon as protocol will be upgraded, new Oracle starts working automatically. Old oracle can't do any reports because report structure was changed.
3. Delete Old Oracle instances.

### Regular Oracle update

There are two different types of update.

- Major
- Protocol upgrade
- Oracle report calculation result was changed (Calculation strategies tweaks)

- Minor
- Performance increase
- Bug fixes not related to report result

Minor updates could be done totally asynchronously.
Major update won't work until >50% of Oracle will be updated.

There are two ways to do Major updates:
1. Such as v2-v3 migration done, but Oracle will check contract version and stops working as soon as version was changed.
2. All oracles update asynchronously. But if the number of all members is ODD and the quorum is more than 50% and only half is updated, there will be no consensus.

# Oracle modules

Each module is a statless, independent unit that could be used in a separate service.

## Entrypoint

There is one entrypoint in Oracle daemon that contain all modules.
For every new finalized epoch, it fetches the slot and block number and calls ***run_module*** method in each module.

## Accounting module

[**LidoOracle**](https://github.com/lidofinance/lido-dao/blob/feature/shapella-upgrade/contracts/0.8.9/LidoOracleNew.sol) smart contract source code.

### Report structure

| Field name | Type | Description |
| --------------------------------- | --------- | -------------------------------------------------------------------------------------------------------- |
| beaconValidators | uint256 | Count all Lido's validators on Consensus Layer |
| beaconBalanceGwei | uint64 | Sum all Lido's validators balances on Consensus Layer |
| stakingModules | address[] | Source of Node Operator |
| nodeOperatorsWithExitedValidators | uint256[] | Node Operator ID in contract |
| exitedValidatorsNumbers | uint256[] | Count of exited validators for each Node Operator |
| withdrawalVaultBalance | uint256 | WithdrawalCredentionals vault balance |
| withdrawalsReserveAmount | uint256 | Amount of deposit buffer to hold. Will be used to fulfill withdrawal request in next Oracle's iteration. |
| requestIdToFinalizeUpTo | uint256[] | Latest withdrawal request that would be fulfilled if all current ETH will be spent to finalize requests. |
| finalizationShareRates | uint256[] | Shares. Fetch from Lido smart contract |

### Fields details

#### withdrawalsReserveAmount

This is the only field that could be tweaked using different strategies.

**Reserve strategy V1**
Total reserve amout is the amount of ether that would be enough to satisfy all withdrawal requests on the next report.

Formula is:

to_reserve = wr_amount + wr_amount_predicted - eth_available - eth_predicted - withdrawable_validators

**wr_amount** - Sum all unfulfilled withdrawal requests.
**wr_amount_predicted** - All wr that could appear in buffer for next day.
**eth_available** - Sum EL and WC vaults.
**eth_predicted** - Sum all predicted income into EL and WC vaults.
**withdrawable_validators** - Sum of all Lido Validators with `balance != 0` and `withdrawable_epoch < current_report_epoch + epoch_in_frame`

#### Exited validators

Exited validators for each Lido Node Operator will be reported in the next way:

Each column is a detail about each Node Operator. Node Operators without any changes in exited validators should be skipped to save gas fee.

[ module1, module2, module2] - stakingModules
[ NO1,     NO1,     NO2    ] - nodeOperatorsWithExitedValidators
[ 10,      10,      20     ] - exitedValidatorsNumbers

:::info
Node operator #1 from module #1 has 10 exited validators.
Node operator #1 from module #2 has 10 exited validators.
Node operator #2 from module #2 has 20 exited validators.
:::

#### Withdrawal requests finalization

Schema is:

[ 5,        10    ] - requestIdToFinalizeUpTo
[ 100,      110   ] - finalizationShareRates

:::info
Finalize all requests from last finalized to request #5 with share rate 100. And requests from #6 to #10 with share rate 110.
:::

There are 3 vaults that would be used to fulfill withdrawal requests.
- All reserved buffered ether in Lido buffer.
- Withdrawal Credentials vault.
- Execution Layer vault.

We should finalize as many requests as possible with current amount of ETH.

:::warning
If bunker mode is on, withdrawal requests **should not** be finalized, so the report should contain empty lists.
:::

#### Solution (Do not do)

Calculate first non-reported report.
Check el and wc balance.

To build next report use data from prev-report.
Use delta of el and wc balance.
(Make sure soft limits works)

#### Solution new

Call view or static function to calculate shares after report and use this number. (handleOracleReport) Use result in finalizationShareRates. List contains only one element. (for testnet at least)

## Ejector module

[**WithdrawalBus**](https://github.com/lidofinance/lido-dao/blob/feature/shapella-upgrade/contracts/0.8.9/ValidatorExitBus.sol) smart contract source code.

The ejector module decides which and how many validators should exit so that by the time we withdraw their balances, we can finalize all the current requests on this moment.

### Report structure

| Field name | Type | Description |
| ---------------------------------- | --------- | --------------------------------- |
| stakingModules | address[] | Source of Node Operator |
| nodeOperatorIds | uint256[] | Node Operator ID in contract |
| validatorIds | uint256[] | Validator's ID on Consensus Layer |
| validatorPubkeys | bytes[] | Validator's pub key |

### Fields details

[ module1, module2] - stakingModules
[ NO1,     NO1    ] - nodeOperatorIds
[ 10,      25     ] - validatorIds
[ key1,    key2   ] - validatorPubkeys

:::info
Validator with pubkey key1 with id #10 from NO #1 from module #1 should exit.
Validator with pubkey key2 with id #24 from NO #1 from module #2 should exit.
:::

### How to choose amount of validators to eject

Oracle fulfills withdrawal requests from 3 sources:
- EL rewards vault
- WC vault
- Buffered ether

The goal of the ejection module is to eject the minimum amount of validators which will be enough to fulfill all current unfinalized withdrawable requests (when the balance of these validators will be withdrawn).

So the formula is:

withdrawal_requests_sum = validators_count * 32ETH + available_eth
available_eth = wc + el + min(buff_eth, reserved_eth)

Where available_eth is:

available_eth = current_eth + predicted_eth

predicted_eth - is amount of ETH that we receive until validators will be withdrawable.

Predicted ETH goes from different sources:
- Predicted EL rewards
- Predicted skimmed rewards
- Predicted withdrawable validators * min(validator_balance, 32) ETH
- Validators asked to exit in 24h * min(validator_balance, 32) ETH

The time of prediction is the amount of time that took the validator to exit. It depends on the exit queue and NO reaction (NO should send signed exit message). The prediction works the next [way](https://hackmd.io/q7lQrq49QJm3zY3IFhnmhw?both#Predictions).

:::warning
In bunker mode, the prediction time will be equal to the amount of time from now till bunker mode ends.
:::

So the final formula is:

$$x = \frac{el_c + el_p + wc_c + sk_p + v_e + v_a - v_i}{32ETH}$$

Where:

N - amount of blocks takes validator to pass from oracle exit event till validator withdrawal.

$x$ - amount of validator to exit
$el_c$ - current EL rewards.
$el_p$ - predicted for EL rewards for N block.
$wc_c$ - current WC balance.
$sk_p$ - predicted amount of skimmed rewards.
$v_e$ - Validators that will pass withdrawal epoch in N blocks.
$v_a$ - All validators we asked to exit. But didn't reacted yet. No info in CL layer
$v_i$ - Validators asked to exit more than 7 days ago, but still exit message wasn't published.

### How to calculate the amount of time for validator to voluntary exit

Details are [here](https://notes.ethereum.org/@hww/lifecycle#4311-Initiate-exit).

$exit\_epochs(state) := \{ v.exit\_epoch \forall v \in V | v.exit_epoch \neq FAR\_FUTURE\_EPOCH \}$
$exit\_queue\_epoch(state) := max\{exit\_epochs(state) \cup \{ current\_epoch + 1 + MAX\_SEED\_LOOKAHEAD \} \}$
$v.exit\_epoch := exit\_queue\_epoch(state)$
$v.withdrawable\_epoch := v.exit\_epoch + MIN\_VALIDATOR\_WITHDRAWABILITY\_DELAY$

Time to exit:
`time_to_exit = withdrawable_epoch - current_epoch + time_to_send_exit_msg_epoch`

Where:
$FAR\_FUTURE\_EPOCH = 2^{64}-1$ epoch.
$MAX\_SEED\_LOOKAHEAD = 4$ epochs.
$MIN\_VALIDATOR\_WITHDRAWABILITY\_DELAY = 2^8$ epochs.

### More details how to calculate values

**$el_p$** - values for median are calculated in this way - `current_balance - N_block_behind_balance + oracle_transfer`
where `oracle_transfer` is ETH that oracle transferred out in this interval.
1
(64.1)->(32) (+32 + 0.1) = 32.1
(32.1, 32) ->(32, 0) (+32+ 0.1) =
Beacon_old - beacon_new
**$sk_p$** - `current_balance - N_block_behind_balance + oracle_ransfer - withdrawn_validators`
where `withdrawn_validators` are all validators withdrawn in this interval.

**$v_i$** - Read all events in 7 days for ejection and sum balances of this Oracles.

### How to choose which validators should be ejected

Big article with discussion is [here](https://research.lido.fi/t/withdrawals-on-validator-exiting-order/3048).

As I see, there are 4 strategies:
1. **Size**
Eject validators from NO with largest amount of active validators.
2. **Randomize**
Exit randomly, but larger NO have a higher chance of being exited and so will be pushed towards the average.
3. **Time**
Exit validators from NO which sum of total lifetime will be the greater.
4. **Balanced**
Validators are exiting in cycles. For example, cycle is 1000 validators. In this cycle, each NO will eject such amount of validators how many percent of its validators in the pool. If lido has 16.5% of all Lido validators in this cycle, it will lose 165 validators.

For now, I've implemented the first strategy. This strategy is simplest to implement and understand.

# Links

- [Withdrawals spec](https://hackmd.io/PkQzohrLT9uvbp6HBlIvDg#Oracle-flow-accounting)
- [Withdrawal landscape](https://hackmd.io/XGrVcicVTm-_IRcIUXPbnQ?view)
- [Oracle off-chain](https://hackmd.io/R6SKBwd7Tke0d7yg8R_mwg?view#Off-chain-oracle)
- [Ejection discuss](https://research.lido.fi/t/withdrawals-on-validator-exiting-order/3048)
- [Oracle source code](https://github.com/lidofinance/lido-oracle#lido-oracle-daemon)
- [Beacon Chain spec](https://ethereum.github.io/beacon-APIs/)
- [Voluntary exit path](https://notes.ethereum.org/@hww/lifecycle#433-Step-3c-Voluntary-exit-path)