Ejector oracle

Main cycle

The oracle collects data on the current state, and in the loop finds one most suitable validator to exit, then mutates the state simulating this validator's exit and repeats the loop until the number of withdrawals is enough.

def get_validators_to_exit(report):
  state = collect_state(report)

  if is_exits_paused(state):
    return []

  if is_enough_eth_to_cover_demand(state, 0):
    return []

  validators_ids_to_exit = []
  while len(validators_ids_to_exit) < state["max_validators_to_exit"]:
    if not has_validators_to_exit(state):
      break

    predicted_eth = predict_available_eth_before_next_withdrawn(state, report)
    if is_enough_eth_to_cover_demand(state, predicted_eth):
      break

    validator_to_exit = get_next_validator_to_exit(state, report)
    validators_ids_to_exit.append(get_validator_ids(validator_to_exit))

    simulate_validator_exit(state, validator_to_exit)
    
  return validators_ids_to_exit

State collection

To find the next validators to exit, Ejector Oracle should first collect the following state from Consensus and Execution layers.

From Consensus Layer node:
- All validators and their states on the reference slot
From Staking Router:
- Public keys of all Lido validators
- Indices of the last requested validator to exit for each Node Operator
- Validator keys statistics for each Node Operator
From Withdrawal Queue contract:
- Uncovered withdrawal requests demand
From Oracle contract:
- Maximum number of exit requests for the current frame
- Recently requested via Exit Bus public keys to exit
From Lido contract:
- Recent withdrawals from Execution Layer Rewards and Withdrawal vaults

def collect_state(report):
  # Fetch data from Consensus Layer node
  all_validators = fetch_all_validators(report.ref_slot)

  # Fetch data from Keys API
  used_lido_pubkeys_by_modules = fetch_used_lido_keys_by_modules(report.ref_block)
  lido_modules_dict, lido_pubkeys_dict = build_lido_keys_indexes(used_lido_pubkeys_by_modules)

  # Fetch data from Staking Router contract
  operators_keys_stats = fetch_operators_keys_stats(lido_modules_dict)

  # Fetch data from Withdrawal Queue contract
  withdrawal_requests_uncovered_demand = fetch_withdrawal_requests_uncovered_demand(report.ref_block)

  # Fetch data from Oracle contract
  max_validators_to_exit = fetch_max_validators_to_exit(report.ref_block)
  recently_requested_pubkeys_to_exit = fetch_recently_requested_pubkeys_to_exit(report.ref_block)
  last_requested_validators_indices = fetch_last_requested_validators_indices(lido_modules_dict)

  # Fetch data from Lido contract
  predicted_pessimistic_rewards_per_epoch = fetch_predicted_pessimistic_rewards_per_epoch(report.ref_block)

  # Filter validators
  lido_validators = filter_lido_validators(all_validators, lido_pubkeys_dict)
  exitable_lido_validators = filter_exitable_lido_validators(report.ref_epoch, lido_validators, lido_pubkeys_dict, last_requested_validators_indices)
  inflight_lido_validators = filter_inflight_lido_validators(report.ref_epoch, lido_validators, lido_pubkeys_dict, last_requested_validators_indices)

  # Collect Staking Router state
  operators_delays = get_operators_delays_dict(lido_validators, last_requested_validators_indices, recently_requested_pubkeys_to_exit, lido_pubkeys_dict)
  operators_state = collect_operators_state(report.ref_epoch, lido_validators, last_requested_validators_indices, operators_delays, lido_pubkeys_dict, operators_keys_stats)

  # Collect CL state
  sweep_duration_epochs = predict_average_sweep_duration_epochs(all_validators)
  total_projected_validators = get_total_projected_validators(all_validators, lido_pubkeys_dict, last_requested_validators_indices)
  churn_limit = get_validator_churn_limit(all_validators, report.ref_epoch)
  exit_epochs = get_max_exit_epochs(all_validators)

  return {
    "all_validators": all_validators,
    "exitable_lido_validators": exitable_lido_validators,
    "inflight_lido_validators": inflight_lido_validators,

    "withdrawal_requests_uncovered_demand": withdrawal_requests_uncovered_demand,
    "max_validators_to_exit": max_validators_to_exit,
    "predicted_pessimistic_rewards_per_epoch": predicted_pessimistic_rewards_per_epoch,

    "lido_modules_dict": lido_modules_dict,
    "lido_pubkeys_dict": lido_pubkeys_dict,
    "operators_state": operators_state,
    "operators_keys_stats": operators_keys_stats,

    "sweep_duration_epochs": sweep_duration_epochs,
    "total_projected_validators": total_projected_validators,
    "churn_limit": churn_limit,
    "exit_epochs": exit_epochs,
  }

Fetching data

Validators from CL

Oracle fetches all validators from CL at the reference slot.

def fetch_all_validators(ref_slot):
  pass

Lido Keys

Fetches all used Lido keys from Keys API.

def fetch_used_lido_keys_by_modules(ref_block):
  (data, meta) = fetch_keys_from_api_provider()
  assert meta["block_number"] > ref_block, "The data is out of date"
  return data

Operators keys stats

Fetches operators' keys statistics for each Staking Module.

def fetch_operators_keys_stats(lido_modules_dict):
  """
  function getNodeOperatorReports(
    uint256 _stakingModuleId,
    uint256[] memory _nodeOperatorIds
  )
  """
  pass

Withdrawal requests uncovered demand

Collects data on the available ETH in the protocol buffer, in Withdrawal and Execution Layer Rewards vaults, the number of ETH required to cover withdrawal requests and returns the size of the uncovered balance.

def fetch_withdrawal_requests_uncovered_demand(ref_block):
  """
  Pessimistic assumption that requests are finalized 1 to 1,
  without taking into account a discount
  """
  withdrawal_requests_demand = fetch_withdrawal_requests_demand(ref_block)
  withdrawal_vault_balance = fetch_withdrawal_vault_balance(ref_block)
  el_rewards_vault_balance = fetch_el_rewards_vault_balance(ref_block)
  buffered_eth = fetch_buffered_ether(ref_block)

  reserved_buffered_eth = min(withdrawal_requests_demand, buffered_eth)
  available_eth_to_cover_demand = withdrawal_vault_balance + el_rewards_vault_balance + reserved_buffered_eth

  return max(withdrawal_requests_demand - available_eth_to_cover_demand, 0)

Report limits

Fetches report limits from the OracleReportSanityChecker.

def fetch_max_validators_to_exit(ref_block):
  """
  function getOracleReportLimits() public view returns (LimitsList memory)
  returns limits["maxValidatorExitRequestsPerReport]
  """
  pass

Recently requested pubkeys to exit

Fetches recently emitted ValidatorExitRequest events from ValidatorsExitBusOracle contract and extract pubkeys from them. The delayed timeout config fetches from the OracleDaemonConfig contract.

TODO: image for missed blocks

def fetch_recently_requested_pubkeys_to_exit(ref_block):
  timeout = fetch_delayed_validator_timeout(ref_block)

  # Rough estimate, since there could be missing blocks in the slots
  max_period_block_range = timeout / SECONDS_PER_SLOT

  from_block = ref_block - max_period_block_range
  to_block = ref_block
  events = fetch_exit_requests_events(from_block, to_block)
  
  # Narrow the events range
  events = filter_events_by_timestamp(events)

  return get_pubkeys_set_from_events(events)

Predicted rewards per epoch

Fetches ETHDistributed and TokenRebased events from Lido contract and calculate average rewards amount per epoch. The rewards prediction period config fetches from the OracleDaemonConfig contract.

TODO: take into account the time between reports since there may be missed reports

def fetch_predicted_pessimistic_rewards_per_epoch(ref_block):
  period = fetch_rewards_prediction_period(ref_block)

  # Rough estimate, since there could be missing blocks in the slots
  max_period_block_range = period / SECONDS_PER_SLOT

  from_block = ref_block - max_period_block_range
  to_block = ref_block
  events = fetch_eth_distributed_events(from_block, to_block)

  # Narrow the events range
  events = filter_events_by_timestamp(events)
  
  return calc_average_rewards_per_epoch_from_events(events)

Last requested validators indices

The Withdrawal Queue contract stores the index of the last validator that was requested to exit. Since validators are requested in strict order from the lowest validatorIndex to the highest, the indexes help find all the previously requested validators without fetching all events.
TODO: validator indexes may be reused in the future

def fetch_last_requested_validators_indices(lido_modules_dict):
  """
  function getLastRequestedValidatorIndices(uint256 moduleId, uint256[] calldata nodeOpIds)
    external view returns (int256[] memory)
  """
  pass

Withdrawal requests demand

def fetch_withdrawal_requests_demand(ref_block):
  """
  Fetches unfinalized steth from Withdrawal Queue
  """
  pass

Lido keys indexes

Build Lido keys indexes

Returns dicts to identify Lido validators by pubkeys and module_id and operator_id by validator pubkey.

def build_lido_keys_indexes(used_lido_pubkeys_by_modules):
  lido_modules_dict = get_lido_modules_dict(used_lido_pubkeys_by_modules)
  lido_pubkeys_dict = get_lido_pubkeys_dict(used_lido_pubkeys_by_modules)

  return (lido_modules_dict, lido_pubkeys_dict)

Lido keys dicts

def get_lido_modules_dict(keys_by_modules):
  modules = {}

  for module in keys_by_modules:
    module_id = module["module"]["id"]
    module_address = module["module"]["stakingModuleAddress"]

    modules[module_id] = {
      "address": module_address,
      "operators": set()
    }

    for key in module["keys"]:
      modules[module_id].add(key["operatorIndex"])

  return modules

def get_lido_pubkeys_dict(keys_by_modules):
  lido_pubkeys_dict = {}

  for sr_module in keys_by_modules:
    module_id = sr_module["module"]["id"]

    for key in sr_module["keys"]:
      pubkey = key["key"]
      operator_id = key["operatorIndex"]
      lido_pubkeys_dict[pubkey] = (module_id, operator_id)

  return lido_pubkeys_dict

Validators filters

Filter Lido validators

def filter_lido_validators(all_validators, lido_pubkeys_dict):
  return list(filter(lambda validator: (validator["validator"]["pubkey"] in lido_pubkeys_dict), all_validators))

Filter active Lido validators

def filter_active_lido_validators(epoch, lido_validators):
  return list(filter(lambda validator: (is_active_validator(validator), epoch), lido_validators))

Filter exitable Lido validators

TODO: exlude validators of inactive operators and stopped modules (do not exclude paused modules)

def filter_exitable_lido_validators(epoch, lido_validators, lido_pubkeys_dict, last_requested_validators_indices):
  def condition(validator):
    is_previously_requested = is_previously_requested_to_exit(validator, lido_pubkeys_dict, last_requested_validators_indices)

    # https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#voluntary-exits
    is_not_exited = validator["validator"]["exit_epoch"] == FAR_FUTURE_EPOCH
    is_active_long_enough = epoch >= validator.activation_epoch + SHARD_COMMITTEE_PERIOD
    is_exitable = is_not_exited and is_active_long_enough

    return not is_previously_requested and is_exitable

  return list(filter(condition, lido_validators))

Filter inflight Lido validators

Returns Lido validators that:
- Were recently requested to exit and the delayed deadline hasn't come yet
- Exited but not yet withdrawn

def filter_inflight_lido_validators(epoch, lido_validators, lido_pubkeys_dict, last_requested_validators_indices):
  def condition(validator):
    is_previously_requested = is_previously_requested_to_exit(validator, lido_pubkeys_dict, last_requested_validators_indices)

    is_exited = validator["validator"]["exit_epoch"] != FAR_FUTURE_EPOCH
    is_withdrawn = validator["validator"]["withdrawable_epoch"] <= epoch and validator["balance"] == 0
    is_exited_and_withdrawable = is_exited and not is_withdrawn

    return is_previously_requested or is_exited_and_withdrawable

  return list(filter(condition, lido_validators))

Filter not exited Lido validators

Returns requested but not exited validators. In other words, validators that match stuck, delayed, and in flight exit requests.

def filter_not_exited_lido_validators(lido_validators, lido_pubkeys_dict, last_requested_validators_indices):
  def condition(validator):
    is_previously_requested = is_previously_requested_to_exit(validator, lido_pubkeys_dict, last_requested_validators_indices)
    is_not_exited = validator["validator"]["exit_epoch"] == FAR_FUTURE_EPOCH
    return is_previously_requested and is_not_exited

  return list(filter(condition, lido_validators))

Filter delayed Lido validators

Returns delayed, including stuck, validators.

def filter_delayed_lido_validators(not_exited_lido_validators, recently_requested_pubkeys_to_exit):
  return list(filter(
    lambda validator: (validator["validator"]["pubkey"] not in recently_requested_pubkeys_to_exit), 
    not_exited_lido_validators
  ))

Misc

def get_operator_last_requested_index(validator, lido_pubkeys_dict, last_requested_validators_indices):
  """
  Gets last requested validator index for Node Operator or -1 if there were no exit requests
  """
  pubkey_operator = get_pubkey_operator(validator, lido_pubkeys_dict)
  return last_requested_validators_indices[pubkey_operator]

def get_pubkey_operator(lido_validator, lido_pubkeys_dict):
  validator_pubkey = lido_validator["validator"]["pubkey"]
  pubkey_operator = lido_pubkeys_dict[validator_pubkey]

  assert pubkey_operator is not None, "Not a Lido key"

  return pubkey_operator

def get_validator_ids(lido_validator, lido_pubkeys_dict):
  module_id, operator_id = get_pubkey_operator(lido_validator, lido_pubkeys_dict)
  validator_index = lido_validator["index"]

  return (module_id, operator_id, validator_index)

Operators state

Collect operators state

Collects the current state of operators: number of exitable keys, delayed validators, target keys to exit, age of validators.

def collect_operators_state(ref_epoch, lido_validators, last_requested_validators_indices, operators_delays, lido_pubkeys_dict, operators_keys_stats):
  operators_dict = {}

  for validator in lido_validators:
    pubkey_operator = get_pubkey_operator(validator, lido_pubkeys_dict)
    is_projected = is_projected_validator(validator, lido_pubkeys_dict, last_requested_validators_indices)

    if not operators_dict[pubkey_operator]:
      ever_deposited_keys = get_ever_deposited_keys(operators_keys_stats, pubkey_operator)
      delayed_validators = get_delayed_validators(operators_delays, operators_keys_stats, pubkey_operator)
      targeted_validators = get_target_validators(operators_keys_stats, pubkey_operator)

      operators_dict[pubkey_operator] = {
        # pre-fill with zero
        "projected_validators_total_age": 0,

        # pre-fill with ever deposited keys
        "projected_validators": ever_deposited_keys,
        
        "delayed_validators": delayed_validators,
        "targeted_validators": targeted_validators,
      }

    # subtract pre-fill if the validator exists
    operators_dict[pubkey_operator]["projected_validators"] -= 1

    if is_projected:
      validator_age = ref_epoch - validator["validator"]["activation_epoch"]
      operators_dict[pubkey_operator]["projected_validators_total_age"] += validator_age
      operators_dict[pubkey_operator]["projected_validators"] += 1

  return operators_dict

Delayed validator exits

Returns the number of validators that were delayed their exit.

def get_delayed_validators(operators_delays, operators_keys_stats, pubkey_operator):
  operator_stats = operators_keys_stats[pubkey_operator]
  delayed_validators = operators_delays[pubkey_operator] - operator_stats["refundedKeysKeysCount"]
  return max(0, delayed_validators)

Target validators

Returns the target number of validators for an operator.

def get_target_validators(operators_keys_stats, pubkey_operator):
  operator_stats = operators_keys_stats[pubkey_operator]
  is_limit_active = operator_stats["isTargetLimitActive"]
  target_validators = operator_stats["targetValidatorsCount"]

  if not is_limit_active:
    return None

  return target_validators

Ever deposited keys

def get_ever_deposited_keys(operators_keys_stats, pubkey_operator):
  operator_stats = operators_keys_stats[pubkey_operator]
  return operator_stats["validatorsReport"]["totalDeposited"]

Operator delays

Collects dict with delayed keys for each operator

def get_operators_delays_dict(lido_validators, last_requested_validators_indices, recently_requested_pubkeys_to_exit, lido_pubkeys_dict):
  not_exited_lido_validators = filter_not_exited_lido_validators(lido_validators, last_requested_validators_indices)
  delayed_lido_validators = filter_delayed_lido_validators(not_exited_lido_validators, recently_requested_pubkeys_to_exit)

  operators_delays = {}

  for validator in delayed_lido_validators:
    pubkey_operator = get_pubkey_operator(validator, lido_pubkeys_dict)

    if not operators_delays[pubkey_operator]:
      operators_delays[pubkey_operator] = 0

    operators_delays[pubkey_operator] += 1

  return operators_delays

Exit queue

Churn limit

Calculates the churn limit for the current state of the chain. The value is not recalculated when the validators exit since the accuracy is sufficient for our purposes. Churn limit calculation in the spec

def get_validator_churn_limit(all_validators, ref_epoch):
  total_active_validators = get_total_active_validators(all_validators, ref_epoch)
  return max(MIN_PER_EPOCH_CHURN_LIMIT, total_active_validators // CHURN_LIMIT_QUOTIENT)

Exit epochs

Returns an array of maximum exit epochs for the current state of the chain. This data is enough not to recalculate the entire validator set on every validator exit cycle

def get_max_exit_epochs(all_validators):
  exit_epochs = [v["validator"]["exit_epoch"] for v in all_validators if v["validator"]["exit_epoch"] != FAR_FUTURE_EPOCH]
  max_exit_epoch = max(exit_epochs)
  return [epoch for epoch in exit_epochs if epoch == max_exit_epoch]

Exit queue epoch

Returns the next exit epoch in the exit queue

def get_exit_queue_epoch(exit_epochs, churn_limit, ref_epoch):
  exit_queue_epoch = max(exit_epochs + [compute_activation_exit_epoch(ref_epoch)])
  exit_queue_churn = len([epoch for epoch in exit_epochs if epoch == exit_queue_epoch])

  if exit_queue_churn >= churn_limit:
    exit_queue_epoch += 1

  return exit_queue_epoch

Withdrawable queue epoch

Returns the next withdrawable epoch in the exit queue

def get_withdrawable_queue_epoch(exit_epochs, churn_limit, ref_epoch):
  return get_exit_queue_epoch(exit_epochs, churn_limit, ref_epoch) + MIN_VALIDATOR_WITHDRAWABILITY_DELAY

Activation exit epoch

Method from the spec. Returns the epoch during which validator activations and exits initiated in epoch take effect.

def compute_activation_exit_epoch(epoch):
  return epoch + 1 + MAX_SEED_LOOKAHEAD

Validators

def get_total_active_validators(all_validators, ref_epoch):
  return len(filter(lambda validator: is_active_validator(validator, ref_epoch), all_validators))

def get_total_projected_validators(all_validators, lido_pubkeys_dict, last_requested_validators_indices):
  return len(filter(lambda validator: is_projected_validator(validator, lido_pubkeys_dict, last_requested_validators_indices), all_validators))

Sweep

Average sweep prediction

Predicts the average epochs of the sweep cycle. In the spec: get expected withdrawals, process withdrawals

def predict_average_sweep_duration_epochs(all_validators, ref_epoch):
  total_withdrawable_validators = len(filter(lambda validator: (
    is_partially_withdrawable_validator(validator["validator"], validator["balance"]) or
    is_fully_withdrawable_validator(validator["validator"], validator["balance"], ref_epoch)
  ), all_validators))

  epochs_for_whole_sweep_cycle = total_withdrawable_validators / MAX_WITHDRAWALS_PER_PAYLOAD / SLOTS_PER_EPOCH
  return epochs_for_whole_sweep_cycle // 2

Withdrawable validators

def has_eth1_withdrawal_credential(validator):
  """
  Check if `validator` has an 0x01 prefixed "eth1" withdrawal credential
  https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#has_eth1_withdrawal_credential
  """
  return validator.withdrawal_credentials[:1] == ETH1_ADDRESS_WITHDRAWAL_PREFIX

def is_fully_withdrawable_validator(validator, balance, epoch):
  """
  Check if `validator` is fully withdrawable
  https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#is_fully_withdrawable_validator
  """
  return (
    has_eth1_withdrawal_credential(validator)
    and validator.withdrawable_epoch <= epoch
    and balance > 0
  )

def is_partially_withdrawable_validator(validator, balance):
  """
  Check if `validator` is partially withdrawable
  https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#is_partially_withdrawable_validator
  """
  has_max_effective_balance = validator.effective_balance == MAX_EFFECTIVE_BALANCE
  has_excess_balance = balance > MAX_EFFECTIVE_BALANCE
  return has_eth1_withdrawal_credential(validator) and has_max_effective_balance and has_excess_balance

Next validator to exit

The algorithm for the validators exiting is based on the algorithm described on the research forum.

The algorithm is supposed to correct the future number of validators for each Node Operator, so we are going to rely on the projected numbers in our calculations. Let's represent the validators and deposits in-flight of one of the Node Operator in the following form, where validators are sorted by their indexes: