changed 4 years ago  
Published Linked with GitHub

Attestation Inclusion Tracking

Introduction

This document describes a method for tracking the performance of Eth2 validators. It tracks the following parameters for each validator:

  • attestation_hits: The validator had an attestation included on-chain.
  • attestation_misses: There was no attestation included on-chain for an epoch.
  • head_attestation_hits: The validator had an attestation included on-chain with a correct beacon_block_root.
  • head_attestation_misses: The validator had an attestation included on-chain with an incorrect beacon_block_root.
  • target_attestation_hits: The validator had an attestation included on-chain with a correct target.root.
  • target_attestation_misses: The validator had an attestation included on-chain with an incorrect target.root.
  • delay_avg: The average of all the attestation delays where an attestation was included on-chain.

It collects CSV data using the lcli tool, then displays that data in Google Sheets.

There's an example spreadsheet where you can view this data for some random validators. You can also copy the spreadsheet, add your own validators and compare them against the global average:

Example Time Span

We provide the tools to run this method across any span of epochs, but we provide an example spreadsheet across the following range:

  • Start epoch: 27112
    • 2021-04-01 00:00:00 UTC
    • Start of Q2.
  • End epoch: 32300
    • 2021-04-24 UTC (not sure exact time)
    • A couple of epochs prior to the mainnet block production issues (started epoch 23202).

This example timespan starts immediately after a prior Q1 2021 validator rewards survey by Ben Edgington and stops just before there were issues with block production on mainnet. This timespan feels recent, unstudied and represents typical operating conditions of the chain.

Method for Generating Raw Data

Method: Rust

https://github.com/paulhauner/lighthouse/blob/etl/lcli/src/etl/validator_performance.rs

Method: English (with some Rust 😉)

Define the ValidatorPerformance struct:

struct ValidatorPerformance {
    /// The validator had an attestation included on-chain.
    pub attestation_hits: usize,
    /// Inverse of `attestation_hits`.
    pub attestation_misses: usize,
    /// The validator had an attestation included on-chain which matched the "head" vote.
    pub head_attestation_hits: usize,
    /// Inverse of `head_attestation_hits`.
    pub head_attestation_misses: usize,
    /// The validator had an attestation included on-chain which matched the "target" vote.
    pub target_attestation_hits: usize,
    /// Inverse of `target_attestation_hits`.
    pub target_attestation_misses: usize,
    /// The validator achieved a `head_attestation_hits` point when the first slot of the epoch was
    /// a skip-slot.
    ///
    /// This is generally *not useful*, it was used to try and debug late blocks on mainnet.
    pub first_slot_head_attester_when_first_slot_empty: usize,
    /// The validator achieved a `head_attestation_hits` point when the first slot of the epoch was
    /// *not* a skip-slot.
    ///
    /// This is generally *not useful*, it was used to try and debug late blocks on mainnet.
    pub first_slot_head_attester_when_first_slot_not_empty: usize,
    /// A map of `inclusion_distance -> count`, indicating how many times the validator achieved
    /// each inclusion distance.
    pub delays: HashMap<u64, u64>,

}

The, allocate a list of [ValidatorPerformance; num_validators], where all values in ValidatorPerformance are either usize::zero or HashMap::empty.

Then,

  1. Collect the all blocks between the start/end epoch.
  2. Load the pre-state of the first (lowest-slot) block.
  3. Start sequentially applying each block to the state.

Each time an epoch boundary is reached, use ValidatorStatus to detemine the participation from the previous epoch (i.e., the epoch for which we have a complete set of attestations) and increment any relevant ValidatorPerformance value for each validator.

Once we have finished applying the blocks to the state, produce a CSV which is a 1:1 representation of the [ValidatorPerformance; num_validators] list with one exception: we produce a delay_avg which is simply the average of all attestation delays.

Generating the CSV and Importing to Google Sheets

  1. Install Lighthouse.
  2. Run a Lighthouse node until it's synced.
  3. After the Lighthouse node syncs, terminate the process so we can read from its database.
  4. Clone this branch and go into the lcli directory.
  5. Run cargo run --release etl-validator-performance --output ./valperf.csv --start-epoch 27112 --end-epoch 32300
  6. Go to the "datasource" tab of the spreadsheet.
  7. File -> Import -> Upload -> upload valperf.csv
  8. Set "Import Location" to "Replace Current Sheet".
  9. Import Data.

Sheets will take a while to load it, your browser might complain.

Notes

  • I should modify this to use the standard HTTP API instead of the Lighthouse DB.
Select a repo