# PubkeyCache Analysis
## Introduction
This is an analysis on the consensus, p2p and beacon api spec on the usage of `pubkeyCache` - a collection of `pubkey2Index` and `index2Pubkey`, and whether they require the knowledge of validators whose initial deposit is not yet finalized (See the rationale of unfinalized cache in [background](#background)). These are the places that require validator's index or pubkey to perform signature verification where `bls.Verify()` or `bls.FastAggregateVerify()` is used.
## Background
The use of `pubkeyCache` (`pubkey2Index` and `index2Pubkey`) cache is critical for CL clients to allow efficient lookup of public keys and indices of validators in many places particularly during state transition. The populating of said cache happen during deposit processing which assign indices to new validators.
In the current deposit flow, the indices assigned to new validators are uniform across all branches in the block tree due to the large follow distance of `Eth1Data` poll. Therefore, a single global fork-independent `pubkeyCache` is sufficient.
[`EIP-6110`](https://eips.ethereum.org/EIPS/eip-6110) proposes a refrom to the deposit flow that 1) Shifts the deposit inclusion and validation to EL 2) Deprecates the `Eth1Data` voting mechanism, CL no longer has a large follow distance and hence a validator index becomes fork dependent ie. a validator with the same `pubkey` can have different indices in different block tree branches.
For the case of active validator, the single global `pubkeyCache` approach is sufficient since the beacon state that first assigns index (by adding it to `state.validators`) to it is already finalized. For the case of a validator that is assigned an index from a block that hasn't been finalized, its index might be different in different fork. Hence we need to have another `pubkeyCache` that is fork-dependent to specifically stores the validators whose initial deposit is unfinalized. This two-cache approach is proposed in Lodestar, different client may have different approach, but a fork-dependent cache is necessary to embrace the change from EIP-6110. Regardless, there is a need to go through the specifications systematically, point out the usage of `pubkeyCache` and whether it needs to look up for only finalized, or both finalized and unfinalized validators.
## tl;dr
tl;dr: `index2Pubkey` is used in a lot more scenarios than `pubkey2Index`. The use cases for `index2Pubkey` do not require unfinalized information. `process_deposit()` is the only place in consensus spec that needs unfinalized information and it utilizes`pubkey2Index`.
In terms of the two-cache approach, `unfinalizedIndex2Pubkey` is not needed since there is not a place that utilizes it. `unfinalizedPubkey2Index`, however, is needed for `process_deposit()`.
| | unfinalized pubkey2Index? | unfinalized index2Pubkey? |
| -------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------------- |
| onBlock - state_transition - verify_block_signature | N/A | No |
| onBlock - state_transition - process_block - process_randao | N/A | No |
| onBlock - state_transition - process_block - process_operations - process_proposer_slashing | N/A | No |
| onBlock - state_transition - process_block - process_operations - process_attester_slashing - is_valid_indexed_attestation | N/A | No |
| onBlock - state_transition - process_block - process_operations - process_attestation - is_valid_indexed_attestation | N/A | No |
| onBlock - state_transition - process_block - process_operations - process_deposit - apply_deposit | Yes | N/A |
| onBlock - state_transition - process_block - process_operations - process_sync_aggregate - eth_fast_aggregate_verify | No | No |
| onBlock - state_transition - process_block - process_bls_to_execution_change | N/A | N/A |
| onBlock - state_transition - process_block - process_voluntary_exit | N/A | No |
| p2p - beacon_block ---- onBlock | N/A | No |
| p2p - beacon_aggregate_and_proof ---- onAttestation | N/A | No |
| p2p - voluntary_exit - process_voluntary_exit | N/A | No |
| p2p - proposer_slashing - process_proposer_slashing | N/A | No |
| p2p - attester_slashing - process_attester_slashing | N/A | No |
| p2p - beacon_attestation_{subnet_id} ---- onAttestation | N/A | No |
| p2p - sync_committee_contribution_and_proof | N/A | No |
| p2p - sync_committee_{subnet_id} | N/A | No |
| p2p - bls_to_execution_change - process_bls_to_execution_change | N/A | N/A |
| p2p - blob_sidecar_{subnet_id} - verify_blob_sidecar_signature | N/A | No |
## Phase 0
### [verify_block_signature](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md?plain=1#L1267)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
During the `state_transition()`, we need to verify the block signature by checking on `SignedBeaconBlock.message` against proposer's public key.
The proposer's public key is obtained via `index2Pubkey` with `SignedBeaconBlock.message.proposer_index`.
From the spec of `verify_block_signature()`, it does not enforce the proposer to be active or not, it is possible to a perform `index2Pubkey` lookup with a `proposer_index` belongs to a validator whose deposit is unfinalized.
However, with the current spec, `verify_block_signature()` is only called from `state_transition()`, which will call `process_block_header()` after `verify_block_signature()`. [`process_block_header()`](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#block-header) enforces the `proposer_index` must match the result from [`get_beacon_proposer_index()`](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#get_beacon_proposer_index) which requires it to be in the current shuffling and henceforth, belongs to an active validator. We do not need to have an unfinalized `index2Pubkey` in this scenario.
### [process_randao](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md?plain=1#L1733)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
`process_randao()` verifies the signing root of the current epoch against the pubkey of the current slot proposer which is calculated by `get_beacon_proposer_index()`.
`get_beacon_proposer_index()` picks the proposer from the list of active validators returned from `get_active_validator_indices()` according to a seed.
Hence, the lookup of `index2Pubkey` for proposer's pubkey must be finalized, because proposer index must be finalized.
### [process_proposer_slashing](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md?plain=1#L1774)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
`process_proposer_slashing()` checks the validity of `signed_header_1` and `signed_header_2` of `ProposerSlashing` by verifying them against the public key of the offending proposer.
The proposer's public key is obtained by looking up `index2Pubkey` with proposer index supplied in `ProposerSlashing`.
A `is_slashable_validator()` check happens before `bls.Verify()` which enforces the offending validator must be active. Therefore unfinalized pubkeyCache is not needed.
### [process_attester_slashing - is_valid_indexed_attestation](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#attester-slashings)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
`is_valid_indexed_attestation()` is called twice in `process_attester_slashing()`. One for each potential slashable attestations to verify the signature against the `attesting_indices`.
Although `is_valid_indexed_attestation()`, the caller of `index2Pubkey`, does not check whether the validator indices in offending attestations are active, the subsequent call in `process_attester_slashing()` checks the offending validator if it is slashable `is_slashable_validator()` and it requires the validator to be active.
### [process_attestation - is_valid_indexed_attestation](https://github.com/ethereum/consensus-specs/blob/eef61448a9aba2b61e77364bb920e028dd5963c1/specs/deneb/beacon-chain.md#modified-process_attestation)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
In `process_attestation`, it first converts `Attestation` into `IndexedAttestation` by calling `get_indexed_attestation()`. There are nested functions that run several levels deep in `get_indexed_attestation()`, but ultimately it returns an `indexed_attesation` for which its `attesting_indices` are from active validators.
The said `indexed_attesation` is then fed into `is_valid_indexed_attestation` that extracts public keys from every validators in `attesting_indices` by calling `index2Pubkey` and finally verify the signature of the attestation with `bls.FastAggregateVerify`.
Since the indices in `indexed_attestation` must belong to active validators, unfinalized cache is not needed.
### [apply_deposit](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#deposits)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `pubkey2Index` |
| Require unfinalized pubkeyCache? | `Yes` |
Given a `BLSPubkey`, `apply_deposit()` initializes a validator or top-up existed validator depending on whether the supplied public key exists in the `BeaconState.validators`.
The `pubkey2Index` lookup here happens during the top-up operation in order to find the index for `increase_balance(state, index, amount)`.
Since `apply_deposit()` accepts public keys from active/inactive validators whose deposit may or may not be finalized, this function needs `pubkey2Index` to contain unfinalized pubkeys.
## Altair
### [process_sync_aggregate - eth_fast_aggregate_verify](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/beacon-chain.md#sync-aggregate-processing)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | `pubkey2Index` and `index2Pubkey` |
| Require unfinalized pubkeyCache? | `No` |
In `process_sync_aggregate()`, it fetches a list of public keys of sync committee participants ie. validators that are in `state.current_sync_committee.pubkeys` and its corresponding `sync_aggregate.sync_committee_bits` is active. The public keys are then used to verify the signature of `sync_aggregate`.
Although `process_sync_aggregate()` does not lookup any sort of pubkeyCache, it does rely on the pulic keys of `current_sync_committee` in the Beacon state, which is populated by [`process_sync_committee_updates()`](https://github.com/ethereum/consensus-specs/blob/eef61448a9aba2b61e77364bb920e028dd5963c1/specs/altair/beacon-chain.md#sync-committee-updates) and [`get_next_sync_committee()`](https://github.com/ethereum/consensus-specs/blob/eef61448a9aba2b61e77364bb920e028dd5963c1/specs/altair/beacon-chain.md#get_next_sync_committee). They require sync committee to be active validators and use `index2Pubkey` for looking up the public keys.
In Lodestar, `pubkey2Index` is also being used to construct `syncCommitteeCache` which stores both public keys and indices in `EpochCache`.
## Capella
### [process_bls_to_execution_change](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md?plain=1#L466)
| | |
| --------------------------------- | ---- |
| `pubkey2Index` or `index2Pubkey`? | N/A |
| Require unfinalized pubkeyCache? | N/A |
Both validator index and public key during the withdrawal credential change is supplied from `BeaconBlockBody.bls_to_execution_changes`, hence no look up to `PubkeyCache` is necessary.
### [process_voluntary_exit](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md#modified-process_voluntary_exit)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
`process_voluntary_exit()` first looks up `index2Pubkey` with `SignedVoluntaryExit.validator_index` for public key of the validator that is exiting.
Before verifying the signature, `is_active_validator()` check is processed which requires the exiting validator must be active.
`process_voluntary_exit` does not need to look up unfinalized pubkeys.
## P2P
The following section covers the signature verifications happened upon receiving objects through each gossipsub topic
### [beacon_block](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/p2p-interface.md#beacon_block)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
The proposer's signature ie. `signed_beacon_block.signature` is checked against the proposer's pubkey, which is obtained via `index2Pubkey` with `proposer_index`. `proposer_index` needs to be in the current shuffling - a subset of currently active validators.
This is only for verifying gossip blocks. After which `onBlock()` is called for further processing and verification on beacon block.
### [beacon_aggregate_and_proof](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/p2p-interface.md#beacon_aggregate_and_proof)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
There are 3 signatures need to be checked here upon receiving `signed_aggregate_and_proof` from the network before proceeding to `on_attestation()`:
* Selection proof `signed_aggregate_and_proof.message.selection_proof`
* Signature of SignedAggregateAndProof `signed_aggregate_and_proof.signature`
* Signature of aggregate attestation `signed_aggregate_and_proof.message.aggregate.signature`
For both selection proof, and `signed_aggregate_and_proof.signature`, it compares them against aggregator's public key which is looked up via `index2Pubkey` with `aggregator_index`.
For the signature of aggregate attestation, it first converts the `aggregation_bits` into indices of attesting validators (no cache is used here), then look up corresponding public key (`index2Pubkey`) for each index. Then compare the keys against signature of aggregate attestation (``signed_aggregate_and_proof.message.aggregate.signature``).
Attesters' indices are derived from `aggregations_bits` intersecting with `get_beacon_committee()` so they must be active. Aggregator's index needs to be in `get_beacon_committee()` as well hence active.
### [voluntary_exit](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#voluntary_exit)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
A signature verification is performed that is identical to [`process_voluntary_exit()`](#process_voluntary_exit) in `process_block()`.
### [proposer_slashing](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#proposer_slashing)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
A signature verification is performed that is identical to [`process_proposer_slashing()`](#process_proposer_slashing) in `process_block()`.
### [attester_slashing](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#attester_slashing)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
A signature verification is performed that is identical to [`process_attester_slashing()`](#process_attester_slashing) in `process_block()`.
### [beacon_attestation_{subnet_id}](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/p2p-interface.md#beacon_attestation_subnet_id)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
Similar to verifying the signature of aggregate attestation in [`beacon_aggregate_and_proof`](#beacon_aggregate_and_proof), we first need to convert the `aggregation_bits` into validator indices. Difference is the bit list here should only contain a single bit and thus only one validator index. We then obtain the public key of said validator through `index2Pubkey` to verify the signature.
The attesting indices are derived from `aggregation_bits` and `get_beacon_committee()` hence they must be active.
### [sync_committee_contribution_and_proof](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#sync_committee_contribution_and_proof)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
There are 3 signatures need to be checked here upon receiving signed_aggregate_and_proof from the network before proceeding to on_attestation():
* Selection proof `signed_aggregate_and_proof.message.selection_proof`
* Signature of SignedAggregateAndProof `signed_aggregate_and_proof.signature`
* Aggregate signature `signed_aggregate_and_proof.message.aggregate.signature`
For both selection proof, and `signed_aggregate_and_proof.signature`, it compares them against aggregator's public key which is looked up via `index2Pubkey` with `aggregator_index`.
For the signature of aggregate signature, it first converts the `aggregation_bits` into indices of sync committee participants (syncCommitteeCache is used), then look up corresponding public key (`index2Pubkey`) for each index. Then compare the keys against signature of aggregate attestation (`signed_aggregate_and_proof.message.aggregate.signature`)
Attesters and aggregator must be part of the sync committee (`get_sync_subcommittee_pubkeys()`) which is pre-computed and embedded in the beacon state. The pre-computation happens in `process_sync_committee_updates()` therefore sync committee must be a subset of active validators.
### [sync_committee_{subnet_id}](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/p2p-interface.md#sync_committee_subnet_id)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
For a single `sync_committee_message`, we need to verify its signature against both the validator pubkey and `beacon_block_root`. `index2Pubkey` is used to look up the public key using `validator_index`.
Note that `subnet_id` is also being verified against `compute_subnets_for_sync_committee()` and it would fail if the validator's index is not in `state.current_sync_committee`. Therefore the message will be rejected will the validator is not active.
### [bls_to_execution_change](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/p2p-interface.md#bls_to_execution_change)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | N/A |
| Require unfinalized pubkeyCache? | N/A |
A signature verification is performed that is identical to [`process_bls_to_execution_change()`](#process_bls_to_execution_change) in `process_block()`.
### [blob_sidecar_{subnet_id}](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/p2p-interface.md#blob_sidecar_subnet_id)
| | |
| --------------------------------- | --- |
| `pubkey2Index` or `index2Pubkey`? | `index2Pubkey` |
| Require unfinalized pubkeyCache? | No |
`blob_sidecar_{subnet_id}` checks on `signed_blob_sidecar.signature` by calling [`verify_blob_sidecar_signature`](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/p2p-interface.md#verify_blob_sidecar_signature) which requires public key of the proposer. `index2Pubkey` is looked up with `signed_blob_sidecar.message.proposer_index`.
Proposer must be in current shuffling which by definition is an active validator, hence unfinalized pubkeyCache is not needed.
## Beacon API
The following is a list of pubkeyCache use in beacon and builder api.
| | unfinalized pubkey2Index? | unfinalized index2Pubkey? |
| ---------------------------------------------------------- | ------------------------- | ------------------------- |
| /eth/v1/beacon/states/{state_id}/validators/{validator_id} | Yes | N/A |
| /eth/v1/beacon/states/{state_id}/validators | Yes | N/A |
| /eth/v1/beacon/states/{state_id}/validator_balances | Yes | N/A |
| /eth/v1/builder/validators | Yes | N/A |
| /eth/v1/validator/register_validator | Yes | N/A |
The apis above use `pubkey2Index` is to convert public key fed by users to validator index. The said validator index will then be used to look up the validator object to either return related info or process the validator registration.
## Summary
We have listed out all the places in the current specifications where the `pubkeyCache` is used. By closely inspecting them one by one, we can conclude
1. `pubkey2Index` is mainly used in `process_deposit()` and beacon apis. It is necessary for it to contain validators whose deposits are not finalized and thus, it is necessary to have a fork-dependent unfinalized cache.
2. `index2Pubkey` is used extensively in variety of places. However, only active indices can ever make use of the cache which are guaranteed to come from finalized deposits. Thus an unfinalized `index2Pubkey` is not necessary.
`pubkey2Index` and `index2Pubkey` for active validators are needed and can continue to exist in a global singleton form. `pubkey2Index` for validators whose deposits are not yet finalized is also needed but should be fork-dependent and is recommended to have an instance attached to the state. `unfinalizedIndex2Pubkey` does not need to exist.
Sign in with Wallet
Connect another wallet