# Beacon light-client progressive server

# Spec

Need to produce two objects

## `LightClientBootstrap`

Parts:

- `header`: already stored always by root, available forever
- `current_sync_committee`: from block's slot, if finalized slot retrieve by period else retrieve by state root
- A: On finalization, check if the sync committee of `period_of(finalized_slot)` is persisted. If not, check if an available state in memory has the same period. If yes, persist its two sync committees. Else, regen states from disk to persist the missing periods.
- `current_sync_committee_branch`: retrieve by block root.
- A: store to DB for every block in hot DB. At finalization migrate to cold DB the canonical checkpoints
- B: store to DB for every block in cold DB. At finalization delete from cold DB the canonical checkpoints
- C: store in memory for every block. At finalization persist to cold DB the canonical checkpoints

To store the parts

**TODO**: Just compute it in `run_migration` with `finalized_state`

def after_import_block(state: BeaconState, block: SignedBeaconBlock):
    block_root = hash_tree_root(block.message)
    
    # TODO: Persist current_sync_committee_branch for checkpoint blocks on each branch,
    # expect prune function to drop abandonded branches and heads
    parent_block_slot = todo!()
    
    # first slot in epoch is always a checkpoint
    if first_slot_in_epoch(block.message.slot):
        db.current_sync_committee_branch.set(block_root, compute_merkle_proof(state, CURRENT_SYNC_COMMITTEE_INDEX))
    
    if (
        # block crossed an epoch boundary, and is not first slot of the epoch: parent is a checkpoint
        first_slot_in_epoch(block.message.slot) and epoch_of(parent_block_slot) < epoch_of(block.message.slot)
        # at least one full epoch of empty slots up to block: parent is a checkpoint of the previous epoch
        or epoch_of(parent_block_slot) < epoch_of(block.message.slot) - 1
    ):
        
        
    
    # Persist sync_committee on finalization, no need to prune
    state_period = compute_period_of(state.slot)
    finalized_period = compute_period_of(state.finalized_checkpoint.epoch)
    
    # TODO: Only finalized root is ok, so sync_committee can only be queried by period
    
    # Store only if not set yet
    if finalized_period >= state_period:
        db.sync_committee.set(state_period + 1, state.next_sync_committee)
    if finalized_period >= state_period - 1:
        db.sync_committee.set(state_period, state.current_sync_committee)

Queried by any block root of `header`. `current_sync_committee_branch` is specific to each `header`. To construct

def get_light_client_bootstrap(block_root: Root) -> LightClientBootstrap:
    # block is always available up to `MIN_EPOCHS_FOR_BLOCK_REQUESTS`, backfill available
    header=block_to_light_client_header(db.block.get(block_root))
    
    # current_sync_committee_branch is available for the range of blocks synced by this node.
    # backfill sync not available
    current_sync_committee_branch = db.current_sync_committee_branch.get(block_root)
    
    header_period = compute_period_of(header.slot)
    finalized_period = compute_period_of(finalized_slot)
    current_sync_committee = if header_period <= finalized_period:
        # TODO may be a race condition, use pivot or some signaling
        db.sync_committee.get(period)
    else:
        # TODO: how to bound this? should it go through a cache?
        retrieve_sync_committee_from_state(header.state_root)

    return LightClientBootstrap(
        header,
        current_sync_committee,
        current_sync_committee_branch,
    )

Pruning strategy

def after_prune_fc(canonical_nodes, non_canonical_nodes):
    for root in canonical_nodes:
        if is_checkpoint_block(node)
            db.current_sync_committee_branch.move_to_cold(root)
    for root in canonical_nodes + non_canonical_nodes:
        db.current_sync_committee_branch.delete_from_hot(root)

def p2p_reqresp_handle('GetLightClientBootstrap', root):
    yield get_light_client_bootstrap(root)

## `LightClientUpdate`

Must be immediately published + queried by sync committee period

_Note: `LightClientFinalityUpdate` and `LightClientOptimisticUpdate` can be derived from `LightClientUpdate`_

After successfully importing a block, run `after_import_block`. In summary:

- persist best update for period to db
- cache latest update in memory
- broadcast `create_light_client_finality_update(update)` to the gossip topic `light_client_finality_update`
- broadcast `create_light_client_optimistic_update(update)` to the gossip topic `light_client_optimistic_update`

**Questions**:

- How to handle best updates that are not cannonical, may they include the wrong sync committee? When comparing an old update, how to we know it's cannonical? Should non cannonical updates be pruned on re-org or latter?

def after_import_block(state: BeaconState, block: SignedBeaconBlock):
    # Persist attested header data
    # ====
    block_root = hash_tree_root(block.message)
    
    next_sync_committee_root = hash_tree_root(state.next_sync_committee)
    db.next_sync_committee_branch.set(block_root, {
        branch: compute_merkle_proof(state, NEXT_SYNC_COMMITTEE_INDEX),
        root: next_sync_committee_root
    })
    db.sync_committee.set(next_sync_committee_root, state.next_sync_committee)

    db.finality_branch.set(root, {
        branch: compute_merkle_proof(state, FINALIZED_ROOT_INDEX),
        root: state.finalized_checkpoint.root
    })
    
    # Create update with `block`'s sync aggregate 
    # ====
    finality_update = create_light_client_finality_update(block)
    
    if finality_update.finalized_slot > cache.latest_finality_update.finalized_slot:
        cache.latest_finality_update = finality_update
        p2p.broadcast('light_client_finality_update', finality_update)
    
    if finality_update.attested_slot > cache.latest_optimistic_update.attested_slot:
        optimistic_update = create_light_client_optimistic_update(finality_update)
        cache.latest_optimistic_update = optimistic_update
        p2p.broadcast('light_client_optimistic_update', optimistic_update)
    
    period = compute_sync_committee_period_at_slot(finality_update.attested_header.slot)
    if is_better(db.best_period_update.get(period), finality_update):
        db.best_period_update.set(period, create_light_client_update(finality_update))

def create_light_client_finality_update(block: SignedBeaconBlock) -> LightClientFinalityUpdate:
    attested_header_root = block.message.parent_root
    attested_header = block_to_light_client_header(db.block.get(attested_header_root))

    update = LightClientFinalityUpdate()
    update.attested_header = attested_header

    # Indicate finality whenever possible    
    finality_branch = db.finality_branch.get(attested_header_root)
    if finality_branch is not None:
        update.finalized_header = db.light_client_header.get(finality_branch.root)
        update.finality_branch = finality_branch.branch

    update.sync_aggregate = block.message.body.sync_aggregate
    update.signature_slot = block.message.slot

    return update

def create_light_client_update(finality_update: LightClientFinalityUpdate) -> LightClientUpdate:
    update = LightClientUpdate(...finality_update)

    # `next_sync_committee` is only useful if the message is signed by the current sync committee
    if update.signature_slot == compute_sync_committee_period_at_slot(update.attested_header.slot):
        next_sync_committee_branch = db.next_sync_committee_branch.get(attested_header_root)
        update.next_sync_committee = db.sync_committee.get(next_sync_committee_branch.root)
        update.next_sync_committee_branch = next_sync_committee_branch.branch
    
    return update

For ReqResp:

- `LightClientUpdatesByRange`: iterate db `best_updates_by_period` bucket for requested period range
- `GetLightClientFinalityUpdate`: returned cached latest update in memory
- `GetLightClientOptimisticUpdate`: returned cached latest update in memory

def p2p_reqresp_handle('LightClientUpdatesByRange', start_period, count):
    for period in range(start_period, start_period + count):
        yield db.best_period_update.get(period)

def p2p_reqresp_handle('GetLightClientFinalityUpdate'):
    yield cache.latest_finality_update

def p2p_reqresp_handle('GetLightClientOptimisticUpdate'):
    yield create_light_client_optimistic_update(cache.latest_finality_update)

## Pruning

def after_prune_fc(canonical_nodes, non_canonical_nodes):
    # Caches to produce `LightClientUpdate` only need non-finalized block roots (potential block parents)
    for root in canonical_nodes + non_canonical_nodes:
        db.next_sync_committee_branch.delete(root)
        db.finality_branch.delete(root)

    # Caches to produce `LightClientBootstrap` only need canonical checkpoint blocks
    for root in [node for node in canonical_nodes if not is_checkpoint_block(node)] + non_canonical_nodes:
        db.current_sync_committee_branch.delete(root)
        
    # TODO: Prune `db.sync_committee`

The long term persisted data cost is:
- 1 sync committee per period: 24576 bytes every 27.3 hours ~= 657KB / month
- 1 next_sync_committee_branch per epoch: 192 bytes every 6.4 minutes ~= 1.3MB / month
- 1 `LightClientUpdate` per period: ~+24576 bytes every 27.3 hours ~= 657KB / month

The temporary non-finalized data cost is:
- 1 next_sync_committee_branch per block
- 1 current_sync_committee_branch per block
- 1 finality_branch per block

# impl

At the end of the `import_block` routine (after commiting to FC and DB), persist LC data

https://github.com/sigp/lighthouse/blob/441fc1691b69f9edc4bbdc6665f3efab16265c9b/beacon_node/beacon_chain/src/beacon_chain.rs#L3126

## Lodestar ref

Most code here: https://github.com/ChainSafe/lodestar/blob/unstable/packages/beacon-node/src/chain/lightClient/index.ts

This data structure interacts with the outside world via:

  /**
   * Call after importing a block head, having the postState available in memory for proof generation.
   * - Persist state witness
   * - Use block's syncAggregate
   */
  onImportBlockHead(
    block: allForks.AllForksLightClient["BeaconBlock"],
    postState: CachedBeaconStateAltair,
    parentBlockSlot: Slot
  )

_https://github.com/ChainSafe/lodestar/blob/618895c92922658c77284f9a31273ade7f894e94/packages/beacon-node/src/chain/lightClient/index.ts#L227_

- Persist post state proofs for `block`
- Persists `block`'s syncAggregate linking it with the previously persisted data of its parent