Queued Attestations for Unknown Heads

TL;DR

The honest-validator spec says we should broadcast attestations as soon as we get the block. However, this means attestations will likely propagate before the block does. To resolve this, the spec says we should cache attestations where we don't know the head block.

However, we can't always signature verify attestations if we don't know the head and this makes it difficult to design a DoS resistant cache.

Detail

The Honest Validator spec delcares:

A validator should create and broadcast the attestation to the associated attestation subnet when either (a) the validator has received a valid block from the expected block proposer for the assigned slot or (b) one-third of the slot has transpired (SECONDS_PER_SLOT / 3 seconds after the start of slot) – whichever comes first.

This means that if a validator receives a block before 1/3rd of the slot, it should broadcast the attestation immediately.

Broadcasting attestations immediately causes an issue where validators start broadcasting their attestations before the block has had time to propagate. The p2p spec defines our behaviour when we get these attestations for an unknown block:

[IGNORE] The block being voted for (attestation.data.beacon_block_root) has been seen (via both gossip and non-gossip sources) (a client MAY queue attestations for processing once block is retrieved).

As we can see, the spec solves the issue of attestations arriving before blocks by saying "a client MAY queue attestations for processing once block is retrieved".

Therefore, a client needs to implement some form of cache to store unknown-block attestations (i.e., where attestation.data.beacon_block_root is not verified\imported). Of course, when we add caches we need to make sure they're DoS resistant. When it comes to this unknown-block attestation cache, we'll have a decent cache if we can satisfy the following properties:

1. Each attestation is cached for a limited period of time.
   • I'm not sure there's "one right answer" for this, but caching each attestation for an epoch seems like an upper limit. A much shorter time is probably more sensible: just enough time to allow the RPC lookup.
2. There is only one attestation per validator.
   • We have this mechanism since we already restrict gossip propagation to this end.
3. Things can only enter the cache if they are signed by a validator.
   • Seems simple, but this is where things get tricky.

Achieving (1, 2) is trivial, however (3) is more difficult. Initially, it seems that it's impossible to verify an attestation if you don't know the head block. However this is not really true; you can (and probably should) use the target to load the shuffling for the attestation and then verify the signature.

But it's frequently the case where the head and the target are the same (i.e. for the first slot of the epoch). In such a case, there is no fail-safe way to verify the signature of the attestation and we're unable to maintain property (3) of the cache.

So, assuming that we can't always signature-verify attestations with an unknown head, we have the following options for the cache:

• Just drop attestations if we cant verify the signature:
  • This means that a client which aims to be maximially profitable and safe should always just wait until 1/3rd of the slot to broadcast attestations (unless we actually start to see network congestion due to this).
• The cache should still only store one attestation per validator, but it doesn't necessarily need to signature verify them:
  • The problem here is that anyone can create rubbish attestations and "steal" the position in the cache fromb the valid, signed attestation.
• The cache should store multiple, potentially non-signature-verified attestations:
  • This means we either have an unbounded cache (risky) or we still have the same position-stealing problem as above.

Summary

Presently Lighthouse always waits until 1/3rd of the slot to broadcast attestations. This is due to legacy reasons (server-sent events didn't exist) and because there's been no driving factor to push the attestations out early.

We'd like to implement the caching of unknown-block attestations, however it's not clear how we can design a safe cache.

At this point, I think it's worth asking:

"How important is it that we broadcast attestations earlier than 1/3rd of the slot?"

It indeed seems nice to spread out the attestation load, but if it's not immediately useful and it's problematic, then we should reconsider it.