Optimistic Sync: Which is the right head for the job?

Assumed prior reading: https://hackmd.io/Ic7VpkY3SkKGgYLg2p9pMg

This document enumerates the consensus client components which use the head and seeks to establish that they are not comprimised by only having access to the optimstic and verified ancestor heads.

Conflicting heads

This document is largely concerned with the scenario when the optimistic head differs from the verified head (either variant of the verified head). We call this having a conflicting optimistic head.

It's clear that this conflict is present when the ExecutionPayload of the optmistic head is not-yet-verified. Notably, we don't need to track the verified heads to know this. We can expect such a scenario during initial sync or due to an outage between execution<>consensus client comms.

Components

When there is a conflicting optimistic head, there are three broad options:

• Use the optimistic head.
• Use the verified ancestor head.
• Use neither (e.g., return an error, return something else).

I have attempted to identify all the scenarios and assigned them an answer to the above question. That is represented in the following table:

The next section will justify the contents of the above table.

Justifications

Why never publish a block when there is a conflict?

Firstly, lets consider the scenario where a node's optimistic head differs from the verified head. As discussed in a previous section (noting an assumption about attester behaviour), the verified tree is always a subset of the optimistic tree. Therefore, the optimistic tree is always the heaviest and choosing to produce on the verified chain is always building a minority chain.

Secondly, since the CL client has all the beacon blocks (and therefore all the execution payloads) there is no fundamental data availability issue. It must be the case that the EL client is still syncing.

Thirdly, lets just make it clear that's it's impossible to build atop an optmistic head. An EL client cannot build a block atop an unknown parent. Therefore, our choice during a conflict is to either (a) produce atop the verified head or (b) don't produce at all.

Given these three points, we can simmer this down to a fundamental question: should a node produce blocks on a minority chain when it knows itself to be behind the rest of the network?

I think the answer is no. I'd argue that producing a minority block from an ancient ancestor does the network more harm than a skip-slot, since they're onerous to process (with current CL architecture) an unlikely to end up in the canonical chain. Notably, the beacon-APIs block endpoint already suggests a 503 error when the CL client is syncing.

That being said, I think we need to be very cautious here. Refusing to produce blocks is the makings of a liveness failure. We assume that there is no data availability problem since all the ExecutionPayloads are known to the CL client. But, we don't have a channel for the EL client to retrospectively request blocks from the CL client; we expect the CL client to be able to sync from it's own P2P network. What if the liveness failure is there? Separately, what if there is a bug which causes EL clients to panic during ExecutionPayload and never return a valid/invalid/syncing response?

What I've just mentioned are issues completely separate to fork-choice, and I'm not sure we want to add complexity to fork-choice for these "what if?"" cases. At some point fork-choice needs to just trust that the other processes in the system are reliable.

As you can see, I've come to a tentative, best-effort guess regarding what to do with block production when there is a conflicting optimistic head. I am keen to hear more thoughts.

Why never publish an attestation when there is a conflict?

Unlike block production, it is possible to attest to an optimistic head*. Attesting requires no information about the ExecutonPayload. That being said, I don't think it is safe to attest to a conflicting optimistic head; that could lead to finalizing invalid payloads. So, like block production, I assert that our choice during a conflict is to either (a) attest to the verified head or (b) don't attest at all.

CL clients already restrict attestation during sync, since attestations to ancient blocks are burdensome to the rest of the network (with the current CL implementations) and they rarely add value.

So, I see it best to simply not attest when there is a conflicting optimstic head.

Just like block production, there are some data-availabilty and crash scenarios to consider. But once again, I'm not certain we must complicate attestation production to support them.

Why never publish sync committee messages when there is a conflict?

Just like attestation production, we could reference an optimistic head in a sync committee message but I don't think it's safe. I believe we only want to communicate verified heads to light-clients since they shouldn't be expected to handle an invalid ExecutionPayload.

I also think there is very little value in producing sync committee messages that reference old blocks. Their block-inclusion scheme is very strict and such messages are unlikely to be included in a block (especially if we choose not to produce blocks atop verified heads during a conflict).

Why never communicate the head to API consumers when there is a conflict?

I'm specifically referring to the "head" value for the block_id field in the standard beacon-APIs. Also, the eth/v1/beacon/headers endpoint.

I can see two clear options for the API when there is a conflicting optimistic head:

1. Return the optimistic head with some value that indicates it is optimistic.
2. Return a 503 "node is syncing" error (or similar).
3. Return the verified ancestor head.

I am not a fan of (1) for two reasons. Firstly, the optimistic head isn't safe and I don't want to tempt users into thinking it is. Secondly, optimstic sync isn't part of the protocol; it's an early-stage client optimistation and I'm not comfortable enshrining it in the API yet.

When it comes to options (2) and (3), I think both of them are reasonable. I think the implementation can choose between them without defying the standard.

What about the P2P components?

I think we should only communicate the verified ancestor head to peers via the status message, and only send blocks to peers via RPC if we know they are verified.

Sending unverified blocks over RPC could amplify the efforts of an attacker and also cause an innocent optimistic node to suffer downscoring from peers who know better.