# Challenge of syncing two distributed systems

In at least three situations as part of the Farcaster network, we are reconciling state on two distributed systems—Farcaster and Ethereum. These situations are:
1. Farcaster SignerAdd and SignerRemove messages <--> ID Registry on-chain state
2. Farcaster UserDataAdd messages <--> Name Registry on-chain state
3. VerificationAdd messages <--> verification claims signed by Ethereum addresses

In all of these situations, we defer to Ethereum state (i.e. SignerAdd messages are only valid if they are signed by the custody address in the ID Registry contract), which means that any consistency guarantees we may be able to make about the Farcaster network are dependent on Ethereum state. For example, given a set of Farcaster messages, two Farcaster hubs will merge the messages and come to the same state if they have processed the same events from the ID and Name Registry on-chain contracts. If they have processed different events, they may converge, but it is not guaranteed.

### Example 1: ID Registry state

The ID Registry stores the current custody address for each fid. Users can add `SignerAdd` messages that authorize EdDSA key pairs to sign messages on behalf of the fid. However, `SignerAdd` messages for an fid are only valid if they are signed by the current custody address for the given fid.

Here is an example set of ID Registry events and messages:


The result of this set of operations is that the fid is owned by address 0xA and the first `SignerAdd` message is valid.


As the ID Registry events are merged, the first `SignerAdd` message starts as valid, becomes invalid when the custody address changes, then becomes valid again with the final event.

### Example 2: Name Registry state

Each fname is owned by a single Ethereum address, and the mapping from fnames to addresses is stored in the Farcaster Name Registry contract. A user can submit a `UserDataAdd` message to the Farcaster network to assign an fname to themselves. Though a `UserDataAdd` message is only valid if the fname in question is currently owned by the user (i.e. if the fname is owned by the same custody address that owns the user's fid).

Consider the set of Name Registry events and messages below:


Given all of these messages, fid A would own fname @alice. Here's the final state of the messages:


What's confusing is that the `UserDataAdd` message from fid A will switch from valid to invalid and back to valid based on the Name Registry events.

### Example 3: Verifications

A verification represents a two-way link between an Ethereum address and an fid.

1. First, the Ethereum address creates an `eth_signature` by signing an object that includes the fid and a block number. The block number is included to order signatures from this address.
2. Second, the fid creates and signs a `VerificationAdd` message that includes the Ethereum address and `eth_signature`. The message’s timestamp is used to order verification messages, like other Farcaster messages.

The immediate goal of verifications it to store a list of verified Ethereum addresses linked to any particular fid. The secondary goal is to maintain the same list but where an Ethereum address can only be associated with a single, specifically the fid the Ethereum address most recently authorized.

VerificationRemove messages are used to remove verifications from an fid. Conflicts between VerificationAdd and VerificationRemove messages are resolves by LWW mechanics (i.e. an Ethereum address can be added, removed, and added by three messages with increasing timestamps).

Consider this example of 5 verification messages from two fids:

Given all of these messages, the final state of the system should be:

* VerificationAdd with timestamp 4 and fid A
* VerificationRemove with timestamp 5 and fid B


What's weird about this result is that the fourth message is only valid once the fifth message has been merged. If the fourth message is submitted before the fifth, it will be rejected because the second message includes a claim from Ethereum address 0x123 with a later block number. But the fifth mesage will remove the claim from block number 2, so re-submitting the fourth message will succeed, overwriting the third message.

## Final questions

None of these situations break the guarantee of strong eventual convergence. However, the final state of the system is non-deterministic. In all three situations, a given message flips from valid to invalid and back to valid. We assume that hubs delete messages once they become invalid. So if all hubs have deleted the message in question by the time Ethereum's state changes to make it valid again, the message will have to be re-submitted by a client. However, if at least one hub in the network is behind or slow to delete the message, differential sync should propagate that message from that hub to the rest of the network once it is valid again.

Are there ways to avoid the awkwardness caused by syncing Ethereum and Farcaster?