changed 3 years ago  
Linked with GitHub

Eudico Ordering Layer Interface

This document is deprecated. Its contents are being adapted and moved to a unified design document: Implementation of Eudico's Ordering Layer

This document describes the interface between the Eudico Filecoin client and its ordering layer subsystem. As the ordering layer is part of the Eudico client, this is an internal interface that Eudico uses to interact with its ordering component. We describe

  1. The API exposed by the ordering layer that is to be invoked by Eudico
  2. The API that Eudico makes available to be called by the ordering layer

In code snippets, we use the Go language syntax. Data types prefixed with t. are abstract types defined by the implementation.

NOTE: The names of functions and data types are up for discussion. Technical details like error handling are omitted for now for simplicity of presentation.

The current implementation in Eudico

The main logic is implemented in the Mine function, which implements the following algorithm:

  1. Retrieve messages and cross-messages from mempool. Those messages can be added into mempool via the libp2p mechanism or by a user via CLI.
  2. Send these messages to the Mir node via SubmitRequest function.
  3. Receive ordered messages from the Mir node's app and push them into the next Filecoin block.
  4. Sync and submit this block over the libp2p network using SyncSubmitBlock function.

Then the block will be validated and applied, all messages from the block will be garbage collected from the mempool.

How SyncSubmitBlock works

SyncSubmitBlock is used "to submit a newly created block to the network through this node". The function restores a full block from the input block header and messages' CIDs. Then it calls ValidateMsgMeta that "performs structural and content hash validation of the messages within this block. If validation passes, it stores the messages in the underlying IPLD block store."

Ordering layer API called by Eudico

The ordering layer exposes the following interface to the Eudico system. It consists of a single function.

SubmitRequests(refs []RequestRef)

Submits request references to the ordering layer for ordering, i.e., tells the ordering layer to order the referenced requests. SubmitRequests receives a list of request references, where a request reference uniquely identifies a message stored by Eudico (most likely inside Eudico's mempool). Eudico submits its messages to the ordering layer as requests.

Request format

A submitted request and the reference to it have, respectively, the following format:

type Request interface {}

type RequestRef struct {
    ClientID t.ClientID
    ReqNo    t.ReqNo
    Type     t.ReqType
    Digest   []byte
}

The Request interface represents anything that can be ordered by the ordering layer. Currently it can be either a Eudico message (signed or cross-net messages) or a reconfiguration messages for the ordering layer itself.

The RequestRef represents a reference to a request. It has the following fields:

  • The ClientID field uniquely identifies the application-level source of the request (Eudico message), i.e., an actor or wallet within a subnet. We will henceforth refer to such entities as clients.
  • The ReqNo (the request number) is the client-local sequence number of the request. In other words, it is the number of requests previously produced by the same client. Each client's first request has ReqNo = 0, the second request has ReqNo = 1, etc. The request numbers must start at 0 and be contiguous. The tuple (ClientID, ReqNo) thus uniquely identifies each request submitted to the ordering layer. Two Request objects with the same ClientID and ReqNo are considered the same and all their fields must be equal. If they are not, the client that produced them is assumed to be faulty.
  • Data is the payload of the request.
  • The Digest field of a RequestRef contains a hash of the referenced Request, computed over all its fields.
  • The Type field defines the type of the referenced request. This is expected to be an enumeration type and for now we consider only 2 possible values:
    • EudicoSignedMessage: the request is a signed Eudico message
    • EudicoCrossMessage: the request is a cross-net Eudico message
    • ConfigRequest: the request is a configuration request for the ordering layer itself

Durability

Before invoking SubmitRequests, all requests referenced in the argument must be durably stored by Eudico in stable storage. Even after a restart, the ordering layer expects to be able to retrieve each of those requests req by calling GetRequest(ref), where ref is the reference to req.

Discussion: Initially, the proposal was to use a "retention index" returned by SubmitRequests for garbage-collection. However, the retention index is connected to the progress of the system state and the submitted transactions do not have any notion of the state yet. The retention index makes sense with respect to garbage-collecting old state, but not necessarily pending transactions.

Question: How does Eudico know when it can delete transactions from its mempool?

Option 1: The ordering layer explicitly tells Eudico which transactions are outdated using ReqNo (called transaction "nonces" in Eudico).

Option 2: The part of the application state that encodes the set of applied transactions is accessible to Eudico and Eudico inspects the state and deletes outdated transactions from the mempool by itself.

Conclusion: Ask Alfonso about how Eudico knows which messages have been applied.

  • Every new tipset it verifies if the message has been applied to update the latest nonce for the address and clean the mempool.

NOTE: If I understand correctly, the Eudico messages stored in the mempool are currently deleted (from the mempool) right after they are applied to the state. If we want to keep this behavior, we need to involve more than just the mempool for looking up requests.

Delivery guarantees

Eudico guarantees that, if SubmitRequest(reqRefs) is called at a correct node, then references to all requests included in reqRefs will eventually be submitted (not necessarily in a single invocation) to the ordering layer at all correct nodes.

Duplication

The ordering layer considers two requests r1 and r2 repeated iff r1.ClientID == r2.ClientID and r1.ReqNo == r2.ReqNo. During normal operation (no restarts), in presence of repeated messages, Eudico should invoke SubmitRequests for exatly one of them. The ordering layer, however, must be able to deal with requests being submitted multiple times and still deliver each request only once, albeit its performance might be decreased.

Authentication

TODO: Can we assume that requests in the mempool are authenticated? What about valid? What is the exact notion of a "valid" request?

  • We do. The mempool already makes some verifications (check signatues, available balance for the address sending the message, etc.). We could extend this verification if needed.

Eudico's API called by the ordering layer

Eudico makes the following functions available for the ordering layer to call.

  • GetRequest(r RequestRef) Request
  • NewBlock(b Block)
  • StateSnapshot(height t.BlockHeight) []byte
  • RestoreState(snapshot []byte, height t.BlockHeight)

GetRequest(reqRef RequestRef) Request

Returns the request that corresponds to the given reference.

NewBlock(block Block)

Announces the delivery of a block to Eudico. The block has the following format.

type Block struct {
    Requests []Request
    Height t.BlockHeight
    Metadata map[string][]byte
}

The Requests field is an ordered sequence of requests contained in the block.

The Height field indicates the block height, i.e. the number of blocks ordered before the new block being announced.

The Metadata field contains arbitrary metadata that the ordering layer might need to attach to the block. It has the form of key-value pairs, with string keys and []byte values, that is to be interpreted by Eudico depending on the particular implementation of the ordering layer used. If, for example, Eudico uses a protocol that signs blocks and relies on knowing those signatures, the Metadata field might contain a signature stored under the signature key. If "mining" is used to produce blocks and the miner needs to be rewarded, the miner's wallet address can be stored under a miner key. These are just simple examples, however, and arbitrary metadata can be encoded under arbitrary keys.

Eudico guarantees that after ApplyBlock returns, the effect of applying the given block (i.e., the resulting state) is persistently stored in stable storage and can be recovered even after a restart of the node.

StateSnapshot(height t.BlockHeight) []byte

Returns a representation of Eudico's state. The returned state snapshot must take all blocks with height lower than height into account.

NOTE: This is intended to be a similar (or even the same) kind of snapshot that is being propagated to the parent network when the node is running in a subnet of hierarchical consensus.

RestoreState(snapshot []byte, height t.BlockHeight)

Restores Eudico's state to the one encoded in snapshot. It is the Eudico state that results from applying the first height blocks.

Select a repo