# Eudico Ordering Layer Interface

---
> This document is **deprecated**. Its contents are being adapted and moved to a unified design document: [Implementation of Eudico's Ordering Layer](/P59lk4hnSBKN5ki5OblSFg)
---

This document describes the interface between the Eudico Filecoin client and its ordering layer subsystem.
As the ordering layer is part of the Eudico client, this is an internal interface that Eudico uses to interact with its ordering component.
We describe
1. The API exposed by the ordering layer that is to be invoked by Eudico
- [`SubmitRequests(refs []RequestRef)`](#SubmitRequestsrequestRefs-RequestRef)
3. The API that Eudico makes available to be called by the ordering layer
- [`GetRequest(r RequestRef) Request`](#GetRequestr-RequestRef-Request)
- [`NewBlock(block Block)`](#NewBlockblock-Block)
- [`StateSnapshot(height t.BlockHeight) []byte`](#StateSnapshot-byte)
- [`RestoreState(snapshot []byte, height t.BlockHeight)`](#RestoreStatesnapshot-byte-height-tBlockHeight)

In code snippets, we use the Go language syntax.
Data types prefixed with `t.` are abstract types defined by the implementation.

> NOTE: The names of functions and data types are up for discussion.
> Technical details like error handling are omitted for now for simplicity of presentation.

## The current implementation in Eudico
The main logic is implemented in the [Mine](https://github.com/filecoin-project/eudico/blob/eudico/chain/consensus/mir/mine.go#L40) function, which implements the following algorithm:
1. Retrieve messages and cross-messages from mempool.
Those messages can be added into mempool via the libp2p mechanism or by a user via CLI.
2. Send these messages to the Mir node via `SubmitRequest` function.
3. Receive ordered messages from the Mir node's app and push them into the next Filecoin block.
4. Sync and submit this block over the libp2p network using `SyncSubmitBlock` function.

Then the block will be validated and applied, all messages from the block will be garbage collected from the mempool.

### How SyncSubmitBlock works
[SyncSubmitBlock](https://github.com/filecoin-project/eudico/blob/0306742e553f6bd6260332b501bb65a5bfc16a76/node/impl/full/sync.go#L53) is used "to submit a newly created block to the network through this node".
The function restores a [full block](https://github.com/filecoin-project/eudico/blob/0306742e553f6bd6260332b501bb65a5bfc16a76/node/impl/full/sync.go#L82) from the input block header and messages' CIDs. Then it calls `ValidateMsgMeta` that "performs structural and content hash validation of the messages within this block. If validation passes, it [stores](https://github.com/filecoin-project/eudico/blob/4ad77a597cc82a6775a69028405d3d14f6491f3f/chain/sync.go#L341) the messages in the underlying IPLD block store."

## Ordering layer API called by Eudico

The ordering layer exposes the following interface to the Eudico system.
It consists of a single function.

### `SubmitRequests(refs []RequestRef)`

Submits request references to the ordering layer for ordering, i.e.,
tells the ordering layer to order the referenced requests.
`SubmitRequests` receives a list of request references, where a request reference uniquely identifies a message stored by Eudico
(most likely inside Eudico's mempool).
Eudico submits its messages to the ordering layer as requests.

#### Request format
A submitted request and the reference to it have, respectively, the following format:

type Request interface {}

type RequestRef struct {
    ClientID t.ClientID
    ReqNo    t.ReqNo
    Type     t.ReqType
    Digest   []byte
}

The `Request` interface represents anything that can be ordered by the ordering layer.
Currently it can be either a Eudico message (signed or cross-net messages) or a reconfiguration messages for the ordering layer itself.

The `RequestRef` represents a reference to a request.
It has the following fields:

- The `ClientID` field uniquely identifies the application-level source of the request (Eudico message),
i.e., an actor or wallet within a subnet. We will henceforth refer to such entities as *clients*.
- The `ReqNo` (the *request number*) is the client-local sequence number of the request.
In other words, it is the number of requests previously produced by the same client.
Each client's first request has `ReqNo = 0`, the second request has `ReqNo = 1`, etc.
The request numbers must start at 0 and be contiguous.
The tuple `(ClientID, ReqNo)` thus uniquely identifies each request submitted to the ordering layer.
Two `Request` objects with the same `ClientID` and `ReqNo` are considered the same and all their fields must be equal.
If they are not, the client that produced them is assumed to be faulty.
- `Data` is the payload of the request.
- The `Digest` field of a `RequestRef` contains a hash of the referenced `Request`, computed over all its fields.
- The `Type` field defines the type of the referenced request.
This is expected to be an enumeration type and for now we consider only 2 possible values:
- `EudicoSignedMessage`: the request is a signed Eudico message
- `EudicoCrossMessage`: the request is a cross-net Eudico message
- `ConfigRequest`: the request is a configuration request for the ordering layer itself

#### Durability

Before invoking `SubmitRequests`, all requests referenced in the argument must be durably stored by Eudico in stable storage.
Even after a restart, the ordering layer expects to be able
to retrieve each of those requests `req` by calling `GetRequest(ref)`, where `ref` is the reference to `req`.

> Discussion: Initially, the proposal was to use a "retention index" returned by `SubmitRequests` for garbage-collection.
> However, the retention index is connected to the progress of the system state
> and the submitted transactions do not have any notion of the state yet.
> The retention index makes sense with respect to garbage-collecting old state,
> but not necessarily pending transactions.
>
> **Question:** How does Eudico know when it can delete transactions from its mempool?
> - Here you can see the relevant data structures that keep track of the messages in the mempool (these are in-memory, they're not perseisted): https://github.com/filecoin-project/eudico/blob/78563829ed55324866e70b920d19cda26d73ca2a/chain/messagepool/messagepool.go#L134-L144
> - This functions gives you a sense behind the logic for message removal. Mainly, messages are removed in the mempool if we see that they've been applied in a tipset, or because they were pruned: https://github.com/filecoin-project/eudico/blob/78563829ed55324866e70b920d19cda26d73ca2a/chain/messagepool/messagepool.go#L307
> - Note: No messages from the mempool are persisted. Here you can follow the live of a message from when it is received through the pubsub channel to it being added to the mempool: https://github.com/filecoin-project/eudico/blob/78563829ed55324866e70b920d19cda26d73ca2a/chain/sub/incoming.go#L358
>
> **Option 1:** The ordering layer explicitly tells Eudico which transactions are outdated
> using `ReqNo` (called transaction "nonces" in Eudico).
>
> **Option 2:** The part of the application state that encodes the set of applied transactions
> is accessible to Eudico and Eudico inspects the state and deletes outdated transactions
> from the mempool by itself.
>
> **Conclusion:** Ask Alfonso about how Eudico knows which messages have been applied.
> - Every new tipset it verifies if the message has been applied to update the latest nonce for the address and clean the mempool.

<!--The retention index `i` returned by `SubmitRequests` must be stored by Eudico and associated with the submitted requests.
Eudico can remove the requests after `Free(retIdx)` has been called by the ordering layer, with `retIdx > i`.
After this point, the ordering layer guarantees to not try to access these requests any more.
-->

> NOTE: If I understand correctly, the Eudico messages stored in the mempool are currently deleted (from the mempool)
> right after they are applied to the state.
> If we want to keep this behavior, we need to involve more than just the mempool for looking up requests.

#### Delivery guarantees

Eudico guarantees that, if `SubmitRequest(reqRefs)` is called at a correct node,
then references to all requests included in `reqRefs` will eventually be submitted (not necessarily in a single invocation)
to the ordering layer at all correct nodes.

#### Duplication

The ordering layer considers two requests `r1` and `r2` *repeated* iff `r1.ClientID == r2.ClientID` and `r1.ReqNo == r2.ReqNo`.
During normal operation (no restarts), in presence of repeated messages, Eudico should invoke `SubmitRequests` for exatly one of them.
The ordering layer, however, must be able to deal with requests being submitted multiple times and still deliver each request only once,
albeit its performance might be decreased.

#### Authentication

> TODO: Can we assume that requests in the mempool are authenticated? What about valid? What is the exact notion of a "valid" request?
> - We do. The mempool already makes some verifications (check signatues, available balance for the address sending the message, etc.). We could extend this verification if needed.

## Eudico's API called by the ordering layer

Eudico makes the following functions available for the ordering layer to call.

- `GetRequest(r RequestRef) Request`
- `NewBlock(b Block)`
- `StateSnapshot(height t.BlockHeight) []byte`
- `RestoreState(snapshot []byte, height t.BlockHeight)`

### `GetRequest(reqRef RequestRef) Request`

Returns the request that corresponds to the given reference.

### `NewBlock(block Block)`

Announces the delivery of a block to Eudico.
The block has the following format.

type Block struct {
    Requests []Request
    Height t.BlockHeight
    Metadata map[string][]byte
}

The `Requests` field is an ordered sequence of requests contained in the block.

The `Height` field indicates the block height, i.e. the number of blocks ordered before the new block being announced.

The `Metadata` field contains arbitrary metadata that the ordering layer might need to attach to the block.
It has the form of key-value pairs, with `string` keys and `[]byte` values, that is to be interpreted by Eudico
depending on the particular implementation of the ordering layer used.
If, for example, Eudico uses a protocol that signs blocks and relies on knowing those signatures,
the `Metadata` field might contain a signature stored under the `signature` key.
If "mining" is used to produce blocks and the miner needs to be rewarded, the miner's wallet address can be stored under a `miner` key.
These are just simple examples, however, and arbitrary metadata can be encoded under arbitrary keys.

Eudico guarantees that after `ApplyBlock` returns, the effect of applying the given block (i.e., the resulting state)
is persistently stored in stable storage and can be recovered even after a restart of the node.

### `StateSnapshot(height t.BlockHeight) []byte`

Returns a representation of Eudico's state.
The returned state snapshot must take all blocks with height lower than `height` into account.

> NOTE: This is intended to be a similar (or even the same) kind of snapshot that is being propagated to the parent network
> when the node is running in a subnet of hierarchical consensus.

### `RestoreState(snapshot []byte, height t.BlockHeight)`

Restores Eudico's state to the one encoded in `snapshot`.
It is the Eudico state that results from applying the first `height` blocks.