# Baking nonces rework design

## Structure

Nonces are stored in a persistent file so that bakers are still able
to reveal their nonces when the time comes even if they shutdown their
bakers.

In order to simplify and optimize nonce revelations, we need to store
extra informations:
- The nonce (pre-image of the hash) itself (necessary)
- The nonce hash that was published in the block (optional:
metadata/UX)
- The level of the block where the nonce was produced (necessary) - optimises query to find block
- The block hash where the nonce was produced - optimises query to check if block is in cannonical chain
- The cycle to which the nonce was produced (optional: it could be
derived from the level) - optimises query to check if nonce is relevant
- A _state_

## State

In order to easily determine what needs to be done with a nonce, we
define possible states.

A nonce has five possible states:
- ~~Published <publish_level>~~ Committed: the nonce was produced and its hash published in a block at level `publish_level`
- Injected <injection_level, op_hash>: the nonce revelation operation was injected in the network (`injection_level` is the head level)
~~- Noticed <notice_level>: the nonce revelation operation was found in a block at level `notice_level`~~
~~- Finalized: the nonce revelation was confirmed as part of the canonical chain~~
~~- Deletion: the block where the nonce's hash was published isn't part of the canonical chain and should be deleted~~

N.b. the last two states allow a safe nonce deletion hence those two
states could be left for UX purposed and cleaned-up later or discarded
entirely along with the nonce. Both are reasonable.

### Transitions (Outdated. See Architecture)

The state automaton's transitions are described as follow:

(initial state) `Published <publish_level>`
We created a nonce hash for a block at level `publish_level`

`Published <publish_level> -> Deletion (final state)`
When receiving a block at level `publish_level + 5` if the block at
`publish_level` isn't the one for which we produced the nonce then the
block is not finalized and the nonce is "orphan". It is safe to delete
it.

`Published <published_level> -> Injected <injection_level>`
We entered the cycle following the one where the nonce's hash was
found in a block **and** the current head's level is _greater than_
`publish_level + 5`. We inject the nonce and remember the level
(`injection_level`) where we injected it.

`Injected <injection_level> -> Noticed <notice_level>`
While monitoring new arriving blocks, we notice that our operation is
part of one of them. We remember the level of the block
(`notice_level`) that contains our nonce revelation operation

`Injected <injection_level> -> Injected <injection_level>`
When receiving a block at level `injection_level + 5` (arbitrary but
must be >=2) , if we did not notice any block that contained our nonce
reveleation operation, it means that our revelation was either lost or
part of a reorganized block. We reinject it and update its
`injection_level` to the current head.

`Noticed <notice_level> -> Finalized (final state)`
When receiving a block at level `notice_level + 5` (arbitrary but must
be >=2) , if the last finalized block's context confirms that our
nonce was successfully revealed then the nonce is considered as
finalized and can be safely discarded.

`Noticed <notice_level> -> Injection <injection_level>`
When receiving a block at level `notice_level + 5` (arbitrary but
must be >=2), if the last finalized block's context **and** the
current's head context **does not** confirm that our nonce was
successfully revealed then we reinject it at `injection_level`.

`Noticed <notice_level> -> Notice <notice_level>`
When receiving a block at level `notice_level + 5` (arbitrary but
must be >=2) new blocks past our `notice_level`, if the last finalized
block's context **does not** confirm that our nonce was successfully
revealed **but** the current's head context confirms it then we update
our notice level.

### Architecture

More formally, the `nonce_data` and `nonce_state` are approximated in the types below.

(* [nonce_data] associates nonce with the block where it was
   committed in. [block_hash], [level], [round] and [cycle]
   are used to skip querying the block store where possible *)
type nonce_data = {
    nonce: Nonce.t
    nonce_hash: Nonce_hash.t
    block_hash: Block_hash.t;
    level: Level.t;
    cycle: Cycle.t;

type nonce_state = 
    | Committed
        (* Nonce was committed:
            - If baker in current cycle: do nothing
            - If baker in cycle + 1:
                - If block with commitment is part of the 
                  cannonical chain: then transition to injected
                - else: safe to delete
            - If current cycle > cycle + 1: too stale, so safe to delete *)
    | Injected of { injected_level: Level.t; op_hash: Block_hash.t } 
        (* Nonce was injected:
            - At each level, get the last finalized block (level - 2)
            - If the revelation operation found in that block: nonce is safe to 
              delete
            - If not found:
                - If pass `re_injection_threshold`: 
                    - consider revelation operation lost, so re-inject
                    - transition to injected with updated level
                - If pass `nonce_revelation_threshold`:
                    - missed out revelation operation, so delete
                - else: do nothing
        *)

type nonce = {
    data: nonce_data;
    state: nonce_state
}

Change to existing code:
1. ~~`Baking_nonces.state` will keep track of nonce states~~ `nonces` will map `nonce_hash` to `nonce_data`
2. On startup, baker will load both legacy and new format with new format taking precedence. For nonces only found in legacy format, routine will make best effort to port legacy nonces to the new format. `save` will write to both legacy and new format (already implemented).
* Alternatively, the baker will fail if the startup routine cannot port all legacy nonces, the routine will ask the user for more block history (We could consider a `octez-client port legacy nonces` capability or some kind of interactive populate). There will be a flag `--ignore-legacy-orphaned-nonces` that could be configured to raise a warning instead of failing. At this point, `Baking_nonces.state` is pre-populated with actively tracked nonces or empty if there are none.
5. On new proposal, filter out nonces into 3 buckets - current (this cycle), stale (too old), pending (to be processed). Process pending nonces by returning new set of nonces with updated state. If deleted, we don't include in the new set. Due to the small number of operations to inject , (worst case: 24576/192=128), it should be fine to inject operation synchronously, which is generally quite low cost anyway.
6. Update `save` to store new format with state
7. On state transition, appropriate side effects need to take place. These are either inject or delete - similar to what happens now.

## Sanity check

At the baker launch, each nonce's state needs to be updated so that we are able to determine/update the nonce's state.

(TODO)