# Why reorg implementation can stall the network
## A.K.A. Loss of execution
The fact that we have introduced new struct `ReorgInfo` event which will invoke reorg in both Orchestrator and Pandora can lead to point of no return, which is loss of execution.
Please read and understand https://hackmd.io/mJhwmrPpQpCBeQ2uHE7NbA before you try to understand current document.

Pandora has the internal fork mechanism. It takes the highest known block in p2p sync.
Vanguard has the fork mechanism. It takes FFG to trigger the reorg.

## Current path
1. When vanguard encounters reorg it will propagate new `minimalConsensusInfo` with additional field `ReorgInfo`.
2. Orchestrator will override its database with fresh informations
3. Orchestrator will propagate `minimalConsensusInfo` with `ReorgInfo` to Pandora
4. Pandora will insert `minimalConsensusInfo` to its db
5. Pandora will trigger reorg, when `ReorgInfo` is present
6. Pandora will set head to reorgInfo, parentBlock number

## Scenario
20 nodes network.
20 consecutive validators on each node. Lets name them (val0, val1...).
Assumption that we have hydra fork protection.
Peer connectivity 19 each side (pandora, vanguard).
Let:
`All of the slots on vanguard side contains a block`
`Pandoras are at block number 220 with hash 0xabc`
`Vanguards are at slot 220 with already propagated block 0xdef and sharding info 0xabc`

`Epoch 6 slots: 221, 222, 223`
`Epoch 7 slots: 224`

On slot 221 val0 starts to produce block `0xdef1` with sharding info `0xabc1` based on block `0xdef` and parent sharding info `0xabc`
Both blocks pandora, vanguard did propagate theirs blocks to 10/19 peers. Range (node0, node1), Range: (node4, node12).

On slot 222 val1 starts to produce block `0xdef2` with sharding info `0xabc2` based on block `0xdef1` and sharding info `0xabc1`
Both blocks pandora, vanguard did propagate theirs blocks to 9/19 peers.
Range (node0, node1), Range(node3, node11).

On slot 223 val2 starts to produce block `0xdef3` with sharding info `0xabc3` based on block `0xdef` and sharding info `0xabc`.
Both blocks pandora, vanguard did propagate theirs blocks to 1/19 peers.
Range (node3, node3).

On slot 224 val3 starts to produce block `0xdef4` with sharding info `0xabc4` based on `0xdef3` and sharding info `0xabc3`.
Both block pandora, vanguard did propagate theirs blocks to all 19/19 peers.
Range (node0, node2), Range(node4, node20).

Vanguard did trigger the reorg. All of the nodes except val3 will receive reorg pair: `VanguardParentHash: 0xdef, PandoraParentHash: 0xabc`.

### Experiment was written down here:
https://github.com/lukso-network/lukso-orchestrator/blob/spike/pandora-execution-loss/cmd/fork/main.go

#### Output of experiment:

2021/11/02 15:35:31 Err counter: 7 received data loss err: 0xabc3 block hash not found in node: 7
2021/11/02 15:35:31 Err counter: 12 received data loss err: 0xabc3 block hash not found in node: 12
2021/11/02 15:35:31 Err counter: 8 received data loss err: 0xabc3 block hash not found in node: 8
2021/11/02 15:35:31 Err counter: 11 received data loss err: 0xabc3 block hash not found in node: 11
2021/11/02 15:35:31 Err counter: 2 received data loss err: 0xabc3 block hash not found in node: 2
2021/11/02 15:35:31 Err counter: 16 received data loss err: 0xabc3 block hash not found in node: 16
2021/11/02 15:35:31 Err counter: 18 received data loss err: 0xabc3 block hash not found in node: 18
2021/11/02 15:35:31 Err counter: 4 received data loss err: 0xabc3 block hash not found in node: 4
2021/11/02 15:35:31 Err counter: 19 received data loss err: 0xabc3 block hash not found in node: 19
2021/11/02 15:35:31 Err counter: 10 received data loss err: 0xabc3 block hash not found in node: 10
2021/11/02 15:35:31 Err counter: 13 received data loss err: 0xabc3 block hash not found in node: 13
2021/11/02 15:35:31 Err counter: 5 received data loss err: 0xabc3 block hash not found in node: 5
2021/11/02 15:35:31 Err counter: 0 received data loss err: 0xabc3 block hash not found in node: 0
2021/11/02 15:35:31 Err counter: 17 received data loss err: 0xabc3 block hash not found in node: 17
2021/11/02 15:35:31 Err counter: 14 received data loss err: 0xabc3 block hash not found in node: 14
2021/11/02 15:35:31 Err counter: 15 received data loss err: 0xabc3 block hash not found in node: 15
2021/11/02 15:35:31 Err counter: 9 received data loss err: 0xabc3 block hash not found in node: 9
2021/11/02 15:35:31 Err counter: 1 received data loss err: 0xabc3 block hash not found in node: 1
2021/11/02 15:35:31 Err counter: 6 received data loss err: 0xabc3 block hash not found in node: 6

## New path that prevents data loss
1. When vanguard encounters reorg it will propagate new `minimalConsensusInfo` with additional field `ReorgInfo`.
2. Orchestrator will override its database with fresh informations
3. Orchestrator will propagate `minimalConsensusInfo` without `ReorgInfo` to Pandora
4. Pandora will insert `minimalConsensusInfo` to its db