Mir-Based Ordering Layer for Eudico - MVP

This document is deprecated. An up-to-date implementation plan can be found in the associated GitHub issue

This document describes in detail how to reach the Y3-M3: Practical consensus protocol (MVP) milestone, after the reaching of which Eudico will have a minimal but working option modular subnet consensus, albeig without extra features or performance optimizations. It is a refinement and concretizition of the high-level description of the ordering layer design.

Items to address (Priority 0)

• [2 weeks] Well-defined Eudico consensus interface ( @matejpavlovic )
  • Guarantees the Eudico mempool has to provide regarding
    • Durability
    • Delivery guarantees
  • Interaction with Eudico's state
    • Applying blocks
    • semantics of snapshots
    • durability of applied blocks
  • Source of system configuration
  • Consensus protocol metadata
    • especially the "merit" assignment required for rewards
    • Miner assignement and block signing mechanism

Current draft on HackMD

• [5 weeks] Preliminary performance evaluation

  • Implement (just for Mir):
    • Load generator (1 weeks)
    • Distributed deployment (2 weeks)
  • Evaluate performance (2 week)

• [8 weeks] Mir App module for the state manager

  • Mempool integration and block assembler (1 week)
  • (Re)Configuration
    • Format of configuration data and config transaction flow (1 week)
    • Reconfiguration
      • Instantiating components with new configuration (1 week)
      • Adding / removing nodes (1 weeks)
      • Garbage-collection of old components (1 week)
    • State transfer (1 week)
    • Debugging / buffer (2 week)

• [4 weeks] Finish consensus layer implementation ( @matejpavlovic )

  • Message retransmission where necessary (0.5 weeks)
  • Instance-local checkpoints in PBFT (0.5 weeks)
  • Proposal distribution: include in PBFT Preprepare (0.5 weeks)
  • Secure crypto module (1 weeks)
    • Understanding crypto in Eudico
    • Wrapping it in a Mir module
  • Properly handling of repeated requests (1 week)
  • buffer (0.5 weeks)

• [3 weeks] Availability layer (Andrei's first task)

• [7 weeks] Tests of basic functionality ( partly @sergefdrv )

  • Tests of Mir itself (4 weeks) (@sergefdrv)
    • Simulated network perturbing messages
    • Logic checking the correctness of Mir output
    • Test cases of chosen scenarios, e.g. crashes
  • Eudico integration tests (1 week) (@dnkolegov)
  • End-to-end tests, probably overlapping with #76 (2 weeks)

• [4 weeks] Mempool interface for Eudico

  • Define the new Mempool's interface (1 week)
    • Querying the mempool for new requests
    • Optional push-style notifications about new requests in the mempool
    • Querying the state of the mempool, if not covered by previous functions
  • Create a Eudico-compatible implementation of the new mempool interface (2 weeks)
    • In particular, ensure per-client FIFO ordering of requests

• [2 weeks] Mir transport module based on libp2p

Total estimate: 31 weeks

Estimates currently chosen rather generously, to be refined.

Potentially also include in the M3 milestone if ahead of schedule (Priority 1)

• Availability layer
  • Durable key-value store for batches and certificates
    • Use out-of-the box key-value store implementation, e.g. go-datastore / BadgerDB.
    • Wrapper as a Mir app
• Restarts and crash recovery
  • Add support for recovery from WAL
    • to ISS
    • to Availability layer
  • Parametrize Eudico startup accordingly
  • Implement (with Eudico):
    • Use/configure Eudico / Lotus benchmarks, if any, for end-to-end evaluation or implement simple ones
• Add persistence to the mempool implementation

Initial assignment

• @sergefdrv:
  • Finish state transfer of ISS
  • Then switch to writing all kinds of test to show that the system is stable
• @dnkolegov:
  • Propose a design of the main interface (with @matejpavlovic)
  • Then make necessary changes in Eudico
• @matejpavlovic:
  • Propose a design of the main interface (with @dnkolegov)
  • Then switch to preliminary performance evaluation