EPM/Staking-miner - commit-reveal based submission spec

- https://github.com/paritytech/substrate/pull/13129 - https://github.com/paritytech/substrate/issues/11762 ### Why? - Staking miners need to submit the full solution at every `signed_phase`; - Expensive tx-fee (although honest miners get refund). - Not staking-miner friendly, especially in a "staking miner market" context. - This may be good or bad (open up the market for redundancy without making it too large). - However, things *do* work nowadays; - Not a priority, compared to other work. - Opening up a can of worms? ### Goals - Initially, miners won't have to pay heavy tx-fee to participate; - Deposit is still required, as slashes may happen. - Less data stored onchain; - In the best case, only 1 solution needs to be stored and processed. ### How? The EPM signed phase is broken down into 3 new phases: `Commit`, `Reveal-N`, `Reveal-Fallback`: 1. **Commit-phase**: Every staking miner submits a commitment, which consists of a solution score. (`SolutionCommitment{ score: ElectionScore}`); - Each commitment is queued (ordered); 3. **Reveal-N**: `N` dedicated slots for the miner with the best commitment to submit the solution. There are only `T::NumberRevealSlots` per election for the best commitments, each with a lenght of `T::RevealSlot` blocks. 4. **RevealFallback**: If no valid solution was revealed during the `Reveal-N` phase, then open the reveal to every miner. All the remaining phases (`Phase::Off`, `Phase::Unsigned`) remain unchanged. --- The commit-reveal based submission consists of separating the election solution submission by OFWs into 2 distinct phases: 1) Commit: the staking miners commit to a solution with a given score; 2) Reveal: the staking miner that submitted the best solution reveals the solution, similar to what happens nowadays. If the worker does not reveal the solution, the feasibility test fails or the final score does not match the committed, the OFW is slashed and the next worker in the commitment queue may submit the solution. In the commitment phase, the staking miner submits the solution commitment (i.e. an `ElectionScore`) and the deposit associated with it; In the reveal phase, each submitter has the opportunity to input the solution associated with the commit. If the OFW misbehaves somehow (wrong commitment, wrong reveal or not showing up), it is sleshed. Each committer has its allocated phase to submit the solution `Phase::Reveal(submitter, left_blocks)`; ### Election Provider phases with commit-reveal: Currently, the EPM phases are the following: ``` elect() + <--T::SignedPhase--> + <--T::UnsignedPhase--> + +-------------------------------------------------------------------+ Phase::Off + Phase::Signed + Phase::Unsigned + ``` With the commit-reveal strategy, the `T::SignedPhase` is broken down into 3 sub-phases, measured in blocks. - `T::Commit`: Time for OFW to submit solution commitments; - `T::DedicatedRevealSlot {AccountId, block}`: A set of dedicated slots for the commitment winner to reveal its solution; Each `T::DedicatedRevealSlot` will have a duration of `T::RevealSlotDuration` blocks until `block`. - `T::FallbackReveal`: In case there was no valid reveal during the dedicated reveal slots, the `FallbackReveal` lets any OFW to reveal their solution. The number of slots for solution reveal is `T::DedicatedRevealSlots::get() + 1`, where the top `n` committers in the commitment queue have a dedicated slot to submit their solution. The last slot would be fallback option open to any committer to reveal a solutions, in case none of the miners with the top scores submitted their solution or their solution was incorrect. In the fallback option the chain would process all the solutions as currently, starting from the better solution. This way can have a larger time slot for each dedicated reveal without constraining the number of committers. The EPM phases with commit reveal is the following: ``` elect() + <--T::Commit--> + <--T::DedicatedRevealSlot * T::RevealSlotDuration--> + <--T::FallbackReveal--> + <--T::UnsignedPhase--> + +---------------------------------------------------------------------------------------------------------------------------------------------------+ Phase::Off + Phase::Commit + Phase::DedicatedRevealSlots + Phase::FallbackReveal + Phase::Unsigned ``` > Note: we could consider collapsing the `Phase::FallbackReveal` and `Phase::Unsigned`, to simplify the logic. In that case, if no solution was correctly revealed during `Phase::DedicatedRevealSlots`, EPM would accept both commited reveals or unsigned solutions. ### Other considerations: - Staking miners DDoS-ing each other for profit? - The commit-reveal approach opens up an opportunity for bad actors to DoS other miners that are placed in front of them in the commitment queue. This has 2 benefits for the attacker: on one hand can increase the chances of their solution being selection and on the other hand their competitors get slashed. - How does the commit/reveal phases play with the [emergency-phase throttling](https://github.com/paritytech/substrate/pull/13040) that we've been discussing about? - Any other 2/3rd order effects from having this design? --- ### Implementation details #### A. Phase management The phase transition/management changes consist of breaking down the signed phase into `commit`, `reveal-n`, `reveal-fallback` phases. Each phase size in blocks depends on pallet configurations and/or state of the current phase. ![](https://i.imgur.com/kDdiRLQ.jpg) #### B. New pallet configs associated types - `T::CommitRevealPhase`: Number of blocks allocated for the commit-reveal phases; - `T::CommitPhase`: Number of blocks allocated for `Phase::Commit`; - `T::RevealSlot`: Number of blocks allocated for each of the dedicated reveal slots; - `T::NumberRevealSlots`: Number dedicated slots for the reveal phase; #### C. `enum Phase` - New phase: `Phase::Commit` - indicates that EPM is accepting commitments for solutions from miners; - New Phase: `Phase:Reveal { submitter, left_blocks}`: indicates that the EPM is expecting a solution from `submitter` and there are `left_blocks` for the solution so be submitted. - New Phase: `Phase:FallbackReveal`: indicates that the EPM is expecting a solution from any submitter. - Remove `Phase::Signed` the signed phase is replaced for `Phase::Commitment`, `Phase::Reveal` and `Phase::FallbackReveal` **Phase invariants**: - `T::CommitRevealPhase >= T::CommitPhase + T::NumberRevealSlots * T::RevealSlot` - The length of `Phase::RevealFallback` is the difference between `T::CommitRevealPhase` and the blocks required for `T::CommitPhase` and all the revel slots. It may be zero if a valid solution was queued during the `Phase::RevealSlot`s. - `Phase::RevealFallback = T::CommitRevealPhase - T::CommitPhase + T::NumberRevealSlots * T::RevealSlot || 0` #### D. Storage - Bounded commitments map to store the submitted commitments during `Phase::Commit`. Best solution can be calculated on the fly when `Phase::Commit` ends (small number of commitments allowed, no need to sorting). ```rust #[pallet::storage] #[pallet::getter(fn commitments)] pub type CommitmentsMap<T: Config> = StorageMap<_, Twox64Concat, ElectionScore, T::AccountId, OptionQuery>; ``` - Remove storage items related to signed submissions, namely `SignedSubmissionMap`, `SignedSubmissionNextIndex` and `SignedSubmissionIndices`. #### E. Events - `Event::CommitmentStored {score: ElectionScore, accountId: AccountId}` - `Event::SubmittedSolution {score: ElectionScore, accountId: accountId}` - emits an event that a solution has been committed with score `score` from commitment `commitment`. - `Event::SkippedReveal {accountId: AccountId}`: when a submitter missed their dedicated reveal slot. #### D. Slashing When a solution reveal for a dedicated solution is a) wrong (i.e. solution does not match commitment) or b) not submitted on time, the account associated with the commitment is slashed (similar to current bad solution slash). --- ### Pre-image pallet for commit-reveal The pre-image pallet allows runtime users to store pre-images of a hash on chain. It can be used to request, store and manage large byte blobs on-chain. **Calls** - `Call::note_preimage (bytes: Vec<u8>)`: register the preimage on-chain; If the preimage was requested with `Call::request_preimage`, no fees or deposits are taken to deposit. - `Call::unnote_preimage (hash: T::Hash)`: clears an unrequested preimage from the runtime storage. - `Call::request_preimage (hash: T::Hash)`: requests a preimage to be uploaded to the chain without paying any fees or deposits. - `Call::unrequest_preimage (hash: T::Hash)`: clears a previously made request for preimage. **Configs** - `type ManagerOrigin: EnsureOrigin<Self::RuntimeOrigin>`: Origin that can request preimages without deposit/fee and manage existing preimages. **Flow using the pre-image pallet for commit-reveal** 1. `CommitPhase` starts; staking miners submit their commitment `(ElectionScore, T::Hash)`. 2. `CommitPhase` ends: EPM select the best commitment (highest election score) and calls `T::PreImage::request_preimage(hash)`, where `hash` is the hash associated with the winning commitment; 3. Winner staking miner calls `PreImage::note_preimage(bytes)` to reveal their solution associated with the commitment hash. 4. At the beginning of each block in `Phase::RevealPhase`, EPM checks if the correct/expecteed solution has been noted in the preimage pallet. - If yes, then run 1) check if the preimage submitter's `accountID` matches winner; 2) the `feasibility_check` of the solution passes. After the submission is processed, call `Call::unrequest_preimage` to clear the preimage. - If no preimage is submitted in the expected time, slash staking miner and proceed for next phase. Advantages of using the pre-image pallet: - Gate keeping solution submission based on the commit hash; - Re-use logic to submit/delete submission blob. - Solution submissions do not need to pay fee/deposit (note that a deposit needs to be made in the EPM side for slashing); --- ### Meeting notes (12/02/23) - Rename back from `Phase::RevealFallback` to `Phase::Signed` (and keep the signed phase as backwards compatible); - [x] Re-assess the usage of the pre-image pallet; - Evaluate costs of storing the submitted commitments as a sorted map (or other) or a flat vec of `(AccountId, ElectionScore)` tuples; - Work on prototype with E2E unit tests;