owned this note
owned this note
Published
Linked with GitHub
# [DRAFT] A Study of Threshold-Decrypted Mempools, MEV and Their Benefit to Users
id: 30
title: A Study of Threshold-Decrypted Mempools, MEV and Their Benefit to Users
team: @itranneo @obadiaa
created: 2023-03-03
status: active
---
## Background and Problem Statement
Encrypted mempools are a class of solutions that help mitigate behaviours considered nefarious to users such as 'frontrunning' or 'sandwiching'.[^1] The basic approach behind them consists in minimizing user data leakage by hiding sensitive transaction data until the blocks in which the transaction is included is committed.
There are [multiple](https://joncharbonneau.substack.com/p/encrypted-mempools) [ways](https://github.com/JBStearn/FRP-18/blob/main/Cryptographic_Approaches_to_Complete_Mempool_Privacy.pdf) to build an encrypted mempool. Designs based on threshold decryption schemes are favorite candidates of projects like [Arbitrum](https://blog.chain.link/arbitrum-and-chainlink-fair-sequencing-services/), [Osmosis](), [Anoma](https://github.com/anoma/ferveo), and [Penumbra](https://protocol.penumbra.zone/main/crypto/flow-encryption/threshold-encryption.html). This is because:
* such designs are elegant as some of them can be overlaid on top of Byzantine Fault Tolerant consensus protocols (ie. by reusing the 2/3 majority assumption and appending threshold key shares to consensus votes).
* such designs mesh relatively well with first-come first-serve ordering protocols to create latency-minimized 'blind-order fairness'.
* the complexity of implementation is seemingly low, especially in the Cosmos ecosystem where this will soon be implementable out of the box using the Cosmos SDK with [abci++](https://youtu.be/cAR57hZaJtM).
* many of these designs are implemented/ready to be implemented today (ie. the additional overhead & latency is acceptable for many networks with a 'small' number of validators, the technology actually works).
However, there are multiple known drawbacks to such solutions. We find that they fall into two categories:
* **security**
* whether the 2/3 honesty majority assumption is reasonable to make given the economic incentive (MEV) for the committee to misbehave
* how this honesty assumption overlays with the consensus majority assumption made if those are the same actors
* how it overlays with a third protocol relying on the same honesty assumption like an first-come first-serve ordering protocol
* non-attributability of misbehavior
* incentive compatibility for committee members
* liveness attacks on the system
* blind-order fairness gameability and other ordering protocols gameability
* reorg incentives on chains without fast finality
* cap on size of the validator set - the overhead currently doesn't work for consensus protocols like ethereum if all validators were participating in the protocol
* **economic welfare**
* welfare loss from latency incurred from encryption/decryption, and additional bandwidth needed (we anticipate this to be minimal)
* welfare loss from markets moving 'slower' than their counterparts and users swapping on stale prices
* welfare loss from sub-optimal blockspace allocation by not being able to see transactions and therefore order them at 'mev-time', or backrun them efficiently (at the extreme, some allocation rules could cause for example spam transactions from searchers attempting to capitalize on probabilistic strategies)
This has prompted some to dismiss the family of solutions entirely, while others have forged ahead, claiming these issues are fixeable, partially charmed by the ease of implementation of such solutions. However, we haven't seen rigorous analysis of the drawbacks listed above yet. We fear this could end up being at the detriment of users of the systems that choose to adopt such solutions.
This leads us to our problem statement for this work: **is using threshold-decrypted mempool a net improvement for users?**
## Plan and Deliverables
We plan to produce a paper or a blogpost with a summary of our findings.
A few steps we plan to take as part of our study include:
* thoroughly analyse the drawbacks and benefits of threshold-decrypted mempools (and the nuances that may exist in their different design & implementation)
* put ourselves in the shoes of an MEV searcher (i.e. a quant trader) and think through the ways a trading opportunity can be seized in a simplified system where all transactions are threshold-decrypted, then consider other cases (eg. where a subset of transactions are threshold-decrypted, where another domain has transactions in the clear etc)
While we plan to share our findings of the study above. We would also like our work to survey the problem space and open the discussion to the broader community by surfacing what we see as interesting research questions needing further work.
If need be, the study may then be refined to account for specificities of selected networks and projects (e.g. to account for specific markets, different MEV governance processes - how it is redistributed and captured within a specific network etc.). We expect our study to be limited by the lack of data about systems using threshold-decrypted mempools, and may thus be revisited once a data trail exists on such systems.
## Further Work
We hope the work above can pave the way for other work and interesting questions we already have in mind, those include:
- devise a unifying framework to think of tradeoffs and of a spectrum between 'complete' mempool privacy and full mempool transparency
- *we believe that a better understanding of the tradeoffs of privacy preserving techniques on blockchains will help protocol designers and their users decide the approach that best aligns with their vision, values and requirements.*
- elucidate potential mitigations to problems found and different market structures that could increase the attractiveness of such solutions and create a testing framework to experiment with different mempool designs through simulations
- explore potentially deeper results that are not only specific to threshold-decrypted mempools but can be generalized to all mechanisms that hide the mempool with no expressivity ('complete' mempool privacy)
Some deeper threads we would hope our initial study or any of the above further work could hit on include: consensus protocol design, leaderless block construction, and security analyses of out-of-protocol threshold-decryption solutions like EigenLayer & Shutter Network versus 'in-protocol' solutions like cosmos sovereign chains using abci++.
## References
- [FRP-18: Cryptographic Approaches to Complete Mempool Privacy by James Stearn](https://github.com/JBStearn/FRP-18/blob/main/Cryptographic_Approaches_to_Complete_Mempool_Privacy.pdf)
- [Ferveo: Threshold Decryption for Mempool Privacy in BFT networks](https://eprint.iacr.org/2022/898) by Joseph Bebel (Anoma), Dev Ojha (Osmosis Labs)
- [Performance of EdDSA and BLS Signatures in Committee-Based Consensus by Li, Sinnino, Jovanovic](https://arxiv.org/abs/2302.00418)
- [Cryptoeconomic Security for
Data Availability Committees by Tas and Boneh](https://arxiv.org/pdf/2208.02999.pdf)
- [Questions around mempool privacy using threshold encryption](https://collective.flashbots.net/t/questions-around-mempool-privacy-using-threshold-encryption/520?u=alex) by @ra (Flashbots)
- [MEV and Fair Ordering - Valeria Nikolaenko and Dan Boneh](https://youtu.be/T1bD7_OTD1o)
- [Penumbra docs on threshold encryption](https://protocol.penumbra.zone/main/crypto/flow-encryption/threshold-encryption.html)
- [John Charbonneau's Survey of Encrypted Mempools](https://joncharbonneau.substack.com/p/encrypted-mempools)
- [Justin Drake on Encrypted Mempools](https://youtu.be/XRM0CpGY3sw)
- [Threshold Decrypted Transactions: Thwarting Front-Running and Enabling Privacy at the Protocol Level by Sunny Aggarwal (Osmosis Labs)](https://youtu.be/6WrFlsDSUYg)
- [Shutterized Beacon Chain](https://ethresear.ch/t/shutterized-beacon-chain/12249)
- [Shutterizing Gnosis Chain: Step 0](https://www.notion.so/Shutterizing-Gnosis-Chain-Step-0-f2b900eea7d744e097a62abd8bacf6e8)
[^1]: whether it is nefarious or not is the subject of a longer piece beyond the scope of this note.
<details>
<summary>misc/brainstorming</summary>
# Brainstorming:
This FRP looks into the market impact of encrypted mempools. While different methods can be employed to hide transaction data before it gets finalized and added to the chain, e.g. using TEEs (see e.g. [this survey](https://www.net.in.tum.de/fileadmin/TUM/NET/NET-2022-07-1/NET-2022-07-1_05.pdf)) or timelock encryption for instance (see e.g. [here](https://vsekar.me/assets/diss.pdf)), this study will focus on the market impact of using threshold encryption methods for hiding transaction data prior to inclusion on the target blockchain.
Threshold encrypting blockchain transactions modifies the content of the messages gossiped on the networking layer and processed by validators. This also affects the information stream accessible to other market participant (i.e. traders, protocol designers monitoring activity on their DApps etc.) which will undoubtedly impacts market's dynamics. This FRP studies such market structure changes.
Scope of work:
- Using a threshold encryption (TEnc) oracle (i.e. without bothering with the specifics of the algorithm), specify how traders' information flows are modified when the transaction pool is TEnc'd.
- This is step 1 of the study. We need to define what is encrypted (just the CALLDATA most certainly), and state what stays plaintext. Once agreed, we will know what is publicly accessible and what isn't, and we'll be able to wear a trader's hat to play the market with the information available to us.
- Plaintext: Tx nonce, source address, gas price etc
- Ciphertext: CALLDATA
- New pieces of data available: the public keys of validators used to TEnc (this information is available to traders and may be used in their decision making process in one way or another)
- What MEV is still possible?
- Censorship (which is MEV, especially when trading on-margin on chain)
- Market impact of encrypted CALLDATA in the mempool?
- Beyond "classical" MEV, less visibility for traders on the order flows, meaning possibly more market swings (the tx pool can serve as a buffer to a on-chain CLOB and help smooth prices by gathering trade intents before they get executed) (AR: Conjectured, need to think a bit more)
- Less transparency on order flows gives room for less fairness and more asymmetry. E.g. "cartels of colluding validators" sharing their key shares to inspect the mempool, leading non-validator traders exposed to market manipulation. **This would create a new shadow market for decryption shares.**
- Note: This is a fondamental topic that is relevant to encrypted mempools in general. By their very nature, encrypted mempools can be a vector for unfair markets, where the decryption key holder (be it a single member or a colluding cartel) can win unfair advantage on other market participants by decrypting mempool data and manipulate markets. While threshold encryption is acclaimed to align security assumption on BFT chains, such claims are not always relevant. In fact, messing up with blocks or doing re-orgs can be observed by all blockchain users. However, in the context of TEnc (even when 2/3 of the keys need to be compromised) a colluding cartel of validators decrypting their encrypted mempools is much harder to notice for other market participants (!) than it to notice anomalies in the block creation!
---
Alex
**Motivation:**
Threshold decrypted mempools are the favorite candidate of projects like Arbitrum, Osmosis, Penumbra as a way to mitigate negative externalities of MEV extraction. The basic approach of a threshold decrypted mempool is to minimize MEV 'leakage' by hiding txs until they’re committed, then (or in parallel) run some kind of ordering protocol.
Tradeoffs of such a technique have been discussed, for example [here](https://github.com/JBStearn/FRP-18/blob/main/Cryptographic_Approaches_to_Complete_Mempool_Privacy.pdf) and [here](https://joncharbonneau.substack.com/p/encrypted-mempools)
However, to our knowledge, we haven’t seen a thorough study of how searchers (traders) would interact with such systems and wehther the system’s intent will stand the test of these actors. This starts from the basic observation that there is still money to be made eg from arbitrage, so the question becomes: how would a searcher seize such opportunities?
We fear this lack of formal research here will cause issues to chains adopting these systems that will negatively impact users, the same way Avalanche implementing leaderless caused them issues when faced with searchers.
**Planned approach to the problem**
- welfare in the system
- security/actors
**deliverables**
at the minimum:
- writing a survey of the problem space & how we think about it, including surfacing the interesting research questions & considering engaging relevant contributors with grants for particular sub-questions.
extras:
- experiment with different designs through simulations
- explore a potentially deeper result there that is not only specific to threshold crypto but can be generalized to all mechanisms that hides the mempool with no expressivity 'complete privacy mempools', like using VDFs.
- devise a unifying framework to think of tradeoffs between 'complete' privacy and full transparency wrt to mempool!
*One opinionated goal of this research is to consider whether we should call for the industry to direct more of its attention either to the MEV-aware block construction line of work, or the threshold encrypted mempool line of work depending on what makes sense for people.*
**deeper threads**
- consensus protocol design, leaderless consensus protocols
- security comparisons of out-of-protocol Tenc solutions like EigenLayer & Shutter Network vs in-protocol solutions
:[1]
These works survey different techniques e.g. using trusted execution enclaves, using timelock encryption, using verifiable delay functions for hiding transaction data on blockchain networks. However, as far as we know there is currently no work that dives deeply into threshold decrypted mempools and the behaviours of agents participating in such systems. One agent we are particularly interested is the trader, also called MEV searcher. It is currently unclear how such a player will behave within blockchain networks hiding pending transactions' data using threshold decryption.
Threshold-decrypting blockchain transactions modifies the content of the messages gossiped on the networking layer and processed by validators. As such, this affects the information stream accessible to market participants (not only traders/MEV searchers, but also, protocol designers monitoring activity on their DApps etc.). Such changes in the streams of available data will undoubtedly impact market dynamics.
The primary motivation for this work is our fear that a lack of research around the market implications of such MEV mitigation techniques could backfire and cause more harm than good to users. Simply put, the goal of this Research Proposal is to study the impact of threshold-decrypted mempools on blockchain systems, and take a first step at answering the question: "is using threshold-decrypted mempool a net improvement for users?".
// need to change the formulation of question, too broad!
Our hypothesis at this stage is that **it depends**.
By producing preliminary research on this topic, we want to suggest an answer to this question and open the discussion with the broader community. We will reason from first principles and keep the study strictly focused on traders' interactions with a blockchain system using threshold encryption as MEV mitigation technique. As such, results of the study may be relevant across a broad set of contexts.
// would consider deleting
To convey our point, we posit this thought exercise: in many systems adopting such encrypted mempool solutions, there will still be money to be made from arbitrage between two DEXs built on top of it, so we ask the questions:
- How would traders seize MEV opportunities on this system in a one-shot game? what about in the repeated game?
- What are tools and techniques at their disposal to gain an advantage over their competitors in these systems?
- What kind of negative externalities might exist from their activity? More broadly, what kind of negative externalities might exist from the activity of economically rational actors in such systems?
</details>
---