--- date: 2022-05-20 tags: mev, research, ethereum title: MEV Monitoring Design --- # MEV Monitoring Design Possible monitoring design in case of using MEV-boost for MEV extraction. ## Participants ### MEV-Boost [MEV-Boost](https://github.com/flashbots/mev-boost) is used by validators to delegate block building to third-party software. It polls relays and fetches from them variants of the suggested payload and the value that could be extracted by using this payload. The payload with the maximum value is selected and included to a block. ### Relay Relays are used by MEV-Boost to get built blocks. In the monitoring design we assume that relays provide a method, which returns the list of payloads and values that were suggested to a validator for a slot. The method has not yet been implemented, but we are [discussing the possibility](https://github.com/flashbots/mev-boost/issues/120). ### Monitoring The monitoring service inspects the operation of the validators and collects data on how many value were extracted from each block, a source of the payload, a recipient of value and other metrics. ## Design ### Overall scheme ```sequence Participant DB Participant Registry Participant Epoch inspector Participant Slot inspector Participant Relay observer Participant CL Participant EL DB->Epoch inspector: Last inspected epoch Epoch inspector-->CL: Next epoch duties CL->Epoch inspector: Proposers duties Epoch inspector-->Registry: Filter validators Registry->Epoch inspector: Lido validators Epoch inspector-->Slot inspector: Inspect slots Note right of Slot inspector: Inspect slots in loop CL->Slot inspector: Slot data Slot inspector -> Slot inspector: Inspect fee_recipient Relay observer->Slot inspector: Slot's suggested payloads Slot inspector->Slot inspector: Inspect block payload EL->Slot inspector: Block with transactions Slot inspector->Slot inspector: Inspect block builder tx Slot inspector->Epoch inspector: Inspected slots Epoch inspector->DB: Save epoch inspection ``` ### Relay observer The service regularly checks the availability of the white-listed relays, fetches suggested payloads from them and stores this information to a database. The inspector service then compares these payloads with what is included in the block by the validators. ```sequence Participant Observer Participant Relays Participant DB DB -> Observer: Last saved slot Observer -> Observer: Calculate current slot Observer -> Observer: Detect unprocessed slots Relays -> Observer: Availability check Relays -> Observer: Suggested payloads Observer -> DB: Save results ``` ### Epoch inspector service The service fetches the finalized epochs one by one and inspects them: 1. Fetches the proposers duties from the consensus layer client. 2. Identifies the validator and node operator for each slot. 3. [Inspects each slot](#Slot-inspector-service). 4. Store the inspection result to DB. ### Slot inspector service The service fetches information about the slot, inspects it and return the result and collected data. The service does the following inspections: #### Slot status Service inspects whether a block is proposed in the slot. #### Source of payload The service calculates `transaction_root` based on the list of transactions in the block and looks for such a root in the list of payloads suggested for this slot by relays. If a match is found, we can say that the payload is suggested by one of the relays from the white-list. Otherwise we expect that the relays are not available and the local payload is used. Or the validator intentionally uses a different payload for its own purposes. #### Block value The service inspects: * How many rewards for the block are sent to the `fee_recipient` and Lido MEV Vault addresses. * Is there a transaction in the block with a transfer from the builder and the value of this transfer. * Is there any suggested payloads with a higher value from the relays. :::warning We need to separate balance changes by block rewards from third-party transactions, such as withdrawals. This will probably require a node with the trace module enabled or a third-party API such as Etherscan. For the Lido MEV Vault, we can track withdrawals by event, but for builder it's more difficult to do so. ::: #### Fee recipient The service inspects the `fee_recipient` address and expects one of the following values: *Builder address* is expected in the case of using a payload from MEV-Boost. In this case, the block must include a transaction to transfer the validator reward to the Lido MEV Vault address. *Lido MEV Vault address* is expected in the following cases: * All relays were unavailable. * Fallback to a local or remote EL client if there is a fault in the MEV-Boost. * The validator included payload for its own interest. *Unknown address* is expected in the following cases: * Address belongs to one of the builders, and for some reason Relay observer did not collect suggested payloads. In this case there is a validator reward transaction in the block. * Configuration error or validator hacking. * Validator intentionally receives MEV and tx fees to its own address. ### Data analysis The results of the inspection are stored in the database and used for further analysis, visualization and alerts. As a result, the following data are collected for each slot: * Was the block proposed in the slot. * Which Node Operator’s validator proposed the block. * What were suggested payloads by the whitelisted relays. * Were the whitelisted relays available at the time of the propose. * Which relay was used to build the block. * Was there a transfer from the builder, to what address and what the value. * How much the fee_recipient balance has increased. * How much Lido MEV Vault balance has increased. Based on this data, the following analysis can be done: * Understand which validators do not use MEV-Boost. * Find validators that extract MEV to unknown address. * Understand how often blocks are missed in slots and what relays may have been selected by MEV-Boost for this slots. * Find validators using a local payload, the possible reasons for using it and the frequency of use. * Calculate and compare the average extracted value over long distances and see if censoring of Lido validators is present. In some cases, it is not immediately possible to detect obvious malicious behavior of a validator, since it's always possible that all relays are unavailable and the validator has to build a block using a local payload. But in large samples it's visible if validators of one of the node operators have regular misbehavior. ### Summary analysis A summary table of possible conclusions based on the collected data. <table> <thead> <tr> <th colspan="2"></th> <th style="text-align: center;"><abbr title="Payment via transaction">Tx</abbr></th> <th>Known payload</th> <th>Unknown payload</th> </tr> </thead> <tbody> <tr> <th rowspan="4"><abbr title="Fee recipient">FR</abbr></th> <th rowspan="2">Lido vault</th> <th style="text-align: center;">+</th> <td colspan="2" style="background: rgba(255,255,0,.1)">😵‍💫 Could be a misconfiguration of the builder</td> </tr> <tr> <th style="text-align: center;">−</th> <td style="background: rgba(0,255,100,.1)">🧐 Builder don't charge a commission</td> <td style="background: rgba(255,255,0,.1)">🧐 Local payload or unknown source is used</td> </tr> <tr> <th rowspan="2">Another</th> <th style="text-align: center;">+</th> <td style="background: rgba(0,255,100,.1)">👍 Most expected option</td> <td style="background: rgba(255,255,0,.1)">🧐 Unknown source is used</td> </tr> <tr> <th style="text-align: center;">−</th> <td colspan="2" style="background: rgba(255,0,0,.075)">😱 Rewards collect to the wrong address</td> </tr> </tbody> </table> The table contains many assumptions and does not take all data into account. For example, payment in a transaction may be less than promised; monitoring may not have all of suggested payloads for a slot; payloads with a larger value may be suggested. The real analysis will require more detailed investigations.