Rust Pre-mortem
Why?
Base is running op-reth in production and will transition their sequencer to op-reth. If op-reth has a chain split with op-geth due to some unknown or introduced bug, the consequences are dire - the chain would need to be rolled back.
This is a pre-mortem document to asses why this happened and how OP Labs can evolve as an organization to prevent this going forward.
Background
op-reth was initially built from Eth Denver 2023 (February) - Paradigm Frontiers 2023 (August) by @refcell and @clabby - two OP Labs engineers. Afterwards, support by OP Labs was not given so the burden fell on reth core contributors to maintain as well as Base engineers (Brian). As Base has scaled, they've adopted op-reth as a solution to op-geth's poor scaling.
OP Labs never committed to running internally. We are just now beginning to run op-reth in our infrastructure as a reactionary measure to Base productionizing the client.