### The Optimism Sequencer: How it works within the Canonical Monorepo


The sequencer collects a set of rollup transactions, batches them, optionally compresses them, and posts them to L1. We will now focus on the block creation and finalization by the sequencer in the canonical op-node.

There are two main components responsible for block creation:
- Rollup Driver
- Execution Engine

The main tasks are accomplished by the *Rollup Driver*, which directs the *Execution Engine* to create the block via the *Engine API*. Once the *OpNode* is initialized, the *Rollup Driver* enters an event loop where the *sequencer* instance keeps calling the `PlanNextSequencerAction` function that returns a delay. In this loop, a timer (`sequencerTimer` **internal** to the function and dynamically updated) is reset after the previously computed delay is expired.

In every iteration, the *Rollup Driver* will craft a *payload attributes* object and send it to the *Execution Engine*. The *Execution Engine* will reply initially with a *payload ID*. Then the *Driver* will request the payload content by providing the ID obtained previously.

Here is a swimlane diagram of the process provided by Optimism [specs](https://specs.optimism.io/protocol/overview.html):


The illustration below instead shows how the different components interact. A subset of all the transactions that eventually will be included in the block come from early stages of the derivation process:
- L1 info transactions
- Deposit transactions
- Upgrade transactions


The following diagram illustrates how data flows between the *Driver* and the *Engine* to accomplish the block building process.


Data Flow:


Let's now go back to the block-building process:
Still inside the loop, after checking whether the *sequencer* action can be performed (a series of check is conducted), we save in the *payload* the result of `RunNextSequencerAction` and spread the content of the payload through the network with `PublishL2Payload`. This is achieved by the *Async Gossipper*, later introduced and called inside the `confirmPayload` to spread the block as soon as it is created.

#### RunNextSequencerAction:
This method handles block creation and encapsulation following these steps:
1. `d.engine.BuildingPayload`: checks if a block is being created and if it is safe it completes. Saving in the *payload* the result of `CompleteBuildingBlock`, otherwise it waits approximating the waiting time.
- `CompleteBuildingBlock` takes the current block that is being built, and asks the *Engine API* to complete the building and obtaining back the envelope of the block.
2. If no block is currently being created, calls the `d.StartBuildingBlock` that uses the *l1Origin* and if no error occurred, we should have a *payload ID* obtained from the `BuildingPayload` now stored inside the `buildingID`.
- `d.StartBuildingBlock`: prepares the *payload attribute* object given the unsafe L2 head and the L1 origin block to build on top of. Accomplishing the following:
1) Checks whether it needs to fetch special transactions and deposits. It returns a `PayloadAttributes`. This contains a preliminary payload.
2) Set the `NoTxPool` variable:
- We set `NoTxPool = true` If we include an empty block (still containing L1 information deposits). With this we exclude any transaction from the transaction pool.
- We set `NoTxPool = false` otherwise. In this case we start the `StartPayload` function where the *Engine Controller* will be responsible to trigger the **ForkchoiceUpdate**.

#### ForkchoiceUpdate
The `ForkchoiceUpdate` triggers the Execution Layer to generate the block. This is handled by the `Execution Client`.

#### Transaction Analysis and Malicious Payload identification

To analyze the transactions contained in the *payload* returned by the *Engine API* the simple and wrong strategy could be to edit the function `RunNextSequencerAction` in the following way:

func (d *Sequencer) RunNextSequencerAction(...) (...) {
	if onto, buildingID, safe := d.engine.BuildingPayload(); buildingID != (eth.PayloadID{}) || agossip.Get() != nil {
		if safe {
			...
		}
		envelope, err := d.CompleteBuildingBlock(ctx, agossip, sequencerConductor)
		if err != nil {
			...
		} else {
			payload := envelope.ExecutionPayload
			d.log.Info("sequencer successfully built a new block", "block", payload.ID(), "time", uint64(payload.Timestamp), "txs", len(payload.Transactions))
			
			payload, err := AnalyzeTransactions(payload)
			
			return payload, nil
		}
	} else {
		...
	}
}

where in `AnalyzeTransactions` we extrapolate the transactions information through `envelope.ExecutionPayload.Transactions`. This way, once we obtained from the *Engine API* the block, we can look into the block and analyze the transactions. But we cannot simply edit the block. Indeed, if we want to remove malicious transactions we need to act inside the *Execution Client*. Here the ad-hoc EVM can run the transactions and remove the dangerous ones with a custom error. So, in the payload received by the *Driver*, the malicious transaction won't be present. The *Driver* is agnostic to the whole process.

We could implement this by forking `op-geth`, `op-erigon` or `op-reth` (reth). Here is where the block building process occurs. The modular, contributor friendly aspect of [`op-reth`](https://github.com/paradigmxyz/reth) suggests that this would be the simpler one to approach.

Instead, in the remote possibility (this would be hard and not suggested) where we decide to edit the block post-receival from the *Execution Engine*, we could achieve this by making a request through a custom API.
The problem here is that the API would accomplish all the same functionalities of an *Execution Client*. Resulting in a very inefficient solution. We would need to replay the block transactions (already executed by the *Execution Client*) and keeping this other client in sync with the L1 and the rest of the Rollup Node.

Finally, for what concerns deposit transactions (and also special transactions), these are always assumed not to generate exploit, and are sent directly to the *Execution Client* and analyzed from there.

#### Execution Engine analysis: *Reth*

Reth [[11]](https://github.com/paradigmxyz/reth) is chosen because of its main characteristics:
- user-friendly
- highly modular
- fast and efficient

Specifically, the modularity aspect is particularly useful to analyze the result of the evm after a single transaction has been executed.

Reth is an *Execution Layer* and is compatible with all *Ethereum Consensus Layer* implementations that support the *Engine API*.

From the associated [docs](https://github.com/paradigmxyz/reth/blob/main/docs/repo/layout.md), we can see where the payload is generated.

Specifically, the payload building process is managed within the `payload/builder` directory. Next, let's examine where the EVM executes the transactions, focusing on where `evm.transact()` is called.

Inside `payload/optimism/src/builder.rs` module, in the `optimism_payload_builder` function the block gets created. The procedure (in the diagram below) is the following: We first extract through `best_transactions_with_attributes` the best transactions and store them in `best_txs`. Then, we iterate with a loop over all the transactions coming from the *Sequencer* (`attributes.transactions`), execute them and, if they don't generate any error, pushing them into `executed_txs`. If they generate errors we skip them (and depending on the error mark them as invalid) and continue the loop. Then we iterate with a new loop over the transactions from the Tx Pool (`best_txs`), these are the ones we want to analyze. After checking the value of `attributes.no_tx_pool` we repeat the same process of before until we fill the block. It's important to note that immediately after they are executed we know whether the tx generated or not an error. So we can avoid to push them into the newly created block. Here, there is a tentative approach to handle a potential malicious transaction:

pub(crate) fn optimism_payload_builder<EvmConfig, Pool, Client> (...) -> Result <..., ...>{
	...
	// this checks if the noTxPool is set to false, in this case we have to fill the block with txs from the tx pool 
	if !attributes.no_tx_pool {
		while let Some(pool_tx) = best_txs.next() {
			...
			let ResultAndState { result, state } = match evm.transact() {
				Ok(res) => res,
				Err(err) => {
					match err {
						EVMError::Transaction(err) => {
							if matches!(err, InvalidTransaction::NonceTooLow { .. }) {
								// if the nonce is too low, we can skip this transaction
								trace!(target: "payload_builder", %err, ?tx, "skipping nonce too low transaction");
							} else if matches!(err, InvalidTransaction::MaliciousPayload { .. }) {
								// Custom check for malicious transaction
								trace!(target: "payload_builder", %err, ?tx, "skipping malicious transaction");
								best_txs.mark_invalid(&pool_tx);
							else {
								// if the transaction is invalid, we can skip it and all of its
								// descendants
								trace!(target: "payload_builder", %err, ?tx, "skipping invalid transaction and its descendants");
								best_txs.mark_invalid(&pool_tx);
                            }
                            continue
                        }
                        err => {
                            // this is an error that we should treat as fatal for this attempt
                            return Err(PayloadBuilderError::EvmExecutionError(err))
                        }
                    }
				}
			}
		}
	}
}

Here is a simplified diagram illustrating what's happening in the code:


Here is what is happening from a high-level perspective:


---

### Bulding a sequencer that uses Magi as the rollup node

Regarding Magi, there is already a PR [#179](https://github.com/a16z/magi/pull/179/) implementing a sequencer in a single commit, for this reason I find it useful to document the main changes of the PR.

To correctly implement the *Sequencer* we first need to enable the possibility of using this component. This means setting `sequencer_enabled` in `magi.rs` as a boolean. Then, I wouldn't touch the derivation pipeline, that transforms the raw batched transactions to a well refined payload attribute. While `noTxPool` is currently hard-coded to `true` in `attributes.rs`, we must be able to choose to enable the sequencing functionality setting it to `bool`. `noTxPool` is crucial because it distinguishes between two different behaviours of the rollup node:
- Verifier
- Sequencer

By setting `noTxPool = false`, transactions from the *tx pool* are supposed to be pushed as a *payload* when calling the `ForkchoiceUpdated`, enforcing the *sequencer* functionality. To avoid appending the transactions we would edit the `derive_transactions` function in `stages/attributes.rs`. We would still need to append special transactions, like L1 info transaction, User deposit transactions, and other special transactions but we should avoid including:

let mut rest = input.transactions;
transactions.append(&mut rest);

these are the transactions of the L2 batches obtained from the L1.
For consistency with the PR and the *op-node*, we could refactor the `build_payload` function in `engine_driver.rs` into two separate functions for simplicity. The first function, `start_payload_building`, would be responsible for obtaining the payload ID, and the second function, `get_payload`, would retrieve the actual block, as illustrated in the sequence diagram above.

The final aspect to address is the propagation of the unsafe generated block across the p2p network. This task is more challenging and involves modifying the network in `network/service/mod.rs`.
This is how the PR addressed it:

Some(payload) = receiver_new_block.recv() => {
    if let Some(signer) = p2p_data.as_ref() {
        match encode_block_msg(payload, signer)
            .map_err(|err| err.to_string())
            .and_then(|tx|{
                swarm.behaviour_mut().gossipsub
                    .publish(self.block_topic.clone(),tx)
                    .map_err(|err| err.to_string())
            }){
                Ok(_message_id) => {},
                Err(err) => tracing::debug!("P2P block broadcast error: {err:?}"),
            };
    } else {
        tracing::warn!("missed signer p2p private key; skip payload broadcast");
    }
}

---

### Astria's Integration of the Sequencing Component into the OP Stack

Let's start first with a small intro on shared sequencers.

To date, sequencers have been implemented as a centralized service. In *Astria* rollups share a single decentralized network of sequencers permissionless to join. Avoiding the risk of transaction censorship.
As stated by the *Astria* [docs](https://docs.astria.org/docs/overview/architecture/the-astria-sequencer/): the *Astria Shared Sequencer* is a network of nodes utilizing *CometBFT* that come to consensus on an ordered set of transactions. The unique feature of the sequencer is that the transactions it includes are not executed (**lazy sequencing**), and are destined for another execution engine (ie. a rollup). This excludes “sequencer native” transactions, such as transfers of tokens within the sequencer chain. Transactions from any given rollup are only ordered on the sequencer, not executed.

A key differentiator here between decentralized sequencers and shared sequencers is that shared sequencers serve multiple blockchains rather than just one and shared sequencers are actually blockchain networks themselves [2]. An advantage of doing so is that they introduce interoperability advantages between rollups.

The main differences that a shared sequencer introduces can be summarized with the following diagrams, presented during this presentation [[6]](https://www.youtube.com/watch?v=Cj4k97SjbtY&ab_channel=ResearchDay):


Here there is an over-simplification of the Astria Architecture, including all its main components:


Having said this, the Optimism support was introduced with the PR [#775](https://github.com/astriaorg/astria/commit/fa8949dfc5d61a065176cee1ee41d09b358ae3b5) and then removed because would cause maintenance burden and because *"There is no astria deployment that uses optimism"*. But expected to be brought back later in time.

The steps to integrate the OP-stack are the following:
- create an optimism hook:

	async fn make_optimism_hook(cfg: &Config,) -> eyre::Result<Option<crate::executor::optimism::Handler>> {
		// Check if the Optimism network integration is enabled in the configuration.
		if !cfg.enable_optimism {
			return Ok(None);
		}

		// Connect to the Ethereum Layer 1 node using the URL specified in the configuration.
		let provider = Arc::new(
			Provider::<Ws>::connect(cfg.ethereum_l1_url.clone())
				.await
				.wrap_err("failed to connect to provider")?,
		);

		// Decode the hex string of the Optimism portal contract address from the configuration.
		let contract_address: Address = hex::decode(cfg.optimism_portal_contract_address.clone())
			.wrap_err("failed to decode contract address as hex")
			.and_then(|bytes| {
				TryInto::<[u8; 20]>::try_into(bytes)
					.map_err(|_| eyre::eyre!("contract address must be 20 bytes"))
			})
			.wrap_err("failed to parse contract address")?
			.into();

		// Return a new handler for the Optimism network
		Ok(Some(crate::executor::optimism::Handler::new(
			provider,
			contract_address,
			cfg.initial_ethereum_l1_block_height,
		)))
	}

- editing the configuration in `astria-conductor/src/config.rs`:

	/// Set to true to enable OP-Stack deposit derivation.
	pub enable_optimism: bool,

	/// Websocket URL of Ethereum L1 node.
	/// Only used if `enable_optimism` is true.
	pub ethereum_l1_url: String,

	/// Contract address of the OptimismPortal contract on L1.
	/// Only used if `enable_optimism` is true.
	pub optimism_portal_contract_address: String,

	/// The block height of the Ethereum L1 chain that the
	/// OptimismPortal contract was deployed at.
	/// Only used if `enable_optimism` is true.
	pub initial_ethereum_l1_block_height: u64,

- A new crate called `optimism.rs` inside `astria-conductor/src/executor`, whose functionality is to watch for OP-Stack L1 deposit events and convert them to deposit transactions to be executed on a rollup based on the OP-Stack. The way it works is by querying for deposit events from the [`OptimismPortal`] contract. When `populate_rollup_transactions` is called, it queries for all deposit events since the last call, converts them to deposit transactions, and prepends them to the other transactions for that rollup block.


- inside the create `astria-conductor/src/executor/mod.rs`, creating the `pre_execution_hook`, which is called to modify the rollup transaction list right before it's sent to the execution layer via `ExecuteBlock`.

	pre_execution_hook: Option<optimism::Handler>,

and then inside the `Executor`:

	if let Some(hook) = self.pre_execution_hook.as_mut() {
				transactions = hook
					.populate_rollup_transactions(transactions)
					.await
					.wrap_err("failed to populate rollup transactions with pre execution hook")?;
	}

#### Astria Architecture: a general overview and how it differs from the canonical op-stack

To fully understand how Astria differs from the canonical op-stack, an technical overview of the architecure is required.
There are 6 components in the architecture but we will focus mainly on 3 of them:
- Rollup
- **Composer**
- **Sequencer**
- **Conductor**
- Relayer
- Data Availability

The first component is the Composer:


Then we have the Sequencer:


And the Conductor:


From the point of view of the Architecture, both functions `process_proposal` and `prepare_proposal` execute the block. The main difference is whether we are acting as proposer or a validator for a new block.
The role of `prepare_proposal` is to propose a block based on the validator's view of the transaction pool and the state, choosing what transactions to include and what additional information (like state commitments) to append. The role of `process_proposal`, on the other hand, is about validating a block that another validator has proposed, deciding whether the block is valid.

### Espresso's Integration of the Sequencing Component into the OP Stack

Here is an overview of the Espresso architecture obtained from their docs [[10]](https://docs.espressosys.com/sequencer/espresso-sequencer-architecture/system-overview):


Here's a brief overview of the architecture components:

- An API Interface for clients to interact with (tipically Json-RPC) for state updates
- A **State Database** populated by the **Executor** a component which executes every block provided by the sequencer
- A **prover**
- The Rollup API can act as an endpoint for clients to submit transactions

Here is another version integrating the data flow more granularly in the process:


From a high-level perspective, the main difference with the canonical op-node:
- The use of a consensus algorithm (Hotshot)
- The shared sequencer is a completely external entity to the Rollup Node, and the Rollup Node act as a subscriber to the sequencer
- The Rollup contract now acts as a checker comparing the new state proof of the node, that is obtained as the result of the execution of the transactions obtained from the sequencer, against the QC.
- Depending on the requirements of our system through Espresso we can easily choose between different trade-offs of trust and latency


Concerning the specific op-stack Espresso integration:
- A modification of the L2 derivation pipeline to enforce that each L2 batch is determined by the Espresso Sequencer (using L1 info). *Justification* are indeed introduced to prove on the L1 that the Espresso Sequencer committed the transactions.
- While Espresso is responsive, the OP-Stack need to satisfy a fixed-rate L2 batch stream for what concerns derivation.


From a high-level overview of the system, The Espresso Sequencer looks similar to the canonical op-stack implementation. In the canonical op-node, the sequencer lets the Execution Client to run the tx and build the payload using also the transactions from the pool. Here, instead, the OP node fetches the already sequenced transactions from the Sequencer. We still have to address how the sequenced txs are checked and executed.

### Conclusions
In a setting where the sequencing task is provided by an external actor to the system, it becomes crucial to understand where the transactions get executed. While within the canonical op-node the Consensus Layer (Driver Engine) is responsible for block creation addressing the Execution Layer to running the tx for validation (and optionally other checks), in a shared sequencer setting deciding where the transaction gets executed is not trivial. We would want the execution as close as possible to the sequencing (in the sense of which transaction need to be added to the block) but at the same time we must address the censorship resistant requirements of the shared sequencer.

To some degree and from my understanding, it is sometime possible to check transactions without executing them. In the case where we want to fully execute them though, an Execution Layer will always be necessary, the closer to the block building process is, the faster we will have the outcome on whether it could be malicious or not.

---

[1] https://www.astria.org/blog/astria-the-shared-sequencer-network
[2] https://blog.jarrodwatts.com/the-ultimate-guide-to-sequencers-in-l2-blockchains
[3] https://docs.astria.org/docs/
[4] https://www.youtube.com/watch?v=vJSoiRDrsKM&ab_channel=0xResearch -> astria & espresso explanation
[5] https://www.youtube.com/watch?v=586oJd6CuCc&ab_channel=Celestia -> ETHDenver shared sequencer
[6] https://www.youtube.com/watch?v=Cj4k97SjbtY&ab_channel=ResearchDay -> Shared Sequencing: Speedrunning the Endgame - Josh Bowen
[7] https://hackmd.io/@astriaorg/HJ6cCpp9T -> Astria: The Shared Sequencer Network
[8] https://docs.espressosys.com/sequencer/releases/cortado-testnet-release/op-stack-integration -> Espresso OP-Stack Integration
[9] https://hackmd.io/@EspressoSystems/EspressoSequencer -> The Espresso Sequencer
[10] https://docs.espressosys.com/sequencer/espresso-sequencer-architecture/system-overview --> Espresso system overview
[11] https://github.com/paradigmxyz/reth --> Reth