# Gomu Gomu No Gatling # Motivation There currently does not exist any standard ope way to benchmark Starknet RPCs both in terms of reads and writes. This leads to each RPC team doing their own in-house benchmarks and a lack of standard to measure performance on new sequencers' implementations such as Katana or Madara. Aptos paved the way for standardization in [this](https://aptosfoundation.org/currents/standardizing-verfiable-performance-benchmarks) article and Starknet should take inspiration from such good practices. In fact, it has historically been hard to correctly benchmark Starknet's reaction under heavy load and gomu gomu should help in that regard. # Global Scope - The tool can be used both as a CLI and a rust crate - Given a config, one can load-test an RPC - It should support both read and writes endpoints, although they should be treated very differently as they do not measure the same metrics. - Comes with working configurations for existing sequencers (devnet, katana, madara, public starknet). RPC [specification](https://github.com/starkware-libs/starknet-specs/tree/master) should also be taken into account. - Explore the use of existing load-testing crates such as [Goose](https://github.com/tag1consulting/goose) - Complete Documentation - Github Action - Full testing suite ## Read Most of the desired features have been inspired from [flood](https://github.com/paradigmxyz/flood), Paradigm's load testing tool for EVM RPCs. - Measure throughput, latency (mean, P50, P90, P95, P99, max) and error rate - Load results in polars frames for easy data analysis - Support multiple rates' increases (linear, constant) - Differential testing ## Write For proper testing of writes you need to setup an identical pre-state before each test, this makes it harder but not impossible as shown by [aptos](https://github.com/aptos-labs/aptos-multi-region-bench). - Make sure we have a stable, standard starting state to avoid biases - Pre-signing of transactions as signing shouldn't be part of the benchmark - Add starknet-related [metrics](https://starkscan.co/stats#speed) such as UOPS or SPS