24.5.0 Besu Release Performance Regression

24.5.0-RC1 vs 23.3.0

Equivalent versions were used as exact versions weren't available:

24.5.0-RC1

24.5.0-RC1 is currently still burning in, but it is functionally equivalent to the test run on these nodes last week (24.4.0-RC3 + haltfix2)

Version string: besu/v24.5-develop-c5b97d2/linux-x86_64/openjdk-java-21

• commit sha is from https://github.com/hyperledger/besu/pull/7059

These nodes have been deleted now, but here is an historical Grafana link for the most recent two day time range following all nodes getting into sync: https://grafana.o11y.web3factory.consensys.net/d/_Mqt4ksnz/besu-full?orgId=19&var-system=dev-elc-bu-tk-mainnet-jframe-halt-1&var-system=dev-elc-bu-tk-mainnet-jframe-halt-10&var-system=dev-elc-bu-tk-mainnet-jframe-halt-2&var-system=dev-elc-bu-tk-mainnet-jframe-halt-3&var-system=dev-elc-bu-tk-mainnet-jframe-halt-4&var-system=dev-elc-bu-tk-mainnet-jframe-halt-5&var-system=dev-elc-bu-tk-mainnet-jframe-halt-6&var-system=dev-elc-bu-tk-mainnet-jframe-halt-7&var-system=dev-elc-bu-tk-mainnet-jframe-halt-8&var-system=dev-elc-bu-tk-mainnet-jframe-halt-9&var-sync_task_quantile=All&var-executor=All&var-vertx_pool_name=All&var-protocol_inbound=All&var-protocol_outbound=All&var-diskdevices=[a-z]%2B|nvme[0-9]%2Bn[0-9]%2B&var-overview_engine_percentiles=All&var-txpool_layer=All&from=1714867200000&to=1715039999000

24.3.0

I can't find any 24.3.3 candidates however the changes since 24.3.0 are minor: https://github.com/hyperledger/besu/compare/24.3.0…24.3.3
https://github.com/hyperledger/besu/releases/tag/24.3.3

24.3.0 Candidates:

• dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-1

• dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-2

  • version string besu/v24.2.0-dev-698ec420/linux-x86_64/openjdk-java-21
  • https://github.com/hyperledger/besu/releases/tag/24.2.0-RC4
  • === https://github.com/hyperledger/besu/releases/tag/24.3.0

• Same two day time range:
  https://grafana.o11y.web3factory.consensys.net/d/_Mqt4ksnz/besu-full?orgId=19&var-system=dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-1&var-system=dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-2&var-sync_task_quantile=All&var-executor=All&var-vertx_pool_name=All&var-protocol_inbound=All&var-protocol_outbound=All&var-diskdevices=[a-z]%2B|nvme[0-9]%2Bn[0-9]%2B&var-overview_engine_percentiles=All&var-txpool_layer=All&from=1714867200000&to=1715039999000

Another candidate is 24.2.0-RC3

• dev-elc-besu-teku-mainnet-simon-24.2.0-RC3-1
• dev-elc-besu-teku-mainnet-simon-24.2.0-RC3-2
• dev-elc-besu-teku-mainnet-simon-24.2.0-RC3-3
  • version string besu/v24.2.0-RC3/linux-x86_64/openjdk-java-21
  • https://github.com/hyperledger/besu/releases/tag/24.2.0-RC3
  • Only minor difference with 24.3.0
  • https://github.com/hyperledger/besu/compare/24.2.0-RC3…24.3.0

Both nodes are running Java 21 on Standard_D4s_v5 with Standard SSD LRS (500 Max IOPS, 100 MBps Max Throughput) and read-only host caching enabled.

24.5.0-RC1 Grafana Link for all nodes

24.3.3 Grafana Link for all nodes

Comparison

dev-elc-bu-tk-mainnet-jframe-halt-1 vs dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-1 Grafana Link

Sync time (CHECKPOINT)

24.5.0-RC1

• 40 hours - dev-elc-bu-tk-mainnet-jframe-halt-1
• 31 hours - dev-elc-bu-tk-mainnet-jframe-halt-2

24.3.0

• 33 hours - dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-1
• 29 hours - dev-elc-besu-teku-mainnet-dev-simon-RC4-teku-2

Blocktime

Block processing and FCU timings

Preload caches

Transaction Pool

GC activity

Number of threads

CPU usage

Memory usage

Disk Usage

Disk IOPS MB/S

24.5.0-RC1

24.3.3

Network activity