HackMD24.7.0-RC1 Besu Release Performance Regression Analysis
TL;DR
No regression found compared to previous release, 24.6.0.
Already known memory issue noted: https://github.com/hyperledger/besu/issues/5969
Sync Time
24.7.0-RC1 (started 07/07/2024)
CHECKPOINT:
- 21.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-01
- 20.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-02
- 22 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-03
SNAP:
- 28.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01
- 29 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-02
- 45.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-03
24.6.0 (started 18/06/2024)
CHECKPOINT:
- 30 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-cp-02
- 35 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-cp-03
SNAP:
- 35 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-01
- 31 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02
- 30 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-03
(I'm withholding judgement on a possible sync speed up until a test is run against 24.7.0 vs 24.6.0 with the nodes launched at the same time).
Comparison - Last 3 hours
dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01
vs
dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02
Blocktime during sync
24.7.0-RC1
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
24.6.0
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Block processing and FCU timings
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Preload caches
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Transaction Pool
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
GC activity
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Number of threads
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
CPU usage
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Memory usage
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Disk Usage
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Network activity
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Disk IOPS MB/S
24.7.0-RC1
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
24.6.0
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
*Why is there a difference in memory usage?
Note that the comparison above was for the last 3 hours and the 24.6.0 nodes were launched at a different time, about 3 weeks earlier. In this time there has been a maintainence restart of the older ndoes.
Zooming out to see compare all the nodes from launch date…
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
The older nodes had a similar memory usage to the newer ones, until the maintenance window where the nodes were restarted: memory usage all dropped and stayed down.
Comparing heapdumps from dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01 and dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02:
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
It looks like this issue: https://github.com/hyperledger/besu/issues/5969
