# 24.7.0-RC1 Besu Release Performance Regression Analysis ## TL;DR No regression found compared to previous release, 24.6.0. Already known memory issue noted: https://github.com/hyperledger/besu/issues/5969 ## Sync Time ### 24.7.0-RC1 (started 07/07/2024) CHECKPOINT: * 21.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-01 * **20.5 hours** - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-02 * 22 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-cp-03 SNAP: * **28.5 hours** - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01 * 29 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-02 * 45.5 hours - dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-03 ### 24.6.0 (started 18/06/2024) CHECKPOINT: * **30 hours** - dev-elc-bu-tk-mainnet-simon-release-24.6.0-cp-02 * 35 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-cp-03 SNAP: * 35 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-01 * 31 hours - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02 * **30 hours** - dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-03 (I'm withholding judgement on a possible sync speed up until a test is run against 24.7.0 vs 24.6.0 with the nodes launched at the _same time_). ## Comparison - Last 3 hours dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01 vs dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02 ### Blocktime during sync #### 24.7.0-RC1 ![Screenshot 2024-07-09 at 4.47.28 PM](https://hackmd.io/_uploads/ByiFpUqvR.png) #### 24.6.0 ![Screenshot 2024-07-09 at 4.48.23 PM](https://hackmd.io/_uploads/SyUiaIqPA.png) ### Block processing and FCU timings ![Screenshot 2024-07-09 at 3.12.51 PM](https://hackmd.io/_uploads/S1veRI9PA.png) ### Preload caches ![Screenshot 2024-07-09 at 3.15.51 PM](https://hackmd.io/_uploads/BJEbAUqvR.png) ### Transaction Pool ![Screenshot 2024-07-09 at 3.16.06 PM](https://hackmd.io/_uploads/ryYzlvqPA.jpg) ### GC activity ![Screenshot 2024-07-09 at 3.15.11 PM](https://hackmd.io/_uploads/B1KSJv9P0.png) ### Number of threads ![Screenshot 2024-07-09 at 4.57.57 PM](https://hackmd.io/_uploads/By1cywcPC.png) ### CPU usage ![Screenshot 2024-07-09 at 3.17.49 PM](https://hackmd.io/_uploads/SyHm08qvA.png) ### Memory usage * [See below for further analysis](#Why-is-there-a-difference-in-memory-usage) ![Screenshot 2024-07-09 at 3.19.15 PM](https://hackmd.io/_uploads/SyFVC8qDR.png) ### Disk Usage ![Screenshot 2024-07-09 at 3.20.31 PM](https://hackmd.io/_uploads/BJ7SCLcwR.png) ### Network activity ![Screenshot 2024-07-09 at 3.21.26 PM](https://hackmd.io/_uploads/S1nBAIqP0.png) ### Disk IOPS MB/S #### 24.7.0-RC1 ![Screenshot 2024-07-09 at 3.23.30 PM](https://hackmd.io/_uploads/r1_dCUqv0.png) #### 24.6.0![Screenshot 2024-07-09 at 3.23.43 PM](https://hackmd.io/_uploads/ByfKCU9D0.png) ---- ## \*Why is there a difference in memory usage? Note that the comparison above was for the last 3 hours and the 24.6.0 nodes were launched at a different time, about 3 weeks earlier. In this time there has been a maintainence restart of the older ndoes. Zooming out to see compare all the nodes from launch date... ![Screenshot 2024-07-09 at 3.33.42 PM](https://hackmd.io/_uploads/Bk57MP5w0.png) The older nodes had a similar memory usage to the newer ones, until the maintenance window where the nodes were restarted: memory usage all dropped and stayed down. Comparing heapdumps from dev-elc-bu-tk-mainnet-simon-release-24.7.0-RC1-snap-01 and dev-elc-bu-tk-mainnet-simon-release-24.6.0-snap-02: ![Screenshot 2024-07-09 at 5.09.45 PM](https://hackmd.io/_uploads/B1AZXP9DR.jpg) It looks like this issue: https://github.com/hyperledger/besu/issues/5969