Motivation I think it would be preferable to reduce this pressure so that we are no longer in a rush and that we do not put too much pressure on the block time processing. I don't think there is a perfect solution but I'm trying to explore some ideas and see what we can do . So ? What is the result if we not set the goal of having a quick state migration but rather choosing to migrate a small number of PMT leaves to Verkle slowly enough so that we do not require too recent machines and that we do not necessarily seek to do this migration quickly and impact block time processing. This can take some time without causing any problems ? Challenges and Solutions Managing Database Size and Bug Risks How can we ensure that this solution does not lead to an overly large database size and also an increased risk of bugs since we have to manage two states for a long period?

Bonsai operates with a flat database and a trie structure simultaneously. This combination allows for faster and more efficient SLOAD operations. In a traditional trie structure, the accounts and slots are located at the bottom of the trie (leaf). To find specific account or slot, you have to go through the entire trie. Each pass through a node of the trie equals a read in the database. The deeper the information is in the trie, the more reads are needed, which can slow down the operation. The introduction of a flat database structure alongside the trie in Bonsai presents a significant advantage. Instead of going through the entire trie, all the leaves (accounts and slots) are directly accessible. This means that you can find any information with a single read in the database, no matter its "depth" in the trie. This makes the search process much faster and more efficient. We have run a series of tests to compare the traditional trie structure with the new combined flat database and trie structure. The results, which we will present later, clearly show that this feature greatly improves performance. The tests show that the combined structure greatly reduces the time and resources needed for SLOAD operations, which allows for faster block processing. Why a new flag for bonsai ? When using snapsync or fastsync, keeping a valid flat database becomes tricky because we sync with many pivot blocks. This leads to our state being a mix of different blocks.

Integrating Verkle Trie with Bonsai: Proposed Approach In this guide, I'll describe the process of leveraging the flat database to limit read access to only one for SLOAD during the transition using the overlay method. Transitioning from PMT to Verkle Trie To facilitate the transition from Patricia Merkle Trie (PMT) to Verkle Trie, one approach that I consider most suitable and advantageous is the overlay method (https://notes.ethereum.org/@parithosh/verkle-transition). The overlay method starts by freezing the PMT at a specific block and initializing an empty Verkle Trie. We continue processing new blocks by adding modifications to the Verkle Trie instead of the PMT. The root hash of the Verkle Trie is then used as the root hash for block validation. Additionally, for each block, a range of leaf nodes (perhaps 10K leaves per block) is transitioned from the PMT to the Verkle Trie. The exact number can be determined later to balance transition time and client loss. It should be noted that after a certain period of time, we can safely delete the PMT trie from the database, which will help reclaim storage space. Once the risk of reorg has passed. During block processing, we need to read the account and storage state from the database. The initial idea for the transition is that when we want to read an account or a slot, we first try to read it in the Verkle tree. If we can't find it there, we try to read it in the Frozen PMT or Frozen Flat DB. This leads to a change in gas costs as we may have an additional database read (fallback mechanism).

Introduction Bonsai introduces several enhancements to the Besu client. Unlike other clients that store nodes of the world state by hash, Bonsai stores them by location in the trie. This approach enables natural pruning by replacing old nodes at the same position in the trie with the new ones. Unlike hash-based storage, which only adds new nodes to the database and keeps growing, Bonsai's technique ensures that there is only one version of the state at any given time. Consequently, it does not support managing reorganizations (reorgs) as it only maintains a single state. Bonsai addresses this limitation by introducing "trielogs" You can visit this link to see a diagram illustrating how the world state is structured. https://ethereum.stackexchange.com/a/6413 Components of Bonsai Trie The trie component of Bonsai stores nodes based on their location in the trie, enabling calculation of the root hash and block validation. By naturally pruning old nodes, the trie keeps the state representation compact and reduces storage requirements. Before Bonsai, the trie nodes was saved like that

In Besu, we often use C, Rust, or Go for the cryptographic part. This is mainly because these languages tend to offer better performance. When we want to add a new native library to Besu, it is relatively straightforward. We simply need to add it to the project. https://github.com/hyperledger/besu-native Integrating a library into Besu can be done by following these general steps: Step 1: Add the Go library for example this one

Introduction Bonsai introduces several enhancements to the Besu client. Unlike other clients that store nodes of the world state by hash, Bonsai stores them by location in the tree. This approach enables natural pruning by replacing old nodes at the same position in the tree with the new ones. Unlike hash-based storage, which only adds new nodes to the database and keeps growing, Bonsai's technique ensures that there is only one version of the state at any given time. Consequently, it does not support managing reorganizations (reorgs) as it only maintains a single state. Bonsai addresses this limitation by introducing "trielogs," Components of Bonsai Tree The tree component of Bonsai stores nodes based on their location in the tree, enabling calculation of the root hash and block validation. By naturally pruning old nodes, the tree keeps the state representation compact and reduces storage requirements. Flat Database The flat database is a storage optimization in Bonsai that stores all leaf nodes in a flat format. This design allows for quick and efficient access to accounts and storages, enhancing the performance of SLOAD operations. Instead of traversing the tree structure, the flat database enables direct access to the required data in a single disk read, resulting in faster query processing.