Han (weihan) - Update 14, 15 & 16

Development Summary

The analysis on Ethereum state size and the key values analysis have been generated. Meanwhile, the code for conversion is completed and the conversion progress has been running. It will take approximately 2 days to finish the conversion analysis (assuming there are no bugs in the process).

Check out my development branch for Verkle Tree here.

Check out my development branch for Geth here.

Check out my local Verkle testnet setup here.

State Storage Analysis

Update on self-hosted node

There was a power outrage when I was running my node, and the database is corrupted, sigh 😔. I tried restarting the node, but got the following error messages:

INFO [11-01|07:57:08.387] Allocated trie memory caches             clean=614.00MiB dirty=1024.00MiB
INFO [11-01|07:57:08.621] Using pebble as the backing database
INFO [11-01|07:57:08.621] Allocated cache and file handles         database=/home/eth-main/execution/data-seed/node-pbss-2/geth/chaindata cache=2.00GiB handles=262,144
INFO [11-01|07:57:11.471] Opened ancient database                  database=/home/eth-main/execution/data-seed/node-pbss-2/geth/chaindata/ancient/chain readonly=false
INFO [11-01|07:57:11.477] Initialising Ethereum protocol           network=1 dbversion=8
WARN [11-01|07:57:11.477] Sanitizing invalid node buffer size      provided=1024.00MiB updated=256.00MiB
INFO [11-01|07:57:11.478] Failed to load journal, discard it       err="journal not found"
INFO [11-01|07:57:11.534] Opened ancient database                  database=/home/eth-main/execution/data-seed/node-pbss-2/geth/chaindata/ancient/state readonly=false
CRIT [11-01|07:57:11.607] Failed to truncate extra state histories err=EOF

This seems to be an ongoing issue on Geth (refer to this issue). Anyways, the node has synced up to block 14907811 (5th June 2022), which is 1M blocks less than expected.

Since the EPF day is around the corner, there isn't much time to debug and resync the node. Hence, I will just carry on with the analysis, though I would love to do it all over again after the EPF so that I can get the full analysis.

Key values analysis

The main reason why I did a full sync from genesis is to check the last block number that each key-value pair was accessed. Here's the output of my analysis:

Number of key-value pairs accessed from block 1 to block 14907811
+-------------------+---------------+---------------+--------------------+--------------------+------------------+
|    BLOCK RANGE    | ACCOUNT COUNT | STORAGE COUNT | ACCOUNT PERCENTAGE | STORAGE PERCENTAGE | RANGE PERCENTAGE |
+-------------------+---------------+---------------+--------------------+--------------------+------------------+
|     1-1051199     |     29133     |    528590     |       0.01%        |       0.06%        |      0.05%       |
|  1051200-2102399  |    355666     |    1045690    |       0.16%        |       0.12%        |      0.13%       |
|  2102400-3153599  |   19442427    |    984221     |       9.02%        |       0.12%        |      1.92%       |
|  3153600-4204799  |    3909482    |   10221100    |       1.81%        |       1.20%        |      1.33%       |
|  4204800-5255999  |   19535302    |   45281140    |       9.06%        |       5.33%        |      6.08%       |
|  5256000-6307199  |   12682407    |   78778712    |       5.88%        |       9.27%        |      8.58%       |
|  6307200-7358399  |   11209987    |   57899289    |       5.20%        |       6.81%        |      6.48%       |
|  7358400-8409599  |   13752072    |   53722555    |       6.38%        |       6.32%        |      6.33%       |
|  8409600-9460799  |   11168617    |   59364265    |       5.18%        |       6.98%        |      6.62%       |
| 9460800-10511999  |   17629223    |   81365754    |       8.18%        |       9.57%        |      9.29%       |
| 10512000-11563199 |   22344746    |   70720477    |       10.37%       |       8.32%        |      8.73%       |
| 11563200-12614399 |   30045584    |   74031128    |       13.94%       |       8.71%        |      9.77%       |
| 12614400-13665599 |   22575867    |   136646139   |       10.47%       |       16.07%       |      14.94%      |
| 13665600-14716799 |   23524144    |   150006706   |       10.91%       |       17.64%       |      16.28%      |
| 14716800-14907811 |    7360654    |   29627335    |       3.41%        |       3.48%        |      3.47%       |
+-------------------+---------------+---------------+--------------------+--------------------+------------------+
|       TOTAL       |   215565311   |   850223101   |      100.00%       |      100.00%       |     100.00%      |
+-------------------+---------------+---------------+--------------------+--------------------+------------------+
Total account KV count from genesis block to the latest block: 215565311 (20.23%)
Total storage KV count from genesis block to the latest block: 850223101 (79.77%)
Total KV accessed from block 1 to block 14907811: 1065788412 (100.00%)

The analysis is shown using a block range of 1051200, which is about 6 months (given a block time of 15s). There are a total of 1.07B key-value pairs up until block 14907811. The most important data point is that only about 19.75% of the total key-value pairs have been accessed in the past 1 year, which corresponds to about 211M key-value pairs. In other words, 80.25% of the total state storage have been redundant, and regular full nodes have been storing them for nothing, for free.

Conversion progress

I got a preimage file from Guillaume, and thanks to that, I can do the conversion from MPT to VKT and analyze the total storage saved with my state expiry scheme. The analysis process looks something like this:

1. Import preimages to the local chaindata database
2. Run offline conversion from MPT to VKT (no expiry)
3. Inspect database
4. Run offline conversion from MPT to VKT (with expiry)
5. Inspect database
6. Compare the inspect results

For step 1, once I have obtained the preimage file, I can execute the following geth command to import the preimages into my node's database:

./geth_verkle db import --datadir node preimages.bin

For step 2, there is an existing command that can do a offline conversion to VKT by using the underlying snapshot. I've modified the inner workings to include state expiry logics and cater to my new VKT format. The command is as follows:

./geth_verkle verkle to-verkle --datadir node --stateexpiry true

For step 3, there is an existing tool that allow users to inspect the database. Here's an example of the output:

+-----------------+-------------------------+------------+-----------+
|    DATABASE     |        CATEGORY         |    SIZE    |   ITEMS   |
+-----------------+-------------------------+------------+-----------+
| Key-Value store | Headers                 | 49.86 MiB  |     90001 |
| Key-Value store | Bodies                  | 8.96 GiB   |     90001 |
| Key-Value store | Receipt lists           | 4.93 GiB   |     90001 |
| Key-Value store | Difficulties            | 4.55 MiB   |     90001 |
| Key-Value store | Block number->hash      | 3.60 MiB   |     90001 |
| Key-Value store | Block hash->number      | 582.91 MiB |  14907812 |
| Key-Value store | Transaction index       | 14.72 GiB  | 439137644 |
| Key-Value store | Bloombit index          | 2.71 GiB   |   7456312 |
| Key-Value store | Contract codes          | 3.43 GiB   |    600037 |
| Key-Value store | Hash trie nodes         | 0.00 B     |         0 |
| Key-Value store | Path trie state lookups | 3.52 MiB   |     90001 |
| Key-Value store | Path trie account nodes | 27.16 GiB  | 232886629 |
| Key-Value store | Path trie storage nodes | 95.53 GiB  | 959044684 |
| Key-Value store | Trie preimages          | 21.37 GiB  | 318962126 |
| Key-Value store | Account snapshot        | 7.94 GiB   | 174286121 |
| Key-Value store | Storage snapshot        | 50.54 GiB  | 709545865 |
| Key-Value store | Account snapshot meta   | 8.23 GiB   | 215565311 |
| Key-Value store | Storage snapshot meta   | 57.80 GiB  | 850223101 |
| Key-Value store | Beacon sync headers     | 1.86 GiB   |   3585187 |
| Key-Value store | Clique snapshots        | 0.00 B     |         0 |
| Key-Value store | Singleton metadata      | 7.83 MiB   |        15 |
| Light client    | CHT trie nodes          | 0.00 B     |         0 |
| Light client    | Bloom trie nodes        | 0.00 B     |         0 |
+-----------------+-------------------------+------------+-----------+
|                            TOTAL          | 305.83 GIB |           |
+-----------------+-------------------------+------------+-----------+

The command is as follows:

./geth db inspect --datadir node

Next week's Action Items

• Complete the analysis on VKT conversion
• Complete the final development update
• Prepare for EPF day presentation