# Kaustinen analytics _Last updated: 2023-11-05_ The following is an automatically generated report from a modified geth node that captures metrics from the Kaustinen network. This modified geth node is planned to continue running in Kaustinen, so as the state grows and have bigger state we can keep track if the metrics makes sense considering the design. Once in a while I'll be updating this document with fresher reports. _Warn: take numbers with a grain of salt -- we're in early stages of Kaustinen and the geth-vkt version is underly heavy development. There might be bugs, optimizable code, etc. If something looks odd, ask!_ ## Tree stats report The analyzer tracks every 200 blocks: - `block_number`: the block number where these metrics where collected. - `depthX_count`: how many leaf nodes are at this particular depth. - `leaf_node_count`: how many leaf nodes are in the tree. - `internal_node_count`: how many non-leaf nodes are in the tree. - `keyvalue_count`: how many key-values are stored in the tree. The following are the stats for the latest registered block: | block_number | depth1_count | depth2_count | depth3_count | depth4_count | depth5_count | leaf_node_count | internal_node_count | keyvalue_count | |--------------|--------------|--------------|--------------|--------------|--------------|-----------------|---------------------|----------------| | 153200 | 0 | 1849 | 335473 | 7080 | 30 | 344432 | 67136 | 1377975 | Interpretation hints: - We could get an average leaf-node fill factor by computing `kevalue_count/leaf_node_count`. Currently, this value is ~5 which makes sense since this is the fill factor for accounts. - The distribution of leaf node depths should be coherent with some ~expected depth regarding `log_256(keyvalue_count)`, with reasonable outliers. ## Witness stats report For each witness in a block we register: - `block_numer`: the block number of the analyzed witness. - `gas_used`: gas used in the block. - `ssz_total_size`: the total bytes size of the SSZ encoded witness. - `ssz_statediff_size`: the total bytes of the SSZ encoded statediff part of the witness. - `ssz_verkleproof_size`: the total bytes of the SSZ encoded verkleproof par of the witness. - `statediff_stem_count`: how many stems are in the _statediff_. - `statediff_currentvalue_non-nil_count`: how many `currentValue` in the statediff aren't null. - `statediff_newvalue_non-nil_count`: how many `newValue` in the statediff aren't null. The following are the top 10 witnesses ordered by `ssz_total_size desc`: | block_number | gas_used | ssz_total_size | ssz_statediff_size | ssz_verkleproof_size | statediff_stem_count | statediff_currentvalue_nonnil_count | statediff_newvalue_nonnil_count | |--------------|----------|----------------|--------------------|----------------------|----------------------|-------------------------------------|---------------------------------| | 136133 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 136127 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132149 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132140 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132136 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132115 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132111 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 132092 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 131814 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | | 131782 | 3430000 | 41914 | 26960 | 14946 | 102 | 408 | 103 | Interpretation notes: - Note that `ssz_total_size ~= ssz_statediff_size + ssz_verkleproof_size`, the separation is done to know how many bytes from the total size come from the _statediff_ vs the _proof_. ## Proof verification report This report shows some experimental metrics of the e2e proof verification times: - `num_keys`: how many keys are proven in this proof. - `numopenings`: how many openings are part of this Multiproof. - `prestate_getelementsforproof_ms`: how many ms takes to collect proof elements from the reconstructed pre-state tree. - `verify_multiproof_ms`: how many ms takes to do the Multiproof verification. - `e2e_verif_time_ms`: how many ms takes for all witness verificatio work (deserializing, reconstructing tree, verifying Multiproof, etc). The following are the top 10 results ordered by `num_keys desc`: | num_keys | numopenings | prestate_getelementsforproof_ms | verify_multiproof_ms | e2e_verif_time_ms | |----------|-------------|---------------------------------|----------------------|-------------------| | 510 | 1620 | 14 | 8 | 36 | | 510 | 1620 | 13 | 9 | 35 | | 510 | 1620 | 14 | 8 | 33 | | 510 | 1620 | 13 | 9 | 32 | | 510 | 1620 | 14 | 7 | 35 | | 510 | 1620 | 13 | 10 | 34 | | 510 | 1620 | 14 | 9 | 35 | | 510 | 1620 | 13 | 8 | 33 | | 510 | 1620 | 14 | 11 | 35 | | 510 | 1620 | 19 | 8 | 41 | Interpretation notes: - The block number for each case isn't registered since it's mostly irrelevant for the metrics. What matters is `num_keys` and/or `numopenings`. ## Proof generation report This report shows some experimental metrics of the e2e proof generation times: - `num_keys`: how many keys are proven in this proof. - `numopenings`: how many openings are part of this Multiproof. - `prestate_getelementsforproof_ms`: how many ms takes to collect proof elements from the reconstructed pre-state tree. - `create_multiproof_ms`: how many ms takes to do the Multiproof proof generation. - `e2e_gen_time_ms`: total time to generate the proof to be inserted in the block. - `e2e_gen_time_ms`: how many ms takes to generate the witness (get elements from tree, generate Multiproof, serialize, etc). | num_keys | numopenings | prestate_getelementsforproof_ms | create_multiproof_ms | e2e_gen_time_ms | |----------|-------------|---------------------------------|----------------------|-----------------| | 510 | 1620 | 115 | 51 | 166 | | 510 | 1620 | 110 | 50 | 161 | | 510 | 1620 | 128 | 49 | 178 | | 510 | 1620 | 108 | 51 | 159 | | 510 | 1620 | 137 | 55 | 194 | | 510 | 1620 | 109 | 48 | 159 | | 510 | 1620 | 112 | 48 | 161 | | 510 | 1620 | 137 | 48 | 186 | | 510 | 1620 | 136 | 52 | 189 | | 510 | 1620 | 108 | 48 | 158 | Interpretation notes: - The block number for each case isn't registered since it's mostly irrelevant for the metrics. What matters is `num_keys` and/or `numopenings`.
Last changed by  
 
Ignacio Hagopian
github.com/jsign, @ignaciohagopian
239

Read more

Verkle Trees - Preimages generation & distribution strategy

block execution: 99.992976msINFO [07-15|13:10:28.917] Collected key values from base tree count=10000 duration=293.583654ms last account=008f65…483efbINFO [07-15|13:10:29.300] Prepared key values from base tree duration=382.488744msINFO [07-15|13:10:30.000] Inserted key values in overlay tree count=10000 duration=1.083027557smigration took: 1.376893246sfillLevels took 4.422121mscollecting nodes took 56.29µstoFrMultiple took 208.536µsupdating commitments took 3.090994mscollecting nodes took 5.472675mstoFrMultiple took 21.644539msupdating commitments took 52.035324mscollecting nodes took 2.3216mstoFrMultiple took 13.16166msupdating commitments took 54.440922mscollecting nodes took 127.455µstoFrMultiple took 639.024µsupdating commitments took 15.478009msFinalize: 211.469993msblock execution: 293.354953msINFO [07-15|13:10:30.971] Collected key values from base tree count=10000 duration=330.318923ms last account=008f65…483efbINFO [07-15|13:10:31.372] Prepared key values from base tree duration=400.638925msINFO [07-15|13:10:32.097] Inserted key values in overlay tree count=10000 duration=1.126096392smigration took: 1.456677516sfillLevels took 2.508844mscollecting nodes took 47.249µstoFrMultiple took 247.035µsupdating commitments took 4.936315mscollecting nodes took 7.428826mstoFrMultiple took 15.575423msupdating commitments took 50.524827mscollecting nodes took 2.413472mstoFrMultiple took 12.737298msupdating commitments took 41.038935mscollecting nodes took 139.996µstoFrMultiple took 648.065µsupdating commitments took 15.552383msFinalize: 193.08911msblock execution: 252.886918msINFO [07-15|13:10:32.963] Collected key values from base tree count=10000 duration=282.597556ms last account=00ab98…a45379INFO [07-15|13:10:33.531] Prepared key values from base tree duration=567.853577msINFO [07-15|13:10:33.900] Inserted key values in overlay tree count=10000 duration=937.057289msmigration took: 1.219926088sfillLevels took 7.629195mscollecting nodes took 136.204µstoFrMultiple took 414.446µsupdating commitments took 7.367287mscollecting nodes took 14.461581mstoFrMultiple took 18.437174msupdating commitments took 81.609759mscollecting nodes took 3.4579mstoFrMultiple took 19.02924msupdating commitments took 70.383834mscollecting nodes took 121.622µstoFrMultiple took 631.732µsupdating commitments took 15.461385msFinalize: 281.168431msblock execution: 239.645635msfillLevels took 269.784µscollecting nodes took 7.291µstoFrMultiple took 25.958µsupdating commitments took 641.648µscollecting nodes took 345.906µstoFrMultiple took 779.019µsupdating commitments took 5.172558mscollecting nodes took 198.328µstoFrMultiple took 601.982µsupdating commitments took 5.064353mscollecting nodes took 48.999µstoFrMultiple took 390.531µs

5/19/2024
Preimages slides

Why is this relevant for the Verkle conversion? We have to migrate all the data from MPT to VKT The keys in MPT are hashed We need the preimage of the hash to rekey it into the VKT Thus, we need all the preimages of the MPT tree Overlay Tree overview & timeline mental model image Current proposed strategy

5/16/2024
EOF + Verkle

EOF Verkle slides

4/22/2024
Verkle Trees - Another iteration of VKTs MSM

This document attempts to do another round of optimizations to the Verkle Tree related Multiscalar Multiplications (MSM).

3/21/2024
Read more from Ignacio Hagopian
Published on HackMD