Try   HackMD

Integrating block operations with RISC Zero (and some early benchmarks)

Date: 2025-04-11
By: unnawut

TLDR

Similar to @junsong 's work on SP1, I've integrated and benchmarked ReamLabs' implementation of beacon chain state transition functions within RISC Zero zkVM.

Similarly, I explored 2 questions:

  1. Can zkVM successfully run STFs (state transition functions)?
  2. How many cycles are needed for each operation?

Findings:

  • Beacon STFs: While SP1's implementation can handle 2^40-sized lists out of the box, RISC Zero failed at both large list instantiation and merkleization. Workarounds on the beacon state were needed to get the benchmarks working, but renders them non-conformant and under-speccing the beacon specs.
  • Compute cycles: I have no knowledge whether cycles of different zkVMs are comparable at all. But to compare them as they are, generally RISC Zero takes more cycles to read the pre-state, but takes less cycles to execute a transition function than SP1's implementation.
  • Early returns: Early returns e.g. negative test cases that return early have insignificant effect on the cycles taken nor the execution time, unlike SP1's benchmark.
  • Execution time: Most operations took generally 20 seconds to execute, except for SyncAggregate which took roughly 95 seconds. A fair result since process_sync_aggregate() traverses over a couple of large lists to verify the committee's signature aggregate, collect the committee's pubkeys, and apply rewards.
  • Total execution time: We are looking at ~69 seconds for block processing (see calculations) excluding the operations listed below. This number is calculated by summing the averages of read pre-state, merkleize and commit while summing the read operation and process time of each operation.
  • Benchmark skewing: Since the BeaconState workarounds were needed especially on extremely large lists like the validator registry, it is likely that the benchmark results are overly optimistic as the benchmark code does not traverse over or merkleize extremely large lists.

Excluded operations

STFs that are not part of the benchmark yet:

  • Parts of process_block():
    • process_execution_payload()
    • process_withdrawals()
    • process_randao()
    • process_eth1_data()
  • process_epoch() and its sub-operations
  • process_slot()

Code

See the full code at https://github.com/ReamLabs/consenzero-bench/pull/4

Details

Run benchmark

git clone https://github.com/ReamLabs/consenzero-bench.git
cd host
make download
make run-attestation

To run the benchmark for all operations, run make all instead of make run-attestation.

Next

Potential future work:

  • Benchmark the rest of excluded operations above
  • Find a fix for lists of size 2^40 within risc0 guest, figure out why it's different from SP1's behavior
  • Use risc0-provided precompiles such as risc0/zkcrypto-bls12_381
  • Explore why early returns do not result in decreased number of cycles and execution time

Appendix: Benchmark Results

Attestation

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
attestation at_max_inclusion_slot 46951041 9313 2068987 902134398 1013 951167972 20.945325334s
attestation correct_attestation_included_at_max_inclusion_slot 46949445 9313 2068816 902102375 1013 951134182 21.14637525s
attestation correct_attestation_included_at_min_inclusion_delay 46949445 9313 2068803 902102375 1013 951134169 21.326215s
attestation correct_attestation_included_at_one_epoch_delay 46949445 9313 2068816 902102375 1013 951134182 21.306923125s
attestation correct_attestation_included_at_sqrt_epoch_delay 46949445 9313 2068803 902102375 1013 951134169 21.182456625s
attestation incorrect_head_and_target_included_at_epoch_delay 46949445 9313 2068816 902102375 1013 951134182 21.26349425s
attestation incorrect_head_and_target_included_at_sqrt_epoch_delay 46949445 9313 2068803 902102375 1013 951134169 21.071597333s
attestation incorrect_head_and_target_min_inclusion_delay 46949445 9313 2068803 902102375 1013 951134169 21.049599208s
attestation incorrect_head_included_at_max_inclusion_slot 46949445 9313 2068816 902102375 1013 951134182 21.021552625s
attestation incorrect_head_included_at_min_inclusion_delay 46949445 9313 2068803 902102375 1013 951134169 21.249822375s
attestation incorrect_head_included_at_sqrt_epoch_delay 46949445 9313 2068803 902102375 1013 951134169 21.653023458s
attestation incorrect_target_included_at_epoch_delay 46949445 9313 2068816 902102375 1013 951134182 21.083295667s
attestation incorrect_target_included_at_min_inclusion_delay 46949445 9313 2068803 902102375 1013 951134169 21.147822042s
attestation incorrect_target_included_at_sqrt_epoch_delay 46949445 9313 2068803 902102375 1013 951134169 21.513312792s
attestation invalid_after_max_inclusion_slot 46951041 9313 2063693 902134398 1013 951162678 21.457018875s
attestation invalid_attestation_signature 46949445 9316 2068803 902102375 1013 951134172 21.20387575s
attestation invalid_bad_source_root 46949445 9313 2068803 902102375 1013 951134169 21.096296417s
attestation invalid_before_inclusion_delay 46949445 9313 2063526 902102375 1013 951128892 21.094976375s
attestation invalid_correct_attestation_included_after_max_inclusion_slot 46949445 9313 2063522 902102375 1013 951128888 21.92022625s
attestation invalid_current_source_root 46949445 9313 2068803 902102375 1013 951134169 21.021565417s
attestation invalid_empty_participants_seemingly_valid_sig 46949445 9317 2068803 902102375 1013 951134173 21.114277666s
attestation invalid_empty_participants_zeroes_sig 46949445 9320 2068803 902102375 1013 951134176 21.201474959s
attestation invalid_future_target_epoch 46949445 9313 2063522 902102375 1013 951128888 21.279273458s
attestation invalid_incorrect_head_and_target_included_after_max_inclusion_slot 46949445 9313 2063522 902102375 1013 951128888 21.706639208s
attestation invalid_incorrect_head_included_after_max_inclusion_slot 46949445 9313 2063522 902102375 1013 951128888 21.113195833s
attestation invalid_incorrect_target_included_after_max_inclusion_slot 46949445 9313 2063522 902102375 1013 951128888 21.397978667s
attestation invalid_index 46949445 9316 2063587 902102375 1013 951128956 20.917019042s
attestation invalid_mismatched_target_and_slot 46952599 9313 2063807 902166415 1013 951196367 20.940857459s
attestation invalid_new_source_epoch 46949445 9313 2068803 902102375 1013 951134169 21.077740667s
attestation invalid_old_source_epoch 46949445 9313 2063522 902102375 1013 951128888 21.186866542s
attestation invalid_old_target_epoch 46949445 9313 2063522 902102375 1013 951128888 21.09252075s
attestation invalid_previous_source_root 46949445 9313 2068816 902102375 1013 951134182 21.0044965s
attestation invalid_source_root_is_target_root 46949445 9313 2068803 902102375 1013 951134169 21.30968925s
attestation invalid_too_few_aggregation_bits 46949445 9137 2068803 902102375 1013 951133993 21.691313708s
attestation invalid_too_many_aggregation_bits 46949445 9314 2068803 902102375 1013 951134170 21.026694583s
attestation invalid_wrong_index_for_committee_signature 46949445 9316 2063587 902102375 1013 951128956 21.040332834s
attestation invalid_wrong_index_for_slot_0 46949445 9316 2063587 902102375 1013 951128956 21.642843292s
attestation invalid_wrong_index_for_slot_1 46949445 9316 2063587 902102375 1013 951128956 21.159867709s
attestation one_basic_attestation 46949445 9313 2068803 902102375 1013 951134169 20.866199792s
attestation previous_epoch 46951041 9313 2068987 902134398 1013 951167972 21.115847167s

AttesterSlashing

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
attester_slashing already_exited_long_ago 46949445 16969 2596007 901681374 1013 951248028 21.133734125s
attester_slashing already_exited_recent 46949445 16969 2596007 901681374 1013 951248028 21.173977167s
attester_slashing attestation_from_future 46949445 16969 2596007 901681374 1013 951248028 21.3420245s
attester_slashing basic_double 46949445 16969 2596007 901681374 1013 951248028 21.163904833s
attester_slashing basic_surround 46951041 16969 2595915 901713397 1013 951281555 21.19834375s
attester_slashing invalid_all_empty_indices 46949445 14119 2064115 902102375 1013 951134287 21.239223792s
attester_slashing invalid_att1_bad_extra_index 46949445 17143 2596019 901681374 1013 951248214 21.064507125s
attester_slashing invalid_att1_bad_replaced_index 46949445 16969 2596007 901681374 1013 951248028 21.032965416s
attester_slashing invalid_att1_duplicate_index_double_signed 46949445 16969 2064236 902102375 1013 951137258 21.658208958s
attester_slashing invalid_att1_duplicate_index_normal_signed 46949445 16969 2064224 902102375 1013 951137246 21.091638541s
attester_slashing invalid_att1_empty_indices 46949445 15544 2064121 902102375 1013 951135718 21.427454083s
attester_slashing invalid_att1_high_index 46949445 17143 2596019 901681374 1013 951248214 21.0830835s
attester_slashing invalid_att2_bad_extra_index 46949445 17143 2596007 901681374 1013 951248202 21.568851375s
attester_slashing invalid_att2_bad_replaced_index 46949445 16969 2596007 901681374 1013 951248028 21.269970667s
attester_slashing invalid_att2_duplicate_index_double_signed 46949445 16969 2596007 901681374 1013 951248028 21.214034167s
attester_slashing invalid_att2_duplicate_index_normal_signed 46949445 16969 2596007 901681374 1013 951248028 21.209318625s
attester_slashing invalid_att2_empty_indices 46949445 15544 2596001 901681374 1013 951246597 21.17934125s
attester_slashing invalid_att2_high_index 46949445 17143 2596007 901681374 1013 951248202 21.193059167s
attester_slashing invalid_incorrect_sig_1 46949445 16972 2596007 901681374 1013 951248031 21.637917125s
attester_slashing invalid_incorrect_sig_1_and_2 46949445 16975 2596007 901681374 1013 951248034 21.104282667s
attester_slashing invalid_incorrect_sig_2 46949445 16969 2596007 901681374 1013 951248028 21.260193667s
attester_slashing invalid_no_double_or_surround 46949445 16969 2064081 902102375 1013 951137103 21.474313166s
attester_slashing invalid_participants_already_slashed 46949453 16969 2596007 901681374 1013 951248036 21.5897305s
attester_slashing invalid_same_data 46949445 16969 2064305 902102375 1013 951137327 21.168564792s
attester_slashing invalid_unsorted_att_1 46949445 16969 2064230 902102375 1013 951137252 21.249876417s
attester_slashing invalid_unsorted_att_2 46949445 16969 2596007 901681374 1013 951248028 21.364061292s
attester_slashing low_balances 46949445 16969 2596007 901681374 1013 951248028 21.759899833s
attester_slashing misc_balances 46949445 16273 2595983 901681374 1013 951247308 21.214858791s
attester_slashing proposer_index_slashed 46951041 14533 2596094 901713397 1013 951279298 21.303822542s
attester_slashing with_effective_balance_disparity 46949445 16273 2595983 901681374 1013 951247308 21.511466625s

BlockHeader

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
block_header basic_block_header 46949445 46610 2073605 902102375 1013 951176268 22.271106416s
block_header invalid_multiple_blocks_single_slot 46949445 46610 2063903 902102375 1013 951166566 21.836859958s
block_header invalid_parent_root 46949445 46610 2073605 902102375 1013 951176268 21.789772791s
block_header invalid_proposer_index 46949445 46610 2073605 902102375 1013 951176268 21.598499417s
block_header invalid_proposer_slashed 46949446 46610 2073605 902102375 1013 951176269 21.677046792s
block_header invalid_slot_block_header 46949445 46610 2063898 902102375 1013 951166561 21.313484834s

BLSToExecutionChange

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
bls_to_execution_change genesis_fork_version 46949445 5785 2064249 902102375 1013 951126087 21.183957292s
bls_to_execution_change invalid_already_0x01 46949445 5785 2064316 902102375 1013 951126154 21.187268042s
bls_to_execution_change invalid_bad_signature 46949445 5785 2064249 902102375 1013 951126087 20.925975416s
bls_to_execution_change invalid_current_fork_version 46949445 5785 2064249 902102375 1013 951126087 21.070619792s
bls_to_execution_change invalid_genesis_validators_root 46949445 5785 2064249 902102375 1013 951126087 21.2142425s
bls_to_execution_change invalid_incorrect_from_bls_pubkey 46949445 5785 2064249 902102375 1013 951126087 21.205106875s
bls_to_execution_change invalid_previous_fork_version 46949445 5785 2064249 902102375 1013 951126087 21.479228542s
bls_to_execution_change invalid_val_index_out_of_range 46949445 5785 2064316 902102375 1013 951126154 21.191799708s
bls_to_execution_change success 46949445 5785 2064249 902102375 1013 951126087 21.108531833s
bls_to_execution_change success_exited 46949445 5785 2064249 902102375 1013 951126087 21.078447s
bls_to_execution_change success_in_activation_queue 46949445 5785 2064249 902102375 1013 951126087 21.165147917s
bls_to_execution_change success_in_exit_queue 46949445 5785 2064249 902102375 1013 951126087 21.285750458s
bls_to_execution_change success_not_activated 46949445 5785 2064249 902102375 1013 951126087 21.087250667s
bls_to_execution_change success_withdrawable 46949445 5785 2064249 902102375 1013 951126087 21.073324667s

Deposit

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
deposit correct_sig_but_forked_state 46949445 23395 225462389 901681424 1016 1174120942 27.159794125s
deposit effective_deposit_with_genesis_fork_version 46949445 23395 225462389 901681424 1016 1174120942 26.653638708s
deposit incorrect_sig_new_deposit 46949445 23398 144279843 901681424 1016 1092938399 24.833346625s
deposit incorrect_sig_top_up 46949445 23398 144279839 901681424 1016 1092938395 24.726540875s
deposit incorrect_withdrawal_credentials_top_up 46949445 23398 144279840 901681424 1016 1092938396 24.513754167s
deposit ineffective_deposit_with_bad_fork_version 46949445 23395 225460345 901681424 1016 1174118898 26.947006083s
deposit ineffective_deposit_with_current_fork_version 46949445 23395 225460340 901681424 1016 1174118893 27.058861333s
deposit ineffective_deposit_with_previous_fork_version 46949445 23395 225460341 901681424 1016 1174118894 26.257233333s
deposit invalid_bad_merkle_proof 46949445 23395 2884776 901681374 1013 951543223 21.901832917s
deposit invalid_wrong_deposit_for_deposit_count 46949445 23395 2884773 901681374 1013 951543220 21.710997834s
deposit key_validate_invalid_decompression 46949445 23395 42410899 901681374 1013 991069346 22.034511459s
deposit key_validate_invalid_subgroup 46949445 23398 42410639 901681374 1013 991069089 22.028026167s
deposit new_deposit_eth1_withdrawal_credentials 46949445 23395 225462400 901681424 1016 1174120953 26.832946125s
deposit new_deposit_max 46949445 23395 225462389 901681424 1016 1174120942 27.464695166s
deposit new_deposit_non_versioned_withdrawal_credentials 46949445 23395 225462393 901681424 1016 1174120946 26.280727083s
deposit new_deposit_over_max 46949445 23395 225462397 901681424 1016 1174120950 25.962010458s
deposit new_deposit_under_max 46949445 23395 225462416 901681424 1016 1174120969 27.043991875s
deposit success_top_up_to_withdrawn_validator 46952599 23395 225462672 901745464 1016 1174188419 27.2920465s
deposit top_up__less_effective_balance 46949445 23395 225462385 901681424 1016 1174120938 26.186673958s
deposit top_up__max_effective_balance 46949445 23395 225462385 901681424 1016 1174120938 26.24522075s
deposit top_up__zero_balance 46949445 23395 225462385 901681424 1016 1174120938 26.374407125s

ProposerSlashing

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
proposer_slashing basic 46949445 12061 2063517 902102375 1013 951131631 21.446020916s
proposer_slashing block_header_from_future 46949445 12061 2063517 902102375 1013 951131631 21.551525167s
proposer_slashing invalid_different_proposer_indices 46949445 12061 2063496 902102375 1013 951131610 21.361049709s
proposer_slashing invalid_headers_are_same_sigs_are_different 46949445 12061 2064217 902102375 1013 951132331 21.483360833s
proposer_slashing invalid_headers_are_same_sigs_are_same 46949445 12061 2064217 902102375 1013 951132331 21.616782584s
proposer_slashing invalid_incorrect_proposer_index 46949445 12061 2063517 902102375 1013 951131631 21.649334333s
proposer_slashing invalid_incorrect_sig_1 46949445 12064 2063517 902102375 1013 951131634 21.865930833s
proposer_slashing invalid_incorrect_sig_1_and_2 46949445 12067 2063517 902102375 1013 951131637 21.360261542s
proposer_slashing invalid_incorrect_sig_1_and_2_swap 46949445 12061 2063517 902102375 1013 951131631 21.879502917s
proposer_slashing invalid_incorrect_sig_2 46949445 12064 2063517 902102375 1013 951131634 21.735738875s
proposer_slashing invalid_proposer_is_not_activated 46949445 12061 2063517 902102375 1013 951131631 21.685065125s
proposer_slashing invalid_proposer_is_slashed 46949446 12061 2063517 902102375 1013 951131632 21.369735375s
proposer_slashing invalid_proposer_is_withdrawn 46949445 12061 2063517 902102375 1013 951131631 21.439356875s
proposer_slashing invalid_slots_of_different_epochs 46949445 12061 2063487 902102375 1013 951131601 21.549007s
proposer_slashing slashed_and_proposer_index_the_same 46949445 12061 2063517 902102375 1013 951131631 21.333808625s

SyncAggregate

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
sync_aggregate invalid_signature_bad_domain 46951041 9156 3347483065 902134451 1081 4296582135 96.5434905s
sync_aggregate invalid_signature_extra_participant 46951041 9156 3347483065 902134451 1081 4296582135 96.979919958s
sync_aggregate invalid_signature_infinite_signature_with_all_participants 46951041 9156 3346092017 902134451 1081 4295191087 96.424272041s
sync_aggregate invalid_signature_infinite_signature_with_single_participant 46951041 9412 231133188 902134448 1016 1180232378 26.81076975s
sync_aggregate invalid_signature_missing_participant 46951041 9156 3335291450 902134451 1016 4284390390 98.028480709s
sync_aggregate invalid_signature_no_participants 46951041 9415 143968275 902134448 1016 1093067468 24.192591042s
sync_aggregate invalid_signature_past_block 46954157 9156 3347483293 902208092 1081 4296659120 95.972678917s
sync_aggregate random_all_but_one_participating_with_duplicates 46951041 9156 3347484761 902134451 1081 4296583831 97.359798417s
sync_aggregate random_high_participation_with_duplicates 46951041 9168 3152419002 902134451 1016 4101517954 92.545889042s
sync_aggregate random_low_participation_with_duplicates 46951041 9332 1652850084 902134451 1016 2601949200 58.012332542s
sync_aggregate random_misc_balances_and_half_participation_with_duplicates 46951041 9172 3249951896 902134451 1016 4199050852 95.952540458s
sync_aggregate random_only_one_participant_with_duplicates 46951041 9412 238621753 902134448 1016 1187720943 27.037502167s
sync_aggregate sync_committee_rewards_duplicate_committee_full_participation 46951041 9156 3347484761 902134451 1081 4296583831 96.832150625s
sync_aggregate sync_committee_rewards_duplicate_committee_half_participation 46951041 9284 1786957800 902134451 1016 2736056868 61.767214667s
sync_aggregate sync_committee_rewards_duplicate_committee_max_effective_balance_only_participate_first_one 46951041 9412 232525958 902134448 1016 1181625148 26.618901125s
sync_aggregate sync_committee_rewards_duplicate_committee_max_effective_balance_only_participate_second_one 46951041 9412 232525958 902134448 1016 1181625148 27.039573541s
sync_aggregate sync_committee_rewards_duplicate_committee_no_participation 46951041 9412 2098142 902134398 1013 951197226 21.410172708s
sync_aggregate sync_committee_rewards_duplicate_committee_zero_balance_only_participate_first_one 46951041 9412 232525932 902134448 1016 1181625122 26.469489458s
sync_aggregate sync_committee_rewards_duplicate_committee_zero_balance_only_participate_second_one 46951041 9412 232525932 902134448 1016 1181625122 26.066832916s
sync_aggregate sync_committee_rewards_empty_participants 46951041 9412 2098142 902134398 1013 951197226 21.139747208s
sync_aggregate sync_committee_rewards_not_full_participants 46951041 9308 1829629323 902134451 1016 2778728415 62.228480708s
sync_aggregate sync_committee_with_nonparticipating_exited_member 46955715 9160 3335293506 902240109 1016 4284502782 96.100792166s
sync_aggregate sync_committee_with_nonparticipating_withdrawable_member 46955715 9160 3335293506 902240109 1016 4284502782 96.7755775s
sync_aggregate sync_committee_with_participating_exited_member 46955715 9156 3347485090 902240109 1081 4296694492 97.351291583s
sync_aggregate sync_committee_with_participating_withdrawable_member 46955715 9156 3347485090 902240109 1081 4296694492 95.366113541s

VoluntaryExit

Operation Test Case Read Pre-State Read Operation Process Merkleize Commit Total Cycles Execution Time
voluntary_exit basic 46949445 4077 2063266 902102375 1013 951123396 21.39809875s
voluntary_exit default_exit_epoch_subsequent_exit 46949445 4077 2063266 902102375 1013 951123396 21.105040541s
voluntary_exit invalid_incorrect_signature 46949445 4077 2063266 902102375 1013 951123396 21.084162208s
voluntary_exit invalid_validator_already_exited 46949445 4077 2063266 902102375 1013 951123396 21.06948825s
voluntary_exit invalid_validator_exit_in_future 46949445 4077 2063266 902102375 1013 951123396 21.1173925s
voluntary_exit invalid_validator_incorrect_validator_index 46949445 4077 2063266 902102375 1013 951123396 21.091872833s
voluntary_exit invalid_validator_not_active 46949445 4077 2063266 902102375 1013 951123396 21.162470458s
voluntary_exit invalid_validator_not_active_long_enough 46949445 4077 2063266 902102375 1013 951123396 21.406543292s
voluntary_exit invalid_voluntary_exit_with_current_fork_version_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.408246292s
voluntary_exit invalid_voluntary_exit_with_current_fork_version_not_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.195314042s
voluntary_exit invalid_voluntary_exit_with_genesis_fork_version_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.246155792s
voluntary_exit invalid_voluntary_exit_with_genesis_fork_version_not_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.30653525s
voluntary_exit success_exit_queue__min_churn 46949445 4077 2063266 902102375 1013 951123396 21.458069333s
voluntary_exit voluntary_exit_with_previous_fork_version_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.370850667s
voluntary_exit voluntary_exit_with_previous_fork_version_not_is_before_fork_epoch 46949445 4077 2063266 902102375 1013 951123396 21.327844s
Last changed by 
 
ReamLabs
0
1
199

Read more

Checkpoint Sync in Ream

Date: 2025-05-31By: Varun Doshi

Jun 2, 2025
Notes from Designing Partial BeaconState (with Multiproof)

Date: 2025-05-09By: Jun Song

May 19, 2025
Validator Management in Beam Clients

Beam chain aims to replace BLS with WOTS+ and this will require several changes to the consensus layer validator management executables.

May 5, 2025
Getting RISC Zero to work with 2^40 lists

This post is my attempt at getting RISC Zero to work with >32-bit lists, with a mix zkVM program optimizations along the way.

Apr 30, 2025
Read more from ReamLabs
Published on HackMD