Flaky dispute past session test

# Flaky dispute past session test ## Observations from local reproduction When running this test locally (requires `--features fast-runtime`) using Zombienet on https://github.com/paritytech/polkadot/pull/7621 + some additional test conditions, 2 scenarios were observed: 1. Alice panics with `Scraped backing votes had invalid signature!` and hence finality stalls (we also stop malus node, 2/4 are online). This is explained by revertion happening to more than a session ago (https://github.com/paritytech/polkadot/issues/7233) caused by slowness of the machine. (There is a sleep 120s [= 1 session duration] in the test) 2. The test concludes "successfully", but finality stalls. I am not quite sure why ``` 3:01:49 PM │ ✅ alice: reports polkadot_parachain_approval_checking_finality_lag is 0 (1ms) ``` this metric succeeds in Zombienet, because clearly approval finality lag is NOT ZERO on alice: ``` 2023-08-16 15:01:08.275 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 81-[11 1111111111 1111111111 1111111111 1111111111 1111111111 11111110]-22 2023-08-16 15:01:08.276 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=60 ``` Nevertheless, we can see that the problem is that alice is lacking approval votes from honest validators and they don't distribute it, because this block (22) is approved in their view, they don't have problems with approval checking lag and hence don't enable aggression. This happens because we have 4 nodes in total, alice is paused first, but then we pause malus node. When alice wakes up, it doesn't see aproval votes from honest validators, but they already approved this block from their local standpoint. 2/4 votes are not enough to approve block 22. ### Suggested mitigation for scenario 2 This scenario when more than 1/3 of nodes are offline at the same time is unlikely in practice, albeit possible. Maybe we should adjust the slashing zombienet test to avoid this pitfall. Alternatively, we could enable aggression in approval-distrution based on general finality lag and not only (local) approval checking lag. ### Scenario 2 logs #### grandpa RPC ``` { setId: 0 best: { round: 76 totalWeight: 4 thresholdWeight: 3 prevotes: { currentWeight: 3 missing: [ 5CbeLW9cpsDUEQg367PAhfHzMAmkkss1pxCxaB1wobPHZFFs ] } precommits: { currentWeight: 0 missing: [ 5CbeLW9cpsDUEQg367PAhfHzMAmkkss1pxCxaB1wobPHZFFs 5EYjcEXFbrLTRb7ZnuKG7KrZrZ7DWhV2QhBBE4KoiEZ4dhk1 5FA9nQDVg267DEd8m1ZypXLBnvN7SFxYwV7ndqSYGiN9TTpu 5HELmVm29J2AXCvfheWW4LmXzpAACDX5HXCMdB5MkFckFHiW ] } } background: [] } ``` #### honest-validator-1.log ``` 2023-08-16 14:50:42.061 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: 🎁 Prepared block for proposing at 22 (4 ms) [hash: 0x39faaeebdb2da25ee16ddcee5ff7758ee8d2b954fe87435e2898c3a04ba42901; parent_hash: 0xf2ce…6329; extrinsics (2): [0xd707…fe0d, 0x720a…c30e] 2023-08-16 14:50:42.069 INFO tokio-runtime-worker babe: 🔖 Pre-sealed block for proposal at 22. Hash now 0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0, previously 0x39faaeebdb2da25ee16ddcee5ff7758ee8d2b954fe87435e2898c3a04ba42901. 2023-08-16 14:50:42.070 INFO tokio-runtime-worker substrate: ✨ Imported #22 (0x6350…dad0) 2023-08-16 14:50:42.079 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activation of leaf 0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0 (22) 2023-08-16 14:50:42.087 DEBUG tokio-runtime-worker parachain::availability-store: Candidate included candidate_hash=0x61110f45cfe69ba9bdb41f9966734bbc2ff61f2f53015a005e1b4fe45f802322 traceID=129023694404961385070153929148912061372 2023-08-16 14:50:42.103 DEBUG tokio-runtime-worker parachain::approval-voting: Imported new block. block_number=22 block_hash=0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0 num_candidates=1 2023-08-16 14:50:42.103 DEBUG tokio-runtime-worker parachain::approval-distribution: Got new blocks [(0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0, 22)] ``` `<snip>` ``` 2023-08-16 14:51:12.118 INFO tokio-runtime-worker substrate: ✨ Imported #26 (0xb184…427d) 2023-08-16 14:51:12.121 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activation of leaf 0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d (26) 2023-08-16 14:51:12.143 DEBUG tokio-runtime-worker parachain::availability-store: Candidate included candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:12.159 DEBUG tokio-runtime-worker parachain::approval-voting: Imported new block. block_number=26 block_hash=0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d num_candidates=1 2023-08-16 14:51:12.159 DEBUG tokio-runtime-worker parachain::approval-distribution: Got new blocks [(0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d, 26)] 2023-08-16 14:51:12.159 DEBUG tokio-runtime-worker parachain::approval-distribution: Processing pending assignment/approvals num=1 2023-08-16 14:51:12.277 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 26-[01010]-22 2023-08-16 14:51:12.287 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=5 2023-08-16 14:51:13.632 DEBUG tokio-runtime-worker parachain::bitfield-signing: Signing Bitfield for 4 cores: [0, 0, 0, 0] relay_parent=0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d 2023-08-16 14:51:14.647 INFO tokio-runtime-worker substrate: 💤 Idle (5 peers), best: #26 (0xb184…427d), finalized #21 (0xf2ce…6329), ⬇ 2.3kiB/s ⬆ 2.3kiB/s 2023-08-16 14:51:14.670 DEBUG tokio-runtime-worker parachain::chain-selection: Prepared 0 stagnant entries for pruning up_to=1692100274 min_ts=0 max_ts=0 2023-08-16 14:51:14.919 INFO tokio-runtime-worker parachain::chain-selection: Unfinalized block reverted due to a bad parachain block. maybe_revert_hash=Some(0xa79889579877b43aa95efc4def58d91f73b02fa51da66d498188187dfa4c3e45) revert_target=24 maybe_reporting_hash=None maybe_reporting_number=None 2023-08-16 14:51:15.066 INFO tokio-runtime-worker parachain::dispute-coordinator: New dispute initiated for candidate. candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 session=1 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:15.084 DEBUG tokio-runtime-worker parachain::availability-recovery: Requesting availability chunks for a candidate candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 num_requests=1 error_count=0 total_received=0 threshold=2 already_requesting_count=0 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:15.090 DEBUG tokio-runtime-worker parachain::availability-recovery: Can conclude availability for a candidate candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 received_chunks_count=2 requested_chunks_count=0 threshold=2 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:15.099 DEBUG tokio-runtime-worker parachain::candidate-validation: About to validate a candidate. validation_code_hash=0xfcd42fa74d7ae08665c2e6e35006490d077fba8c03e781a5b6e4d7870c5f876b para_id=Id(1000) 2023-08-16 14:51:15.132 DEBUG tokio-runtime-worker parachain::candidate-validation: Acquired executor params for the session: ExecutorParams([]) validation_code_hash=0xfcd42fa74d7ae08665c2e6e35006490d077fba8c03e781a5b6e4d7870c5f876b para_id=Id(1000) [2mAug 16 12:51:15.193[0m [31mERROR[0m runtime: panicked at 'Invalid parachain block data: Error', /path/to/cumulus/pallets/parachain-system/src/validate_block/implementation.rs:92:10 2023-08-16 14:51:15.145 DEBUG tokio-runtime-worker parachain::pvf: enqueueing an artifact for execution validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:15.145 DEBUG tokio-runtime-worker parachain::pvf: assigning the execute worker validation_code_hash=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(1v1) 2023-08-16 14:51:15.145 DEBUG tokio-runtime-worker parachain::pvf: starting execute for /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-1/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=95674 validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:15.196 WARN tokio-runtime-worker parachain::pvf: execution worker concluded, error occurred: InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) artifact_id=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(1v1) worker_rip=false 2023-08-16 14:51:15.196 INFO tokio-runtime-worker parachain::candidate-validation: Failed to validate candidate para_id=Id(1000) error=InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) 2023-08-16 14:51:15.196 WARN tokio-runtime-worker parachain::dispute-coordinator: Candidate 0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 considered invalid: ExecutionError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })") 2023-08-16 14:51:15.228 INFO tokio-runtime-worker parachain::chain-selection: Unfinalized block reverted due to a bad parachain block. maybe_revert_hash=Some(0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d) revert_target=26 maybe_reporting_hash=None maybe_reporting_number=None 2023-08-16 14:51:15.401 DEBUG tokio-runtime-worker parachain::dispute-distribution: Not importing empty batch candidate_hash=0x22248dd3133074ef283fc029696cd4c11c67031259b0f57265c40003e69466b8 traceID=45383551086429594700792699688317670593 2023-08-16 14:51:15.547 DEBUG tokio-runtime-worker parachain::dispute-distribution: Not importing empty batch candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:16.327 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 23-[11]-22 2023-08-16 14:51:16.332 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=0 2023-08-16 14:51:18.025 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: 🙌 Starting consensus session on top of parent 0x4a36bf641cb78dc7c34984f60121bf7f501754185e07763ae30a4a07e94cb36c 2023-08-16 14:51:19.651 INFO tokio-runtime-worker substrate: 💤 Idle (5 peers), best: #26 (0xb184…427d), finalized #21 (0xf2ce…6329), ⬇ 0.8kiB/s ⬆ 1.5kiB/s ``` #### alice ``` 2023-08-16 14:48:48.047 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: 🎁 Prepared block for proposing at 7 (2 ms) [hash: 0x74163810d2b9bf58396a212a33866dab5c7fefce26c6545c5dbb4b4a21570788; parent_hash: 0x1c55…fc2e; extrinsics (2): [0x8bfb…afdc, 0xb329…7eb5] 2023-08-16 14:48:48.054 INFO tokio-runtime-worker babe: 🔖 Pre-sealed block for proposal at 7. Hash now 0x75a6c0ea2d3c271fe8c571179a2e50c6ec8577636b489f89056499aad6c0974f, previously 0x74163810d2b9bf58396a212a33866dab5c7fefce26c6545c5dbb4b4a21570788. 2023-08-16 14:48:48.056 INFO tokio-runtime-worker substrate: ✨ Imported #7 (0x75a6…974f) 2023-08-16 14:48:48.063 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activati ``` `<stopped>` `<resumed>` ``` 2023-08-16 14:53:20.566 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: 🎁 Prepared block for proposing at 17 (20 ms) [hash: 0xdb8dd26db1206cf03975c4f2025dd094bd594d5b7f79b4b19ef711648eb31f49; parent_hash: 0x151b…1df6; extrinsics (2): [0xfb0f…db3c, 0xc1a1…01d8] 2023-08-16 14:53:20.587 INFO tokio-runtime-worker babe: 🔖 Pre-sealed block for proposal at 17. Hash now 0x816e9f888d22d945962eecdfc7e32189770a761fc248591532a43fcc1dd32efd, previously 0xdb8dd26db1206cf03975c4f2025dd094bd594d5b7f79b4b19ef711648eb31f49. 2023-08-16 14:53:20.587 WARN tokio-runtime-worker babe: 👶 Epoch(s) skipped: from 1 to 2 2023-08-16 14:53:20.587 INFO tokio-runtime-worker babe: 👶 New epoch 2 launching at block 0x816e…2efd (block slot 282031733 >= start slot 282031722). 2023-08-16 14:53:20.587 INFO tokio-runtime-worker babe: 👶 Next epoch starts at slot 282031742 2023-08-16 14:53:20.590 INFO tokio-runtime-worker substrate: ✨ Imported #17 (0x816e…2efd) 2023-08-16 14:53:20.590 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activation of leaf 0x816e9f888d22d945962eecdfc7e32189770a761fc248591532a43fcc1dd32efd (17) 2023-08-16 14:53:20.594 DEBUG tokio-runtime-worker parachain::approval-voting: Insta-approving all candidates block_hash=0x816e9f888d22d945962eecdfc7e32189770a761fc248591532a43fcc1dd32efd 2023-08-16 14:53:20.595 DEBUG tokio-runtime-worker parachain::approval-voting: Imported new block. block_number=17 block_hash=0x816e9f888d22d945962eecdfc7e32189770a761fc248591532a43fcc1dd32efd num_candidates=0 2023-08-16 14:53:20.595 DEBUG tokio-runtime-worker parachain::approval-distribution: Got new blocks [(0x816e9f888d22d945962eecdfc7e32189770a761fc248591532a43fcc1dd32efd, 17)] 2023-08-16 14:53:20.614 DEBUG tokio-runtime-worker parachain::approval-voting: Block finalized block_hash=0xf2ce79b81cf7e2ab9c09e8b571dc6d19be72722bb38a952aaf1b6fe043986329 block_number=21 2023-08-16 14:53:20.617 DEBUG tokio-runtime-worker parachain::dispute-distribution: Dispute coordinator slow? We are still waiting for data on next active leaves update. 2023-08-16 14:53:20.696 DEBUG tokio-runtime-worker parachain::network-bridge-rx: action="PeerDisconnected" peer_set=Validation peer=PeerId("12D3KooWCS9asDXemmFMSVrBGANQWCYfCynbTvYh1wTKsm64EeQ9") 2023-08-16 14:53:21.627 DEBUG tokio-runtime-worker parachain::chain-selection: Prepared 0 stagnant entries for pruning up_to=1692100401 min_ts=0 max_ts=0 2023-08-16 14:53:21.680 INFO tokio-runtime-worker substrate: 💤 Idle (4 peers), best: #39 (0xd025…74ca), finalized #21 (0xf2ce…6329), ⬇ 46.1kiB/s ⬆ 16.1kiB/s 2023-08-16 14:53:23.637 DEBUG tokio-runtime-worker parachain::approval-voting: Block 0x6350…dad0 is 17 blocks deep and has 1/1 candidates unapproved 2023-08-16 14:53:23.637 DEBUG tokio-runtime-worker parachain::approval-voting: assigned. candidate_hash=0x61110f45cfe69ba9bdb41f9966734bbc2ff61f2f53015a005e1b4fe45f802322 block_hash=0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0 tranche=16 next_wakeup=Some(3384380821) status=1/0/4 triggered=true approved=false traceID=129023694404961385070153929148912061372 2023-08-16 14:53:23.637 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 39-[1111111111 11111110]-22 2023-08-16 14:53:23.640 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=18 2023-08-16 14:53:24.155 INFO tokio-runtime-worker substrate: ✨ Imported #40 (0x6a74…9ffb) 2023-08-16 14:53:24.156 DEBUG offchain-worker runtime::election-provider: [#40] 🗳 resubmit offchain thread output: Err(MinerError::Lock("recently executed.")) 2023-08-16 14:53:24.165 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activation of leaf 0x6a74161c55505be4228151b382947c1f952504d35fc0714f30b62ca753249ffb (40) 2023-08-16 14:53:24.181 DEBUG tokio-runtime-worker parachain::availability-store: Candidate backed candidate_hash=0xc2e04d51c8d5a50df24d5c76ebfae32481b11581cd2a54ce2ff4c7510c89e300 traceID=259034873903761408177747379092614472484 2023-08-16 14:53:24.195 DEBUG tokio-runtime-worker parachain::availability-store: Stored chunk index for candidate. candidate_hash=0xc2e04d51c8d5a50df24d5c76ebfae32481b11581cd2a54ce2ff4c7510c89e300 chunk_index=0 traceID=259034873903761408177747379092614472484 2023-08-16 14:53:24.197 INFO tokio-runtime-worker parachain::dispute-coordinator: Processing unapplied validator slashes session_index=1 candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 n_slashes=1 traceID=339732803593143142233024039371907190662 2023-08-16 14:53:24.197 INFO tokio-runtime-worker parachain::dispute-coordinator: Couldn't find inclusion parent for an unapplied slash 2023-08-16 14:53:24.200 DEBUG tokio-runtime-worker parachain::approval-distribution: Got new blocks [(0x6a74161c55505be4228151b382947c1f952504d35fc0714f30b62ca753249ffb, 40)] 2023-08-16 14:53:24.200 DEBUG tokio-runtime-worker parachain::approval-distribution: Aggression enabled min_age=22 max_age=40 2023-08-16 14:53:24.200 DEBUG tokio-runtime-worker parachain::approval-distribution: Encountered old block pending gossip topology lag=18 2023-08-16 14:53:24.200 DEBUG tokio-runtime-worker parachain::approval-voting: Imported new block. block_number=40 block_hash=0x6a74161c55505be4228151b382947c1f952504d35fc0714f30b62ca753249ffb num_candidates=0 2023-08-16 14:53:25.670 DEBUG tokio-runtime-worker parachain::bitfield-signing: Signing Bitfield for 4 cores: [1, 0, 0, 0] relay_parent=0x6a74161c55505be4228151b382947c1f952504d35fc0714f30b62ca753249ffb 2023-08-16 14:53:26.628 DEBUG tokio-runtime-worker parachain::chain-selection: Prepared 0 stagnant entries for pruning up_to=1692100406 min_ts=0 max_ts=0 2023-08-16 14:53:26.683 INFO tokio-runtime-worker substrate: 💤 Idle (4 peers), best: #40 (0x6a74…9ffb), finalized #21 (0xf2ce…6329), ⬇ 1.9kiB/s ⬆ 0.8kiB/s 2023-08-16 14:53:27.641 DEBUG tokio-runtime-worker parachain::approval-voting: Block 0x6350…dad0 is 18 blocks deep and has 1/1 candidates unapproved 2023-08-16 14:53:27.641 DEBUG tokio-runtime-worker parachain::approval-voting: assigned. candidate_hash=0x61110f45cfe69ba9bdb41f9966734bbc2ff61f2f53015a005e1b4fe45f802322 block_hash=0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0 tranche=16 next_wakeup=Some(3384380821) status=1/0/4 triggered=true approved=false traceID=129023694404961385070153929148912061372 2023-08-16 14:53:27.641 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 40-[1 1111111111 11111110]-22 2023-08-16 14:53:27.641 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=19 ``` #### honest-validator-0.log ``` 2023-08-16 14:51:12.091 INFO tokio-runtime-worker babe: 🔖 Pre-sealed block for proposal at 26. Hash now 0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d, previously 0x470e4e5d78e42f50adc8bcff9c3f4bce60a41aaeec646d8bc178751a529483ea. 2023-08-16 14:51:12.091 INFO tokio-runtime-worker substrate: ✨ Imported #26 (0xb184…427d) 2023-08-16 14:51:12.092 DEBUG tokio-runtime-worker parachain::pvf-checker: Examining activation of leaf 0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d (26) 2023-08-16 14:51:12.110 DEBUG tokio-runtime-worker parachain::availability-store: Candidate included candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:12.151 DEBUG tokio-runtime-worker parachain::approval-distribution: Got new blocks [(0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d, 26)] 2023-08-16 14:51:12.151 DEBUG tokio-runtime-worker parachain::approval-voting: Imported new block. block_number=26 block_hash=0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d num_candidates=1 2023-08-16 14:51:12.151 DEBUG tokio-runtime-worker parachain::approval-distribution: Distributing our assignment on candidate (block=0xb184…427d, index=0) 2023-08-16 14:51:12.151 DEBUG tokio-runtime-worker parachain::approval-distribution: Importing locally a new assignment message_subject=MessageSubject(0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d, 0, ValidatorIndex(3)) 2023-08-16 14:51:12.192 DEBUG tokio-runtime-worker parachain::candidate-validation: About to validate a candidate. validation_code_hash=0xfcd42fa74d7ae08665c2e6e35006490d077fba8c03e781a5b6e4d7870c5f876b para_id=Id(1000) 2023-08-16 14:51:12.237 DEBUG tokio-runtime-worker parachain::candidate-validation: Acquired executor params for the session: ExecutorParams([]) validation_code_hash=0xfcd42fa74d7ae08665c2e6e35006490d077fba8c03e781a5b6e4d7870c5f876b para_id=Id(1000) 2023-08-16 14:51:12.307 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 26-[01010]-22 2023-08-16 14:51:12.313 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=5 2023-08-16 14:51:13.600 DEBUG tokio-runtime-worker parachain::bitfield-signing: Signing Bitfield for 4 cores: [0, 0, 0, 0] relay_parent=0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d 2023-08-16 14:51:14.610 DEBUG tokio-runtime-worker parachain::pvf: promoting WIP artifact /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/prepare-artifact-ePUmkcQ5wu to /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=96769 2023-08-16 14:51:14.610 DEBUG tokio-runtime-worker parachain::pvf: prepare worker concluded validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce worker=Worker(1v1) rip=false 2023-08-16 14:51:14.610 DEBUG tokio-runtime-worker parachain::pvf: enqueueing an artifact for execution validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.610 DEBUG tokio-runtime-worker parachain::pvf: spawning an extra worker 2023-08-16 14:51:14.611 DEBUG tokio-runtime-worker parachain::pvf: enqueueing an artifact for execution validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.611 DEBUG tokio-runtime-worker parachain::pvf: spawning an extra worker 2023-08-16 14:51:14.618 DEBUG tokio-runtime-worker parachain::pvf: enqueueing an artifact for execution validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.618 DEBUG tokio-runtime-worker parachain::pvf: enqueueing an artifact for execution validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.618 DEBUG tokio-runtime-worker parachain::pvf: execute worker spawned worker=Worker(1v1) 2023-08-16 14:51:14.618 DEBUG tokio-runtime-worker parachain::pvf: assigning the execute worker validation_code_hash=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(1v1) 2023-08-16 14:51:14.618 DEBUG tokio-runtime-worker parachain::pvf: starting execute for /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=97718 validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.627 DEBUG tokio-runtime-worker parachain::pvf: execute worker spawned worker=Worker(2v1) 2023-08-16 14:51:14.627 DEBUG tokio-runtime-worker parachain::pvf: assigning the execute worker validation_code_hash=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(2v1) 2023-08-16 14:51:14.627 DEBUG tokio-runtime-worker parachain::pvf: starting execute for /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=97719 validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.633 DEBUG tokio-runtime-worker parachain::chain-selection: Prepared 0 stagnant entries for pruning up_to=1692100274 min_ts=0 max_ts=0 2023-08-16 14:51:14.654 INFO tokio-runtime-worker substrate: 💤 Idle (5 peers), best: #26 (0xb184…427d), finalized #21 (0xf2ce…6329), ⬇ 1.3kiB/s ⬆ 2.6kiB/s [2mAug 16 12:51:14.717[0m [31mERROR[0m runtime: panicked at 'Invalid parachain block data: Error', /path/to/cumulus/pallets/parachain-system/src/validate_block/implementation.rs:92:10 2023-08-16 14:51:14.730 WARN tokio-runtime-worker parachain::pvf: execution worker concluded, error occurred: InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) artifact_id=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(2v1) worker_rip=false 2023-08-16 14:51:14.731 DEBUG tokio-runtime-worker parachain::pvf: assigning the execute worker validation_code_hash=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(2v1) 2023-08-16 14:51:14.731 DEBUG tokio-runtime-worker parachain::pvf: starting execute for /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=97719 validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.731 INFO tokio-runtime-worker parachain::candidate-validation: Failed to validate candidate para_id=Id(1000) error=InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) 2023-08-16 14:51:14.731 WARN tokio-runtime-worker parachain::dispute-coordinator: Candidate 0x22248dd3133074ef283fc029696cd4c11c67031259b0f57265c40003e69466b8 considered invalid: ExecutionError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })") 2023-08-16 14:51:14.752 DEBUG tokio-runtime-worker parachain::pvf: assigning the execute worker validation_code_hash=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(1v1) 2023-08-16 14:51:14.752 DEBUG tokio-runtime-worker parachain::pvf: starting execute for /tmp/zombie-b1c477a5afed5ee241aad0e11021e68c_-73454-XJLwhSPOSd7m/honest-validator-0/data/chains/westend_local_testnet/db/full/pvf-artifacts/wasmtime_0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce_0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 worker_pid=97718 validation_code_hash=0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce 2023-08-16 14:51:14.765 DEBUG tokio-runtime-worker parachain::approval-distribution: Distributing our approval vote on candidate (block=0x6350…dad0, index=0) 2023-08-16 14:51:14.765 DEBUG tokio-runtime-worker parachain::approval-distribution: Importing locally a new approval message_subject=MessageSubject(0x6350b5bc7ba92baeb96fd5205af698562aa83205cf5f73d8edc7699ec328dad0, 0, ValidatorIndex(3)) [2mAug 16 12:51:14.818[0m [31mERROR[0m runtime: panicked at 'Invalid parachain block data: Error', /path/to/cumulus/pallets/parachain-system/src/validate_block/implementation.rs:92:10 [2mAug 16 12:51:14.820[0m [31mERROR[0m runtime: panicked at 'Invalid parachain block data: Error', /path/to/cumulus/pallets/parachain-system/src/validate_block/implementation.rs:92:10 2023-08-16 14:51:14.806 INFO tokio-runtime-worker parachain::chain-selection: Unfinalized block reverted due to a bad parachain block. maybe_revert_hash=Some(0xa79889579877b43aa95efc4def58d91f73b02fa51da66d498188187dfa4c3e45) revert_target=24 maybe_reporting_hash=None maybe_reporting_number=None 2023-08-16 14:51:14.830 WARN tokio-runtime-worker parachain::pvf: execution worker concluded, error occurred: InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) artifact_id=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(2v1) worker_rip=false 2023-08-16 14:51:14.830 WARN tokio-runtime-worker parachain::pvf: execution worker concluded, error occurred: InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) artifact_id=ArtifactId { code_hash: 0x3dd1061f49ed6ed80030e628dd3f21bdb7165a93ca11cd32231fcd802f1265ce, executor_params_hash: 0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314 } worker=Worker(1v1) worker_rip=false 2023-08-16 14:51:14.830 INFO tokio-runtime-worker parachain::candidate-validation: Failed to validate candidate para_id=Id(1000) error=InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) 2023-08-16 14:51:14.831 WARN tokio-runtime-worker parachain::approval-voting: Detected invalid candidate as an approval checker. reason=ExecutionError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })") candidate_hash=0x22248dd3133074ef283fc029696cd4c11c67031259b0f57265c40003e69466b8 para_id=Id(1000) traceID=45383551086429594700792699688317670593 2023-08-16 14:51:14.831 INFO tokio-runtime-worker parachain::candidate-validation: Failed to validate candidate para_id=Id(1000) error=InvalidCandidate(WorkerReportedError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })")) 2023-08-16 14:51:14.831 WARN tokio-runtime-worker parachain::approval-voting: Detected invalid candidate as an approval checker. reason=ExecutionError("execute: execute error: AbortedDueToTrap(MessageWithBacktrace { message: \"wasm trap: wasm `unreachable` instruction executed\", backtrace: Some(Backtrace { backtrace_string: \"error while executing at wasm backtrace:\\n 0: 0x26e271 - <unknown>!rust_begin_unwind\\n 1: 0x4736 - <unknown>!core::panicking::panic_fmt::h3f583988bd6f4628\\n 2: 0x4c5f - <unknown>!core::result::unwrap_failed::h6d35dec810131ffe\\n 3: 0x22c933 - <unknown>!cumulus_pallet_parachain_system::validate_block::implementation::validate_block::h85e5c745e918850a\\n 4: 0x18d870 - <unknown>!validate_block\\n 5: 0x3068a7 - <unknown>!<wasm function 3762>\" }) })") candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 para_id=Id(1000) traceID=339732803593143142233024039371907190662 2023-08-16 14:51:14.877 DEBUG tokio-runtime-worker parachain::dispute-distribution: Sending rate limit hit, slowing down requests occasion="in start_sender" candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 num_wakes=0 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:14.901 DEBUG tokio-runtime-worker parachain::dispute-distribution: Sending rate limit hit, slowing down requests occasion="in start_sender" candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 num_wakes=1 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:14.917 INFO tokio-runtime-worker parachain::dispute-coordinator: New dispute initiated for candidate. candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 session=1 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:15.330 INFO tokio-runtime-worker parachain::chain-selection: Unfinalized block reverted due to a bad parachain block. maybe_revert_hash=Some(0xb184df0ffc19c807602da5465fd82be299361f707d4506f6080b8cc02dbe427d) revert_target=26 maybe_reporting_hash=None maybe_reporting_number=None 2023-08-16 14:51:15.823 DEBUG tokio-runtime-worker parachain::dispute-distribution: Not importing empty batch candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 traceID=339732803593143142233024039371907190662 2023-08-16 14:51:16.344 DEBUG tokio-runtime-worker parachain::approval-voting: approved blocks 23-[11]-22 2023-08-16 14:51:16.346 DEBUG tokio-runtime-worker parachain::approval-distribution: Received `ApprovalCheckingLagUpdate` lag=0 ``` #### malus ``` 2023-08-16 14:50:57.837 INFO tokio-runtime-worker substrate: 💤 Idle (5 peers), best: #24 (0xa798…3e45), finalized #21 (0xf2ce…6329), ⬇ 4.5kiB/s ⬆ 3.1kiB/s 2023-08-16 14:51:00.148 INFO tokio-runtime-worker substrate: ✨ Imported #25 (0xc596…cc82) 2023-08-16 14:51:02.838 INFO tokio-runtime-worker substrate: 💤 Idle (4 peers), best: #25 (0xc596…cc82), finalized #21 (0xf2ce…6329), ⬇ 3.4kiB/s ⬆ 3.0kiB/s 2023-08-16 14:51:07.844 INFO tokio-runtime-worker substrate: 💤 Idle (4 peers), best: #25 (0xc596…cc82), finalized #21 (0xf2ce…6329), ⬇ 1.1kiB/s ⬆ 1.4kiB/s 2023-08-16 14:51:12.100 INFO tokio-runtime-worker runtime::election-provider: [#26] 🗳 Starting phase Unsigned((true, 26)), round 2. 2023-08-16 14:51:12.114 INFO tokio-runtime-worker substrate: ✨ Imported #26 (0xb184…427d) 2023-08-16 14:51:12.850 INFO tokio-runtime-worker substrate: 💤 Idle (4 peers), best: #26 (0xb184…427d), finalized #21 (0xf2ce…6329), ⬇ 1.8kiB/s ⬆ 2.2kiB/s 2023-08-16 14:51:15.066 INFO tokio-runtime-worker parachain::dispute-coordinator: New dispute initiated for candidate. candidate_hash=0xff96286f9cb595f9e901d3545ceacf863a00fdf7caa52c274eccd1090cdcda37 session=1 traceID=339732803593143142233024039371907190662 ``` `<stopped>`

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.