Incomplete Mainnet Upgrades Cause Failed ExecuteTransactions From State Sync

# Incomplete Mainnet Upgrades Cause Failed ExecuteTransactions From State Sync We are running Aptos Fullnode on Mainnet these errors appear when the state syncing accidentally stops. ``` Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:28.855798Z [sync-driver-3] INFO aptos-move/aptos-vm/src/aptos_vm.rs:1069 Executing block, transaction count: 2000 {"first_version":56856833,"name":"execution","txn_id":0} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:30.528239Z [sync-driver-3] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction infos don't match. version: 56856834, txn_info:TransactionInfo: [txn_hash: 008ca83c, state_change_hash: 111ff228, event_root_hash: 41434355, state_checkpoint_hash: None, gas_used: 592, recorded_status: MoveAbort { location: 2f46fd0be3811c1280b63a1c07a4eead16b520b178e11ceedca591d1f9299c5f::sw33t_extend, code: 196609, info: Some(AbortInfo { reason_name: \\\"ERR_NOT_OWNED_TOKEN\\\", description: \\\"\\\" }) }], expected_txn_info:TransactionInfo: [txn_hash: 008ca83c, state_change_hash: 111ff228, event_root_hash: 41434355, state_checkpoint_hash: None, gas_used: 592, recorded_status: MoveAbort { location: 2f46fd0be3811c1280b63a1c07a4eead16b520b178e11ceedca591d1f9299c5f::sw33t_extend, code: 196609, info: None }]\"","name":"storage_synchronizer"} ``` We enabled the "ExecuteTransactions" mode for syncing, which is mandatory for us. ``` state_sync: state_sync_driver: bootstrapping_mode: ExecuteTransactionsFromGenesis continuous_syncing_mode: ExecuteTransactions ``` Due to an issue in the source code the fullnode executes the txn differently so the node got stuck. We use source code with the latest tag [aptos-node-mainnet_7b988a4477446a828e93b9652f9d9c45f1e23029](https://github.com/aptos-labs/aptos-core/releases/tag/aptos-node-mainnet_7b988a4477446a828e93b9652f9d9c45f1e23029) When the following code compares the txn info of the fullnode execution with the mainnet's, the hash value is different, because the error message of the fullnode execution is more detailed and contains "Some(AbortInfo { reason_name: \"ERR_NOT_OWNED_TOKEN\", description: \"\" })", while the txn info of the mainnet has a None value. https://github.com/aptos-labs/aptos-core/blob/aptos-node-mainnet_7b988a4477446a828e93b9652f9d9c45f1e23029/execution/executor-types/src/executed_chunk.rs#L98 ```rust if txn_info_hashes != expected_txn_info_hashes { for (idx, ((_, txn_data), expected_txn_info)) in itertools::zip_eq(self.to_commit.iter(), transaction_infos.iter()).enumerate() { if &txn_data.txn_info != expected_txn_info { bail!( "Transaction infos don't match. version: {}, txn_info:{}, expected_txn_info:{}", self.result_view.txn_accumulator().version() + 1 + idx as u64 - self.to_commit.len() as u64, &txn_data.txn_info, expected_txn_info, ) } } unreachable!() } else { Ok(()) } ``` It looks like the validators in the mainnet are not currently finished upgrading. So we need a fix to make the mainnet data and the fullnode execution results consistent. ## To reproduce Run Aptos Fullnode on Mainnet with "ExecuteTransactionsFromGenesis" bootstrapping mode, and sync to version: [56856834](https://explorer.aptoslabs.com/txn/56856834) **Stack trace/error message** ``` Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:28.346592Z [sync-driver-3] INFO state-sync/state-sync-v2/state-sync-driver/src/bootstrapper.rs:469 {"message":"Highest synced version is 56856832, highest_known_ledger_info is V0(LedgerInfoWithV0 { ledger_info: LedgerInfo { commit_info: BlockInfo { epoch: 884, round: 20056, id: HashValue(59b09e9952a729ef0e726b72f0c69b0e1a7254a9e26b0b639ccb4c8b948d236f), executed_state_id: HashValue(23a464d25764b2bfc6b4dece5eb2bf0e7ccfb16a6503c85d65d4a13cb11bca57), version: 56968516, timestamp_usecs: 1671961064388350, next_epoch_state: Some(EpochState [epoch: 885, validator: ValidatorSet: [ed84f8724f703c17075fb528b5713424e794f79ac82273d8f747b3f26fdd9ecd: 156540148481933, eb5d1dfba88204036132b2bf4f4600f20822738e01985401ba45e75c464d1965: 158743845083610, 46cbb9afc2255c8afcf2a0c134fe9066a9edaf0ada139fc234030c09b984100d: 101420907514432, 29cf6ca262ee369b944bc3daa18879e1911f7d5aea120b3c9a54174f610f4630: 101420776620445, 4e28ca55c3ccde886560d667d60cd2d1193e742ff5aede3ba93b3860a97e207f: 101420776976523, 5ae6127c854988607bb3375484e61ba10e2ce5e6ca9ff92a7b322149ddbbf552: 678838931943432, 32ad233a939bfbafb8d9056c0ae2eba58828d8baf5582277578ced38477f0f14: 365470435622157, 29c4513282335179295fa7c6a225ca9e2e17b3e14e30a463969d29b87ccb8db6: 161030010884121, 1187093055547879056b22e3d3e80f491bc1b4387c3642d934da0fedaa5e5f28: 101623387837924, 5d31f0d7a171e5ee6a6e2f11917f5a157d745e993f299a8b75acb961bd9da4e0: 291033504191045, f25c7b01c68f7ca3583dde23cbaefd14f930f6655237be627435b9727b7c9034: 507509972877456, 29a37d38db6717a61ba0cad4ffe01d15a320888601f15b3cf920feb0f9813abf: 506699058967475, 60b7bb4774ca6bebe6fd9204252f3abcd528afa497854d8fd0e19c1c6d7616f7: 507307507731196, 99d3561f2c46a3755e681a5b2199886720257404856cd3eb2b1c2566e585ea11: 506902206784712, 39af34a3586c7ac7c4947885f3693bd748969d70b54db38655419060c4fe833f: 1014209615530863, bfa99851a3abd9f64e919c5c33c32b591564eb0ebb3e838fcc456d7ae0cc6ec2: 101419021607061, c48683f9f684d2845c18e8d33d5f6c8581acfc1eb1c78b1de25562d136877e10: 101419449603065, 60b00bc1f6cda15c9cee94bab2e44dc7727701503da80d88c23c8436c4d597a4: 101420846228173, d4e43fa15a4cf475b35f7278f02ae5a694e14fbd7ef2494e39e7978781b3eefe: 101420824131989, da557f6d45952778e0c4c7296dc8f8303d846d5453800efd7b03cfbd5f8def04: 101420878921391, 947c6451ad9a8659cba4e60a6da41086eb66e73fc07936bd3b7044873b5e094d: 2195971121857219, 86da6288dd50be1653aa2a8269acf881d3b4b414938ca7ab3b662c2dfeb794df: 2200936894276646, e8fac9c24a3bfc71ed79d466fa3c2c2b4787f1f9686e1d1e9e97fde473898788: 2200821681463057, ee49776eff9fd395eb90d601449542080645e63704f518b31c6f72b6a95d7868: 2201767997720920, 48179547b95487f77214c12e2561d53531122a91e5f79ee9be968ce8185bc444: 2200799354040568, b5064f2e6d2d4887c836c8cbd90d23a75cd7afd4530e4a82f06bf528abcb30d6: 2200790737416753, fcbc1da54b125c093d49bf4a604a1854d38f99b6cffc09255d8af5446898ddba: 2200788488555847, 4c893cf8d614f503e982df5b48e9443bab4ea6be4f922a5d3858305c0a3af3ee: 2200923324367563, c32f662cd9718f02d8a8e5628f8f642fa27cd9b5f457b406ed734901a4939e34: 2000314813040520, 324df1e27c4129a58d73851ae0e9366064dc666a73e747051e203694a4cb257: 2000312928795864, 3d654d35d558d6381fe6a0484d81c4fed128cd8adc1ca7a1482c297da80ee039: 1648576522194513, ea733e480c379c61dae9cfb3f14a388f6a683d7a797c135f86b924fb7a2e8cdb: 101185804548500, bcb83e07eed46120a0143bd9861ea82495293228b9cd64cc43bda722edaeb7ad: 101302636347043, c067ce10f1faa4812fe608e2d4b3e1fe0bb629e91cfff532cccc23803a624414: 101299730682964, c1f31446852220ffdf2fe294fd42fa5d7ced48c7d0f1378d6813ae8d972f1adf: 101185887197608, 52b48fb6e6883d4cdf244cde8d7e972ea2d8ece9b5e484ed5085aacef0e306a1: 101186661008895, 9dfeca70acb3e01328d9dbbda4a6c23913fcd81ef25f74fab52cb2e96b0a1486: 101186567711078, 85bec1dcb618004064ab52a594321e8f047bac53be8244c5d10ac136128e30f7: 101289092208647, 4cfeaf68aacfd0cd459c292837854ce058b9ba6c1cc35e8684b1a5030b1dc922: 101188087678959, 447fc1fc130cf9245d9ca291785d0f8a00c3ebdc26b2e4c6165801b0af01c917: 101185837124843, d60280852b50c341b22bba06562a817293d7b834eac251e7600b25d8bdd9efa3: 101186466230274, dca6aaf410e984463b37e192652b3c09b0ae3968b3e0171bb076ae0e21112ef9: 101188481420296, d2dc57e95669c9bf102cb5693fe8b2a8b10d330213d7ddd9fb417f86f183dc1d: 101301664071795, ae139fd4e266beb28925227dcd220d89348117680e48d4aa20685ae102ae2a13: 2200798558081182, 568fdb6acf26aae2a84419108ff13baa3ebf133844ef18e23a9f47b5af16b698: 101417570296018, 99d623e94981c2a6d41306785d69d66c237c6d3ac305cb6f32307785fde7fda2: 2200776136566440, 2fa9d01fb4a1c2fdac5f45419b98213604bc215817d4b394ab372e5d59d2753c: 168362418869975, 35b922386bd35db0f33c732a70d7b6a2e91e463812c8d710f925f6da0e0ace07: 101414579217652, 4b2c524fc36698caf9d049b6088faa002cea3bbcbf5ed764f59768b4fe96efcf: 101185557209509, 18a3ef480b7883d05b917873a8b6d5f5a5f763372b650242ede89836a67cdce8: 101301145479953, c807d24a6ac70599cc9b72c649eb6b00c34a0e9c704447c44d75fade07213a26: 1859454038339666, adbfddd3d10cad1eed59d2c0793df0bfd1d16255309c38b34b8351daaddecccc: 2194361702901328, e9f7d36334768683b85ec1df892dfd8fc389c8f6dbe0b890195d579a6e390c78: 101296386473751, e1abb4ea11673aad3137052dce831eb20a1907815cb4a03ccac38ba7a9e988d3: 101301184322009, 6fdb6516f550ceada7d1fb687b52e4d334c0805a856f17c85f1eb852cbe66ffb: 501790735333153, 925ec5ab56572964dc3cc69bb86c224367dbfc186f6831c26639f9428326c2b4: 101183471792547, 6c8a3474cb49202515d121fea0f3217d303e41f6bdc43e615f1cd90855118089: 1562793708631648, df0996b6998ec080ea7931ccd87400fa74eb57f8129bb0f11996ae8bae12d565: 229366289611443, b840f1cfae5aaa4b50a11a6a23e2e859717e2f355fcd614592b219fc4d4e44a3: 101298711338044, 8d5c9e82d612c390b5dea347918b6421146bb638724854300eed065cc4211a6e: 101183622548735, 32f23e884c0c6f663997c7d4602ef0916bae15492833ee8e7266135e40ff8b14: 101298897261760, cf2a5df3737cd33344dc332416c35278431aed26ff684491b1bf7f9d19e3ee93: 101183806086646, b3254509d1cd652d6e1ca06fd37f3d20aaa42f7faeaf297577b85cf92d2f7462: 2194325999015491, a6eb54c274da96778a6702740a763f6aa7b02eef3d1efc2623d28289d185118: 2194323277344283, 1c6a051d9a5ec4cf4200bab21a794d3afe8259d625f6b78aa4d01ab4652327d5: 1562515139105254, 649127a4d1b26b4cf936f7b795917dbcb571c59c2926ae473fe4d9df3575790e: 1693653361533868, 9df537cbe360f51499f53ca27b83f1c3a92fb7b068bc79eb8e33c85a50f84380: 115743340695140, be4e66094c14a8c69361173fbd6694279db5fd8401bcb1d9c5df066755820a30: 115743475556278, 22dd12212d76bc5844b4a0605bf59fa56547f42a40fb42267e29adca5efecfc8: 940961327603728, 6b5b8d5904ee259636314d6d914f1a59bb6430126dd91a6118202dc734b1df8a: 110233049441591, f08bb83cf89d299e4cddee7ef987b850a35a399900ba22860cc1553da3682a84: 101181663825589, 904bc056aa17a45c2715113e06cb42f5762581a7b839206f5348938db69425be: 101192709412228, 6a1d3313a58719cf5abc1aaee6b4b74bc7e2c69036a8a02f02eb4f3ab4a3d678: 101189529782419, b87476d8877cfc324a520cef4d0cd6e42e26ec3ce613a8127220e2e444561e16: 764635381708155, b225277ba076b6620447f7b10abb30911966eb66d25fb526b7f09c2227425a11: 385814922780102, c29a231581f243ddcbf97de8b93c8aabb466dd6912ce5ba12579ff2f13cb8da5: 385815091605391, fc307ecd61ac025f5ec1828d2114d11b474b4d8a1b0c228c38614dc96b940048: 385815037672986, e8a71d40da724ac1ff522d9686dbf1da21bab2d803441df73c86c4bae76d399: 138243540597978, 371dfdee9fbecaa358683936c5946aa28393067fa60e1e185c4250da461e15cf: 2200650847238814, 9da88926fd4d773fd499fc41830a82fe9c9ff3508435e7a16b2d8f529e77cdda: 2000174780860050, 8e943e68e8e4a1ebb7e5a250b081af507d40e616283f38d2d2a2a4b6523c02ac: 101181843981819, 46b51b26710797faed701722bfc809386110c9720328f2b431e193cb5dc161f7: 2200648241602093, 302b69423fde5c7dd333ee50d48df16f076072e61f95e0f6db5685ee6c70d35b: 101182027329493, f2b6feeea4546648c257c7e33ff4f0aa8db3bb2949cca065c9db015478363458: 101182393921008, 81d41a31fb5d3f827d188511e6fca1ddbed6cfa0ffbd46fa6d84a7adfe59f356: 2195758211993242, 71debfa733376c4f2dc8970de8962931fe107c7dc1daad3bf189101bfd4adc4b: 101180670037810, 4cc22e5d2e84794c3c672b0b34145898ee971a4ff042eabae23627547f6dac80: 2194255869920474, dc37463c5782469908d7ff43873390075980a8fa5b1c0770457c48f8891a5bff: 653294353123169, 6fd3ee60a084952162d62ead1a09f4ccd611f17608789ae45eba4f54967a295e: 506773794086784, ef191e643c2ed2d103de4109a506e020f5dde818cbc1ffe5e4ee36328cd22e9b: 2200657501173941, 209154f6e735562ff13b963b91588039a162fa95f409c068f917f27b762dfff8: 1780830825155162, 9a41af5c002f91dc29cb2843ab823e3d6cdaa6cc86e6331646a4f1328609a744: 1652278867111337, 483cf8942c76d86ed28a9849940abdea14952a551be84c38c844208e13d976d4: 101182198229470, 67af497cdd68361b18dda2409f244ebf6f776aee99094e16a5f371e39456de35: 101259905318044, 3eaf3ebc49555c705c34ec1ffe77404bdb7383b388476c36538b92b7b1b806c9: 2001488130814707, ffb19e4519bc434d3a064652064d14dd34fabc8f785f6be8670d7465d8419c45: 2200582746519670, b4a4f1ef8b0702d85547dc444571a473f736e1205a86db36dad13815ad9bbbf6: 2200588657777932, a286a9187c86b4a3cef60478e5c61bb45757b795a8e6577ebbf0fa990db574cc: 1589623759957344, f05d2fe6618a3cb4258e94f2da24c1ee71ae4e7942b4f8b5836ad5267e59568f: 1714761988045168, dc3d30891191132b668fef5304e32b869e53741ee4f3fbb514e9ae2b05dcfc50: 330677897255158, 8be2ba62bfd783e5fbff57a07acf2a9a95f4d234b3729fbfda9c63f3f42fb78f: 1659554927434314, 401a881d9770f054a1b8cd6829f3d65a040bbe9b387ebcce50bddf0f853077d9: 182673739907286, ]]) }, consensus_data_hash: HashValue(2ad5f89b57d98870d874c03237ccc6511c39213b946f7a7a564004d771737b4b) }, signatures: AggregateSignature { validator_bitmask: BitVec { inner: [127, 119, 111, 149, 131, 214, 50, 251, 113, 90, 210, 81, 88] }, sig: Some(93a18d0ff752fb8db5e384376c675ef125e77566d0d54cdbb57c11428f802e0f23beff9f5a4293d9eeb67328200aaa0d0d705d968a4b2630f2eefb01d35f542b066db42f609b5ab3e8672f2ae77fe4f9801cca4fa578250b9be691e0f600792e) } }), bootstrapping_mode is ExecuteTransactionsFromGenesis.","name":"bootstrapper"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:28.346665Z [stream-serv-2] INFO state-sync/state-sync-v2/data-streaming-service/src/streaming_service.rs:223 {"event":"success","message":"Stream created for request: StreamRequestMessage { stream_request: GetAllTransactions(GetAllTransactionsRequest { start_version: 56856833, end_version: 56876885, proof_version: 56968516, include_events: false }), response_sender: Sender { complete: false } }","name":"handle_stream_request","stream_id":730} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:28.446881Z [stream-serv-3] INFO state-sync/state-sync-v2/data-streaming-service/src/streaming_service.rs:326 {"event":"success","message":"Data stream initialized.","name":"initialize_stream","stream_id":730} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:28.855798Z [sync-driver-3] INFO aptos-move/aptos-vm/src/aptos_vm.rs:1069 Executing block, transaction count: 2000 {"first_version":56856833,"name":"execution","txn_id":0} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:30.528239Z [sync-driver-3] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction infos don't match. version: 56856834, txn_info:TransactionInfo: [txn_hash: 008ca83c, state_change_hash: 111ff228, event_root_hash: 41434355, state_checkpoint_hash: None, gas_used: 592, recorded_status: MoveAbort { location: 2f46fd0be3811c1280b63a1c07a4eead16b520b178e11ceedca591d1f9299c5f::sw33t_extend, code: 196609, info: Some(AbortInfo { reason_name: \\\"ERR_NOT_OWNED_TOKEN\\\", description: \\\"\\\" }) }], expected_txn_info:TransactionInfo: [txn_hash: 008ca83c, state_change_hash: 111ff228, event_root_hash: 41434355, state_checkpoint_hash: None, gas_used: 592, recorded_status: MoveAbort { location: 2f46fd0be3811c1280b63a1c07a4eead16b520b178e11ceedca591d1f9299c5f::sw33t_extend, code: 196609, info: None }]\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:30.536669Z [sync-driver-3] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56858833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:30.545081Z [sync-driver-3] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56860833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:30.553347Z [sync-driver-3] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56862833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:31.454611Z [sync-driver-0] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56864833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.054780Z [sync-driver-0] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56866833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.063305Z [sync-driver-0] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56868833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.071570Z [sync-driver-0] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56870833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.554976Z [sync-driver-0] ERROR state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:905 {"message":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56872833.\"","name":"storage_synchronizer"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.746298Z [sync-driver-0] WARN state-sync/state-sync-v2/state-sync-driver/src/driver.rs:421 {"error_notification":{"error":{"UnexpectedError":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56862833.\""},"notification_id":5679},"message":"Received an error notification from the storage synchronizer!","name":"synchronizer_notification"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.746311Z [sync-driver-0] INFO state-sync/state-sync-v2/state-sync-driver/src/utils.rs:238 {"message":"Terminating the data stream with ID: 730, notification and feedback: Some(NotificationAndFeedback { notification_id: 5679, notification_feedback: InvalidPayloadData })","name":"driver"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.746334Z [sync-driver-0] WARN state-sync/state-sync-v2/state-sync-driver/src/driver.rs:421 {"error_notification":{"error":{"UnexpectedError":"Storage synchronizer error: \"Failed to execute/apply the storage data chunk! Error: Transaction list too new. Expected version: 56856833. First transaction version: 56864833.\""},"notification_id":5680},"message":"Received an error notification from the storage synchronizer!","name":"synchronizer_notification"} Dec 25 18:57:33 aptos-node[17778]: 2022-12-25T10:57:33.746344Z [sync-driver-0] INFO state-sync/state-sync-v2/state-sync-driver/src/bootstrapper.rs:420 Waiting for the storage synchronizer to handle pending data! ``` ## System information Aptos Core Version: [aptos-node-mainnet_7b988a4477446a828e93b9652f9d9c45f1e23029](https://github.com/aptos-labs/aptos-core/releases/tag/aptos-node-mainnet_7b988a4477446a828e93b9652f9d9c45f1e23029) Rust Version: rustc 1.64.0 (a55dd71d5 2022-09-19) Computer OS: Ubuntu 18.04.6 LTS (GNU/Linux 4.15.0-163-generic x86_64)

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.