# Sync Error - 2021-09-15
## Last Batches Forged Info
Batch | Block | Coordinator
32719 | 5497131 | 0x715ea
32718 | 5497131 | 0x715ea
32717 | 5497131 | 0x715ea
32716 | 5495987 | 0x9302d
32715 | 5495984 | 0x9302d
All batchs until the batch 32428 was forged by 0x9302d
The batch 32428 was forged at block 5493050 by 0x715ea
## Steps to reproduce:
Synchronization from scratch:
`go run main.go wipedbs --mode coord --cfg cfg.builder.internalTestnet.toml`
Geth server: http://3.67.36.55:8545
`go run main.go run --mode coord --cfg cfg.builder.internalTestnet.toml`
Every time I start the synchronization from scratch I get the error in different batches.
```
First time in batch 31300
Second time in batch 17193
Third time in batch 8590
```
Error: synchronizer.go line 982.
`stateDB.BatchNum (8590) != evtForgeBatch.BatchNum = (xxxx)`
xxxx is the current batch that doesn't match.
**Conjecture:** the synchronizer jumps the ethereum block that contains the next batch needed. Also, if the current blatch is higher than the needed one, It doesn't try to look for the needed ethereum block (that contain the proper batch), It only looks for later blocks
In bootcoordinator after discard the block 5493049 this error is shown:
```
2021-09-15T09:06:29Z ERROR node/node.go:797 Synchronizer.Sync: stateDB.MTRoot (8562053543920230074719444761882318320373526795094447640926973251588268741885) != forgeBatchArgs.NewStRoot (4420853456904067943149301489824932475145521099832158874273450016532564933486)
/home/ubuntu/hermez-node/synchronizer/synchronizer.go:985 github.com/hermeznetwork/hermez-node/synchronizer.(*Synchronizer).rollupSync()
/home/ubuntu/hermez-node/synchronizer/synchronizer.go:590 github.com/hermeznetwork/hermez-node/synchronizer.(*Synchronizer).Sync()
/home/ubuntu/hermez-node/node/node.go:732 github.com/hermeznetwork/hermez-node/node.(*Node).syncLoopFn()
/home/ubuntu/hermez-node/node/node.go:788 github.com/hermeznetwork/hermez-node/node.(*Node).StartSynchronizer.func1()
{"err": "stateDB.MTRoot (8562053543920230074719444761882318320373526795094447640926973251588268741885) != forgeBatchArgs.NewStRoot (4420853456904067943149301489824932475145521099832158874273450016532564933486)"}
```
After second attempt to syncronize the bootcoordinator fully we got this another error:
```
2021-09-15T15:52:51Z DEBUG synchronizer/synchronizer.go:553 Syncing... {"block": 5497131, "ethLastBlock": {"Num":5503302,"Timestamp":"2021-09-15T15:49:51Z","Hash":"0x6df1f006c75003e7bcc2a045a0a273d50fe77c635cc5b2a019dd853193c0ba3b"}}
2021-09-15T15:52:51Z DEBUG statedb/statedb.go:225 Making StateDB checkpoint {"batch": 32717, "type": "synchronizer"}
2021-09-15T15:52:52Z DEBUG statedb/statedb.go:266 Making StateDB Reset {"batch": 32716, "type": "synchronizer"}
2021-09-15T15:52:52Z ERROR node/node.go:797 Synchronizer.Sync: stateDB.MTRoot (8562053543920230074719444761882318320373526795094447640926973251588268741885) != forgeBatchArgs.NewStRoot (4420853456904067943149301489824932475145521099832158874273450016532564933486)
/home/ubuntu/hermez-node/synchronizer/synchronizer.go:985 github.com/hermeznetwork/hermez-node/synchronizer.(*Synchronizer).rollupSync()
/home/ubuntu/hermez-node/synchronizer/synchronizer.go:590 github.com/hermeznetwork/hermez-node/synchronizer.(*Synchronizer).Sync()
/home/ubuntu/hermez-node/node/node.go:732 github.com/hermeznetwork/hermez-node/node.(*Node).syncLoopFn()
/home/ubuntu/hermez-node/node/node.go:788 github.com/hermeznetwork/hermez-node/node.(*Node).StartSynchronizer.func1()
{"err": "stateDB.MTRoot (8562053543920230074719444761882318320373526795094447640926973251588268741885) != forgeBatchArgs.NewStRoot (4420853456904067943149301489824932475145521099832158874273450016532564933486)"}
```
The last batch saved on the stateDB at bootcoordinator da internaltestnet is BatchNum32716.
## Resolution
It was identified that we have 2 problems.
### Problem 1 - Wrong state in the blockchain
The wrong state in the chain was caused by some tests forging batches with fake provers, theses tests forged batches without sending the data required in order to rebuild the state from the blockchain, this caused the node to not be able to compute the state change, making the node synchronization to get stuck.
This sequence of L1 transactions shows de problem, as you can see, everytime the state is changed by a forgeBatch call, the field `l1L2TxsData` provides the data in order to recalculate the state, allowing the node to update its state when sycnrhonizing with the block chain, and when the state remains this field is empty. _To see the tx data, click in `Click to see More` and then in `Decode Input Data`.
https://goerli.etherscan.io/tx/0x51bd7fb3a048c10e3ab258558a90a1705508a8d648bb161c7575047a73d4f62b
https://goerli.etherscan.io/tx/0xd0a7e593f1420dc524d709c9544fbd78b004180faaa7bf280a9658efb52dcb42
https://goerli.etherscan.io/tx/0xb2fdd3de177ae902476c76381d87eacf99e7ea827f609f5896696940d1d7a9df
https://goerli.etherscan.io/tx/0x9697db68b1a3f51e055011ce14d802e0a877c44c09c9d49b06206a4dc9ff5acb
https://goerli.etherscan.io/tx/0x2a965614013012404756860ca7fadc5f769aee9000cda5210c99ddc8c3431402
https://goerli.etherscan.io/tx/0xe31c7c7edc51f623a1734116efc7c7495fc60e71500768858723c8f3ce73534b
https://goerli.etherscan.io/tx/0x1e79cc380e2793f10cfbb02e63215b1c59a44f7888fd7fbbb12c0f39cf74b198
https://goerli.etherscan.io/tx/0x088f832896c400e1d690e35b7841151d364f9abae2dba9ac21bbf7ce3ec43360
https://goerli.etherscan.io/tx/0xb48ef7cf55f9c7cc704a919fb191cca3f3c159787c729934a4792bdeaf740ba5
https://goerli.etherscan.io/tx/0x64781fd769176cd680779dbafabb1b4c1a033d37fdf8ca3adc441777ab6784ed
https://goerli.etherscan.io/tx/0x3d7baad0502e8066325656ededbf1d3f06670cd65f7602d306cdc51fda919cf0
https://goerli.etherscan.io/tx/0x692e191ea1beb7c4065208e92ab262a98a9f80d42096ea0911c4879d7b2ff731
https://goerli.etherscan.io/tx/0x0bf1f12192977d71822cf7987002ea45dfa1be65c65be3b125f32252f89bb868
https://goerli.etherscan.io/tx/0xa53ce708b288c520c6777a545fdadbde9b5f65afc1639f7ff09c18c0ff34ce44
https://goerli.etherscan.io/tx/0x2cc760ee0b0015fb524b81b70b71f22f5fb7822e74387ab74b3708e6c063b5b4
The wrong transaction is this one: https://goerli.etherscan.io/tx/0x0bf1f12192977d71822cf7987002ea45dfa1be65c65be3b125f32252f89bb868
### Problem 2 - Wrong state reset state in the node.
When something wrong happens during the sychronization, a reset routine is executed in order to maintain the databases synchronized, theoretically this routine tries to revert the changed made during the synchronization in one data base to keep then synchronized.
Looks like this routine is not working when the state computed is different from the state provided by the block chain, causing the databases to get out of sync and making the synchronization to get stuck in a infinite loop of error messages.
An issue was open in order to handle this error, here is it: https://github.com/hermeznetwork/hermez-node/issues/1142
Sign in with Wallet
Connect another wallet