---
# System prepended metadata

title: 'Paseo Storage Root Mismatch at block #1008649'

---

# Paseo Storage Root Mismatch at block #1008649

#### tl;dr

If there is a current transaction happening and the runtime wants to get the next key we are actually returning the value that the desired next key holds instead of the next key itself.

### Just to share the investigations steps


- On yesterday's call with Jimmy we rollback to a commit before the inclusion's of storageDiff changes (that discards the usage of trie snapshots and uses a lightweight mechanisms to track nested transactions)

- So w/o using the storageDiff changes we were able to get the correct state root, but after the storageDiff the mismatch was happening.

### So what I did was:

Logged insertions and deletions with their respective keys as well as starts and commits of nested transactions, this enables us to track in what order the keys are inserted/deleted in the trie and when a transaction starts or when is committed.

So the next step is to execute two test runs: one before and one after the storageDiff changes. Then we have the logs when it passes and the logs when it fails, so we should compare both to find in what point the data does not match.

_left is the logs before the `storageDiff` changes and on right is after the changes_
![image](https://hackmd.io/_uploads/HyYBslbmA.png)

Now I need to compare line by line looking up for a small difference, so to speed up this work I used a tool for text comparision that showed me all the diffs between these files

![image](https://hackmd.io/_uploads/HkJinlbQC.png)

The first diff was enough to give me the clue about the problem: In the first case we delete a specific key and in the second we dont.

> deleting: 0x196e027349017067f9eb56e2c4d9ded5a2ee677da3917cc29ee3b29c9f94c8658c58f3b5989bb72cdf070000

What I did after was searching for this specific key hex `0x196e027...` in the two outputs and I find that in both they inserted the same key in the exact same order (and line)

![image](https://hackmd.io/_uploads/ryHcaeZ7A.png)

Also another thing important to notice is that the inserting (in both outputs) happens inside a **transaction**

![image](https://hackmd.io/_uploads/ryz4Al-X0.png)

So, okay, both tests: started the transaction, inserted the key without problems in the exact same order but one deletes it and the other don't.

Then I thought that the problem might be related to how Gossamer returns keys to the runtime, given that to delete a key you need first to get the key. So I noticed that the method `NextKey` (which is resposible for, given a key, return the immediatelly next key in lexicographic order) when there is an ongoing **transaction** it was wrongly returning the value that the immediatelly next key holds instead of the key itself.

After fixing the method to return the proper next key and the test that previously was failing with `Storage Root Mismatch` now is passing!

