# Uni-5 v12 alpha app hash

---

jacob/reece call 1/22/2023

Jacob believes that the issue is entirely gas related. The reason is that The chain is not actually stopping and the block has the same contents. So if we look at different nodes, we get exactly the same upcoming block. however we still get the block.header.apphash error. nodes don't crash because the state is the same but the gas is different. So instead they just reject the proposal.

The proposal contains numerous transactions. nodes are rejecting the proposal because they each calculate gas for their own and that is different. So they are finding that the gas in the proposal block does not match what their node finds.

followup:

* roll back to pre upgrade
* run the upgrade again with the [fixed upgrade handler](https://github.com/CosmosContracts/juno/pull/525) (good catch!)

If doesn't work:

* remove x/globalfees
* roll back again to pre upgrade
* upgrade again without globalfees

if that doesnt work:

* remove globalfees and feeshare
* roll back again to pre upgrade
* upgrade again without globalfees

sus transactions:

* mike.exe store code
* multiple choice voting

We sure to run the suspicious transactions again. most likely whatever is causing the problem is in one of these suspicious transactions.

**longer term**
this is a good case for IBC test. If we are unable to make IBC test work so that we can simulate upgrades in CI, I'd like to recommend that we talk to Osmosis and copy exactly what they're doing. I talked to them about it recently.

---

| moniker | CPU | go version | TM | Expected | Got | Round | iavl-disable-fastnode setting | minimum-gas-prices |
| -------- | --------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| Nodes.Guru | AMD | 1.19.3 | ? | 3F5...114 | E9D...713 | 144 | false | 0ujunox
| jabbey | AMD | 1.19.5 | Default | B54...976 | 198...768 | 49 | false |
| ZenChainlabs | Intel | 1.19.5 | Default | EE2...721 | F5D...1DE | false | 0ujunox |
| Stakely.io | AMD | 1.19.4 | Default | B37...826 | F5D...1DE | 65 | false |
| kingnodes | Intel | 1.19.3 | Default | EE2...721 | ? | ?? | unset |
| needlecast | AMD| 1.19 | Default | F5D...1DE | EE2..721 | ?? | false |
| GoldenRatio | Intel | 1.19.2 | Skip | EE2...721 | F5D...1DE | ?? | true | 0ujunox |
| Highlander | AMD | 1.19.2 | Default | EE2...721 | E9D...713 | ?? |
| Disperze | AMD | 1.19 | Default | 198...768 | 593...2C0 | 66 | false | 0ujunox |
| Ezstaking | AMD | ? | ? | ? | ? | ?? |
| BlueStake | AMD | 1.19.3 | Default | AB4...408 | EE2...721 | 77 | false | 0ujunox
| Cros-testnest | AMD | 1.18.7 | Default | EE2...721 | 3F5...142 | 45 | false |
| windpower | Intel | 1.19.5 | Default | EE2...721 | 3F5...142 | 90 | false
| blockpane | AMD | 1.19.5 | Default | 5D6...4C0 | EE2...721 | 95 | unset
| Oni Validator | AMD | 1.19.2 | Skip | 1BA..05A | E9D...713 | 96 | true | 0ujunox |
| Silk Nodes | Intel | 1.18.2 | Skip | 92E...BE1 | E9D...713 | 96 | false | 0ujunox |
| WhisperNode | AMD | 1.19.5 | Default | EE2...721 | F26...753 | ?? | false |
| Polkachu | AMD | 1.19.3 | Default | EE2...721 | E9D...713 | 96 | false | 0.025ujunox |
| SpacePotato | Intel | 1.19.5 | Default | 7F0...3CA | E9D...713 | 96 | false |
| BitValidator | AMD | 1.18 | Default | F26...753 | E9D...713 | ?? | false |

# Summary of things It Ain't™️

- it ain't AMD vs Intel
- it ain't go version
- it ain't Skip vs default
- it ain't iavl-disable-fastnode config

## Nodes.Guru
- go1.19.3
- Expected 3F55999CF91714FB92462F67DCB4DA7CECE773BB2355A9F5FB20B3DD6785114
- Got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713
- TM Default
- CPU AMD Ryzen 5 3600 6-Core Processor

## Jabbey
- go1.19.5
- Expected B547040366BCBD7A5EEA962ED55AF179399FA0B19F3E97A14A42E3F4C3A23976
- got 198BA992B760DD9A5A7D240D438934775D3CBFAA0767E3A7B4A9850ABEA1D768
- TM Default
- AMD Ryzen 9 5950X 16-Core Processor

## ZenChainlabs
- go1.19.5
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- got F5DBE0E877206D2BF73E8530387A97F3D0672E81DD5314925D5F152261A7C1DE
- Default
- Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz

## Stakely.io
- go version go1.19.4
- Expected B3792DC2B08FFA5A9AED583EA938AD4D14D2CFDAF4FFDBD164CC27A527E27826
- got F5DBE0E877206D2BF73E8530387A97F3D0672E81DD5314925D5F152261A7C1DE
- TM default
- AMD Ryzen 9 5950X 16-Core Processor

## goldenratio

- go1.19.2
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- got F5DBE0E877206D2BF73E8530387A97F3D0672E81DD5314925D5F152261A7C1DE (this changes per round)
- TM Skip
- Intel(R) Xeon(R) E-2386G CPU @ 3.50GHz

## Oni
- go1.19.2
- Expected 1BA78B25A270770F52A3B568C1F372EA3C4CD5622AA12067EF35E105B879D05A
- got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713 (this changes per round)
- TM Skip
- AMD Ryzen 7 3700X 8-Core Processor

## needlecast

- go1.19
- Expected F5DBE0E877206D2BF73E8530387A97F3D0672E81DD5314925D5F152261A7C1DE
- got EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- TM default
- AMD Ryzen 5 5600X 6-Core Processor

## Haggis
- go1.19.2
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- got 198BA992B760DD9A5A7D240D438934775D3CBFAA0767E3A7B4A9850ABEA1D768
- Tendermint default
- AMD EPYC 7282 16-Core Processor

## BlueStake
- go1.19.3
- Expected AB4DDB01E42C74407324AE453E1A9ACBBD1834F34F1DE7CFEE59E35CAFE4A408
- got EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- Tendermint default
- AMD EPYC 7282 16-Core Processor

## windpowerstake
- go1.19.5
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- got 3F55999CF91714FB92462F67DCB4DA7CECE773BB2355A9F5FB20B3DD67851142
- Tendermint default
- Intel i3-12100 4-Core Processor

## blockpane
- go1.19.5
- Expected 5D6FBBC529B421B64377DCF8E364D6FEF690F8278B1EE2DD8475219B2CBE74C0
- got EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- Tendermint default
- Ryzen 9 7900X 12 cores

## Silk Nodes
- go1.18.2
- Expected 92EC086E21AFBF6B5D7E4851D38CC3E10FE70AC19D6F4AACB662DE9983CA4BE1
- Got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713
- Tendermint Skip
- Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz

## WhisperNode
- go1.19.5
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- Got F26A670A202522819AD3E50A9BDBB60996B23B1FA0B920CBD0EC82E371700753
- Tendermint default
- AMD Ryzen 9 5950X 16-Core Processor

## Polkachu
- go1.19.3
- Expected EE232DECFDCD0832B08803B12507C01F2D272D59BCE5B1CA6E17E93D2632A721
- Got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713
- Tendermint default
- Ryzen™ 5 3600 CPU

## SpacePotato
- go1.19.5
- Expected 7F0C96B6F0A0A763C60549D16605D5C365BD595EC9E8162069DEE9D30E0453CA
- Got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713
- Tendermint default
- Intel Xeon

## BitValidator
- go1.18
- Expected F26A670A202522819AD3E50A9BDBB60996B23B1FA0B920CBD0EC82E371700753
- Got E9D8BF94AC69102721BBF807F9D61F6F3218A802346BE0B5A11AFFB2A78A2713
- Tendermint default
- AMD EPYC 7B12