**related to**: https://github.com/paritytech/polkadot-sdk/issues/3245
**PR addressing this issue**: https://github.com/paritytech/polkadot-sdk/pull/3639
**v1.1.3 hotfix release notes**: https://hackmd.io/df-_KPMCTFiH0gENgA2LlQ?edit
**Kusama hotfix release (v1.1.3)**: https://kusama.polkassembly.io/referenda/359
**Polkadot hotfix release (v1.1.3)**: https://polkadot.polkassembly.io/referenda/581
see [Ledger recovering](#Ledger-recovering) for next steps.
### Problem statement
Some ledgers have an unexpected `Bonded` and `Ledger` states across all chains (but not necessarily inconsistent/critical at this point in time). There are some ledger controllers which are stashes of *another* ledger. This is not a problem per se, but it may lead to inconsistent storage states when calling `set_controller` or deprecate the controller in such a state. It can also lead to a bonded tuple `(stash, controller)` ending up not having an associated ledger (as seen in the `check_payees` issue https://github.com/paritytech/polkadot-sdk/issues/3245).
> Note: currently, it is not possible to bond or set a controller with an account that is different from the ledger's stash. However, there are ledgers in both Polkadot and Kusama which have these characteristics, which was probably made possible from past staking logic.
Imagine the following scenario:
```
// Bonded<T> map state:
Bonded(A) = B; // stash A has controller B
Bonded(B) = C; // B is also a stash of another ledger, with controller C
Bonded(C) = D;
// Ledger<T> map state:
Ledger(B) = SL_a // staking ledger with stash A is controlled by B
Ledger(C) = SL_b
Ledger(D) = SL_c
```
The current `set_controller` code is the following:
```rust!
fn set_controller(stash) {
let controller = Ledger::<T>::get(stash).unwrap().controller();
<Bonded<T>>::insert(&stash, &stash);
<Ledger<T>>::remove(controller);
<Ledger<T>>::insert(stash, ledger);
}
```
When we try to migrate the e.g. stash `A` to have the same controller as its stash, we have:
```
// set_controller(A)
Bonded::insert(A, A); // OK
Ledger::remove(B) // OK
Ledger::insert(A, ledger) // OK
```
However, if we try to migrate the stash `B` to have the same controller as its stash, we run into a problem:
```
// set_controller(B)
Bonded::insert(B, B) // NOK, since B is controller of A
Ledger::remove(C) // OK
Ledger::insert(B, ledger) // NOK since B is controller of A
```
Thus, when we have a controller that is also a stash of another ledger, we may run into critical storage inconsistencies when setting the controller of the ledger to its stash. `set_controller` calling order seems to matter here, but I think it's safer to assume that all double bonded ledgers should *NOT* be migrated with the current code.
#### `None` ledgers
As we've seen in Westend (https://github.com/paritytech/polkadot-sdk/issues/3245), we may end up with a tuple of bonded `(stash, controller)` accounts that is not associated with a ledger, ie:
```rust!
// given a bonded pair (stash, controller),
assert!(Ledger::<T>::get(&stash) == None && Ledger::<T>::get(&controller) == None); // this is true
```
This is a second-order effect of killing a ledger which *was* double bonded and the `set_controller` has been called on it, ie. to reproduce:
1. given `Bonded(A, B)`, where `B` is a stash of another ledger;
2. call `set_controller(A)`
3. call `kill_stash(A)`
### Solving the current inconsistencies
A potential solution is to force unstake/chill the ledgers that have a controller which is a stash in another ledger. This can be achieved by either:
0. merge https://github.com/paritytech/polkadot-sdk/pull/3639 to make sure that `set_controller` is called without ending up in inconsistent states, AND/OR
1. add a new condition to `chill_other` whereby a ledger with a controller that is a stash as well can be chilled by anyone, OR
2. explain this issue to the community and open a referendum to call `force_unstake` on the ledgers with the faulty state.
In any case, we should prevent the faulty ledgers to succesfully call `set_controller`. This is implemented by https://github.com/paritytech/polkadot-sdk/pull/3639.
### `try-runtime` improvements
The current `try-runtime` do not provide enough visibility over the ledger inconsistencies and the above described issues are currently latent in all chains. Only once the affected ledger is killed or migrated, the `try-runtime` checks will fail (as it happened [in this issue](https://github.com/paritytech/polkadot-sdk/issues/3245)). This PR [TODO]() adds invariant will to signal the inconsistencies beforehand:
- A bonded (stash, controller) pair should have only one associated ledger. I.e. if the ledger is bonded by stash, the controller account must not bond a different ledger (do not fail in this case, alert with warn. once there are no more ledgers with a faulty state, we can enable the fail).
- A bonded (stash, controller) pair must have an associated ledger.
- A bonded controller can only control *one* ledger at any given time.
---
### Example: "Double bonded" in Polkadot
An example of the "double bonded" occurrence in Polkadot have the follwoing characteristics:
```rust!
for (stash, controller) in Bonded::<T>::iter() {
if Ledger::<T>::get(&stash).is_some() && Ledger::<T>::get(&controller).is_some() {
// this happens and bonded ledgers are different.
}
}
```
In this case, the a controller of a ledger is a stash of a different ledger. This is not possible to happen anymore in the current logic (currently, with controller deprecation, a ledger will have only a stash associated, where controller is the same) and the ledgers in that state seem to have been bonded long time ago.
The break down of these cases per chain is in the section [Possible ledger and bonded states](#Possible-ledger-and-bonded-states).
An example of this occurrence in Polkadot:
```
- Bonded(stash, controller):
controller: 5ECEUHjCbsXSTjNuCBzYUBNi754JfgegqjNRzpo7J1FaAaXx (0x5e363dbb43c700e5915151b713b0dfa5aaa2cbb5bb79159cea844f4f840b4326)
stash: 5G22rfCwhkxKYiibJ3at51iHVgNWjRfTxBeSuEAtfbpYj7Xu (0xaee72821ca00e62304e4f0d858122a65b87c8df4f0eae224ae064b951d39f610)
- Ledger(controller):
controller_ledger:
StakingLedger {
stash: 5G22rfCwhkxKYiibJ3at51iHVgNWjRfTxBeSuEAtfbpYj7Xu,
total: 4423101721494,
active: 4423101721494,
unlocking: BoundedVec([]),
claimed_rewards: [...])
}
- Ledger(stash):
stash_ledger:
StakingLedger {
stash: 5EQp17cLvQegphHsfaoUMJ4nVvD3aizSpGXmrwZgA14Vwof1,
total: 20000000000,
active: 20000000000,
unlocking: BoundedVec([]),
claimed_rewards: BoundedVec([...])
}
```
### Current state across all the chains
Obtained with subxt https://github.com/gpestana/subxt-playground
**Polkadot**
- 64611 bonded stash/controller pairs, from which ledger is
- double bonded: 9
- stash bonded: 0
- controller bonded: 5156
- migrated: 59444
- none: (⚠️ TODO: re-run)
**Kusama**
- 23806 bonded stash/controller pairs, from which ledger is
- double bonded: 22
- stash bonded: 0
- controller bonded 5088
- migrated: 18684
- none: (⚠️ TODO: re-run)
**Westend**
- 72599 bonded stash/controller pairs, from which ledger is
- double bonded: 0
- stash bonded: 0
- controller bonded: 0
- migrated: 72583
- none: 16 (no associated ledger)
---
### Possible ledger and bonded states
1. **Ledger with controller** ✅
- given a tuple `Bonded[stash, controller]` and `stash != controller`:
- `Ledger(controller) != None`
- `Ledger(stash) == None`
- `ledger.stash == stash`
2. **Migrated ledger** ✅
- given a tuple `Bonded[stash, controller]` and `stash == controller`:
- `Ledger(controller) != None`
- `ledger.stash == stash | controller`
3. **Ledger with controller, double bonded** ⚠️⚠️⚠️
- given a tuple `Bonded[stash, controller]` and `stash != controller`:
- `Ledger::<T>::get(controller) != Ledger::<T>::get(stash) != None`
- 💣💣 this state is OK but a time bomb (calling `set_controller` may lead to inconsistent state and affect the ledgers).
4. **Lingering ledger for stash/controller tuple** ❌
- given a tuple `Bonded[stash, controller]`
- `Ledger(controller) == None`
- `Ledger(stash) == None`
- This state is the result of calling `set_controller` and `kill_stash` on a ledger which has a controller that is a stash of *another* ledger.
---
## Ledger recovering
#### Polkadot
state as of 18th March
```
🙈 🙉 🙊 Duplicate controller found: 12gmcL9eej9jRBFT26vZLF4b7aAe4P9aEYHGHFzJdmf5arPi | stash 1: 13SvkXXNbFJ74pHDrkEnUw6AE8TVkLRRkUm2CMXsQtd4ibwq | stash 2: 12gmcL9eej9jRBFT26vZLF4b7aAe4P9aEYHGHFzJdmf5arPi
⚙️ ⚙️ ⚙️ is_validator controller: false | stash1 false | stash 2: false
🙈 🙉 🙊 Duplicate controller found: 138fZsNu67JFtiiWc1eWK2Ev5jCYT6ZirZM288tf99CUHk8K | stash 1: 138fZsNu67JFtiiWc1eWK2Ev5jCYT6ZirZM288tf99CUHk8K | stash 2: 12YcbjN5cvqM63oK7WMhNtpTQhtCrrUr4ntzqqrJ4EijvDE8
⚙️ ⚙️ ⚙️ is_validator controller: false | stash1 false | stash 2: false
```
#### Kusama
state as of 18th March
```
🙈 🙉 🙊 Duplicate controller found: EVe3hL6XumVKNyrjrjqK5LHNSETvM4djs25wMi9nYr5VMqv | stash 1: EVe3hL6XumVKNyrjrjqK5LHNSETvM4djs25wMi9nYr5VMqv | stash 2: ESGsxFePah1qb96ooTU4QJMxMKUG7NZvgTig3eJxP9f3wLa
⚙️ ⚙️ ⚙️ is_validator controller: false | stash1 false | stash 2: false
🙈 🙉 🙊 Duplicate controller found: EyXct79ZDWdQfcSgJTG5texKM9wJj3quyh1ugPDVSkSt3Xm | stash 1: DggTJdwWEbPS4gERc3SRQL4heQufMeayrZGDpjHNC1iEiui | stash 2: EyXct79ZDWdQfcSgJTG5texKM9wJj3quyh1ugPDVSkSt3Xm
⚙️ ⚙️ ⚙️ is_validator controller: false | stash1 false | stash 2: false
🙈 🙉 🙊 Duplicate controller found: E7o2xM99q6ckmFbKJmmmvVyKzs1FDEbLZHjKbPy7mMG9KKt | stash 1: E7o2xM99q6ckmFbKJmmmvVyKzs1FDEbLZHjKbPy7mMG9KKt | stash 2: Du2LiHk1D1kAoaQ8wsx5jiNEG5CNRQEg6xME5iYtGkeQAJP
⚙️ ⚙️ ⚙️ is_validator controller: false | stash1 false | stash 2: false
🎭 🆘 ⚠️ 🚧 Ledger overwritten for Controller EVe3hL6XumVKNyrjrjqK5LHNSETvM4djs25wMi9nYr5VMqv at block: 22215683 with hash: 0xbe6d95c70ca18835bb235ef49093b400c7025838467d6ee3a0c469051f193496
🎭 🆘 ⚠️ 🚧 Ledger overwritten for Controller EyXct79ZDWdQfcSgJTG5texKM9wJj3quyh1ugPDVSkSt3Xm at block: 22248236 with hash: 0x98998c058ef91549c67296e77bc5ee420418771626ef40a7b2aeb7e16465ae9a
🎭 🆘 ⚠️ 🚧 Ledger overwritten for Controller E7o2xM99q6ckmFbKJmmmvVyKzs1FDEbLZHjKbPy7mMG9KKt at block: 21623718 with hash: 0x776c959c9fd0e1ee166e70be727e85765b076cfbcab3a588c6e2b795e0722425
```
- [ ] when was last time that each corrupted stash lock has been updated? was it supposed to be updated?
Sign in with Wallet
Connect another wallet