WIP - notes on potential improvements on the Substrate testing framework.
**Goals**
- Easy to use/set up for any developers;
- Less verbosity;
- Try keeping the magic at a minimum
- Backwards compatibility -- developers don't need to use the new framework/helpers to write tests. This should help with the added "magic", since developers are not obliged to use it.
### 1. macro helpers
- Implement macros or generate code that allows tests to call that simplify most used calls, e.g. `fn next_block`
e.g.
```rust
pub fn next_block() {
SafeMode::on_finalize(System::block_number();
Balances::on_finalize(System::block_number();
System::on_finalize(System::block_number());
System::set_block_number(System::block_numbe() + 1);
System::on_initialize(System::block_number();
Balances::on_initialize(System::block_number());
SafeMode::on_initialize(System::block_number());
}
```
could be generalize for any mock runtime and provided as a macro `next_block!()`
- transactional rollback macros (by @ggwpez in [this PR](https://github.com/paritytech/substrate/pull/12092/files#r1180496982))
```rust
/// Do something hypothetically by rolling back any changes afterwards.
///
/// Returns the original result of the closure.
macro_rules! hypothetically {
( $e:expr ) => {
frame_support::storage::transactional::with_transaction(
|| -> TransactionOutcome<Result<_, sp_runtime::DispatchError>> {
sp_runtime::TransactionOutcome::Rollback(Ok($e))
},
)
.expect("Always returning Ok; qed")
};
}
/// Assert something to be would be [*hypothetically*] `Ok` without actually doing it.
///
/// Reverts any storage changes made by the closure.
macro_rules! hypothetically_ok {
($e:expr $(, $args:expr)* $(,)?) => {
let result = hypothetically!($e);
assert_ok!(result $(, $args)*);
};
}
```
### 2. Test externalities
- Make test externalities builder implementation less verbose by providing a sensible default (that is flexible wrt to the runtime) that can be easily extended.
- Generate helpers for setting storage types in the test externalities builder for all pallets in the mock runtime.
### 3. `try_state` call out of the box after every test
If the test externalities code (builer, etc) would be generated with macros, it would be useful to include running `try_state` hooks in all pallets of the mock runtime after each test.
- Related to [Automatic checking try_state hooks in tests #13463](https://github.com/paritytech/substrate/issues/13463)
### 4. Simplify `frame_system::Config` and other impls
How could we make implementing the config of certain/all pallets (system, balances, etc) less verbose by defining sensible defaults that can be easily re-written by the dev?
### 5. Automatically generate benchmarks from annotated test
Long shot, worth looking into as it would ease one of the biggest pains for FRAME developers (benchmarking).