**Goals:**
- [ ] Quick overview of the weight system in Substrate: why they are required, how are they used in practive (benchmarking is out of scope);
- [ ] `frame_executive`: How does the weight refund work? Does it override the dispatchable pre-defined max weight?
- [ ] `frame_executive`: What happen when the refunded weight is larger than the pre-defined max weight?
- [ ] `frame_executive`: How and where is the fee calculated in the executive module?

---

- Currently, the `fn do_withdraw_unbonded` returns a `DispatchResultWithPostInfo` which encodes the result of a dispatchable with *post dispatch information*, i.e. it sends to the caller some additional information that is only known after the dispatch has been made. In this case, the `postinfo` is the real weight consumed by the function, to allow the caller (in the extrinsic path) to refund unused weight (which is set by the worst case benchmarks). The `DispatchResultWithPostInfo` is transformed from a `Weigth` using the `from/into` implementation for the dispatch result.
- However, it makes sense for the `fn do_withdraw_unbonded` to return the real weight it took to run the computation or an error, which consists of a `Result<Weight, DispatchResult>` or `Result<Weight, DispatchError>`. There are a couple issues with returning these results instead of a `DispatchResultWithPostInfo`:
- 1. If the computation fails, the caller needs to know how much weight the errorer computation has taken, which doesn't seem to be trivial to pass to the caller site when the `Err(DispatchError)`/ `Err(DispatchResult)` is passed.
- 2. `DispatchResult` is deprecated and the `DispatchResultWithPostInfo` should be used instead (as per the docs).

- One solution is to encode the used weight in the `DispatchResultWithPostInfo<PostDispatchInfo>`, where the post dispatch info is the amount of weight used in the computation, instead of how much it did not consume (as the current PR has). In this case, the caller still needs to know that the consumed weight has to be refunded (as it should, actually).

- So it seems that the current PR has a bug: instead of returning the weight consumed by the `fn do_withdraw_unbound`, it returns the how much it has not consumed in certain cases (in general, this doesn't make any sense).

- Remember: the `fn do_withdraw_unbonded()` is used by two extrinsics: `fn unbond()` and `fn withdraw_unbonded()`, thus the callers should handle the returned `DispatchResultWithPostInfo` appropriately to refund the weight if required.

#### Weight Refunding in Substrate

- The dispatchable functions in Substrate require the developer to specify a default weight through the annotation `#[pallet:weight(X)]`. The default dispatchable weight should be a reasonable value based on the heavy operations that occur in the dispatchable, e.g. storage access and computation.
- Sometimes, the computation of a dispatchable depends on the inputs or the storage state. In the example bellow, the computation required to run the extrinsic will depend on the user input.

#[pallet::weight(??)]
pub fn do_something(
	origin: OriginFor<T>,
	user_input: usize,
) -> DispatchResult {
	
	if user_input == 0 {
		// do nothing
	} else if user_input == 1 {
		T::do_1_write();
	} else {
		T::do_100_reads_and_writes();
	}
	
	Ok(())
}

- Since it is only possible to know the consumed resources at runtime, it is not trivial for the developer to know what's the correct default dispatchable call.
- This is where the Substrate benchmarks [link]() come into play. The idea of benchmarking is to allow the developer to calculate a sensible max weight for the callable. The benchmarking allows the developer estimate the weight costs of a callable, given multiple permutations of inputs and storage state.
- Benchmarking is a powerful tool for FRAME developers and one tool that every developer ends up working with a lot. For more info on how to generate and run benchmarks for pallets, there are a bunch of useful resources online [here](), [here]() and [here]().

**Weight refunds**

- While it is possible to "encode" different execution paths and storage states in the default weight with benchmarking, there are cases when the difference of computation required for different execution paths is very large and charging the default weight to the caller may be unfair. Consider the example 1. again: the difference between the operations/weight required for the a `caller_A` that calls the extrinsic with user input is `1` and `caller_B` that calls the extrinsic with input `10` is very large. It would make sense for the `caller_A` not to be changer with the default weight of the callable.
- Fortunately, Substrate has the capability for weight "refund" post dispatch, where the default weight can be re-set after the callable has ran.
- The weight refunding logic is pretty straightforward: instead of returning a `DispatchResult`, the callable can return a `DispatchResultWithPostInfo` where the `PostInfo` is the real weight used by the call.
- Note that the refund in bounded by the default weight, so defining a sensible default weight with benchmarking is still relevant as it sets the maximum weight that a callable requires. More importantly, the dispatcher will look at the default weight pre-dispatch to decide whether the transaction fits in the block being produced. Thus, developers still need to run benchmarks for the pallet callables to signal the dispatcher how much computation (time and space?) is required to execute the transaction prior to dispatch.
- The example 1. can be re-written with weight refunding as follows

#[pallet::weight(T::WeightInfo::do_something())]
pub fn do_something(
	origin: OriginFor<T>,
	user_input: usize,
) -> DispatchResultWithPostInfo {
	
	let used_weight = if user_input == 0 {
		// do nothing, return no weight used
		None
		
	} else if user_input == 1 {
		T::do_1_write();
		Some(T::DbWeight::get().writes(1))
		
	} else {
		T::do_100_reads_and_writes();
		Some(T::DbWeight::get().reads_writes(100, 100));
	}
	
	Ok(used_weight.into())
}

### `DispatchInfo` and `PostDispatchInfo` structs

Each callable has an associated `frame_support::dispatch::DispatchInfo` which is automatically generated (how? confirm) based on the callable's `#[pallet::weight]` annotation. The `DispatchInfo` bundles the default information about the weight of the callable, the dispatch class and whether calling the dispatchable should result on fee payment:

/// A bundle of static information collected from the #[pallet::weight] attributes.
pub struct DispatchInfo {
    pub weight: Weight,
    pub class: DispatchClass,
    pub pays_fee: Pays,
}

Each pallet dispatchable returns either a `DispatchResult` or a `DispatchResultWithPostInfo` (what enforces this? the expanded on macro or something else?). The final weight of executing a transaction that returns `DispatchResult` is the default weight defined in the `#[pallet::call]` annotation. On the other hand, the `DispatchResultWithPostInfo` will signal the dispatcher how much weight was actually used by the callable.

The `PostDispatchInfo` encapsulates the information about post dispatch weight updates. The field `actual_weight` should have the actual post weight that the callable used and should be explicitly set by the developer when a weight refunding is necessary. `PostDispatchInfo` has a implements a couple of helpful methods that are used in the block execution flow:
- `fn calc_unspent()`, which is ...
- `fn calc_actual_weight()`, which is ...

/// Weight information that is only available post dispatch.
/// NOTE: This can only be used to reduce the weight or fee, not increase it.
#[derive(Clone, Copy, Eq, PartialEq, Default, RuntimeDebug, Encode, Decode, TypeInfo)]
pub struct PostDispatchInfo {
	/// Actual weight consumed by a call or `None` which stands for the worst case static weight.
	pub actual_weight: Option<Weight>,
	/// Whether this transaction should pay fees when all is said and done.
	pub pays_fee: Pays,
}

impl PostDispatchInfo {
	/// Calculate how much (if any) weight was not used by the `Dispatchable`.
	pub fn calc_unspent(&self, info: &DispatchInfo) -> Weight {
		info.weight - self.calc_actual_weight(info)
	}

	/// Calculate how much weight was actually spent by the `Dispatchable`.
	pub fn calc_actual_weight(&self, info: &DispatchInfo) -> Weight {
		if let Some(actual_weight) = self.actual_weight {
			actual_weight.min(info.weight)
		} else {
			info.weight
		}
	}
    
    // ...
}

### `CheckWeight` signed extension

> (answer https://substrate.stackexchange.com/questions/3963/unsigned-transactions-vs-signed-extensions)

Signed extensions are used to extend the data and logic associated with transactions.

`construct_runtime!()` takes in the `Block` type. In the default polkadot node implementation, the block type is a `generic::Block` that is generic over an `Header` and an `UncheckedExtrinsic`. The `UncheckedExtrinsic` is generic over the `SignedExtra` which is a type that is a tuple over all the signed extensions to be applied on the unchecked extrinsic. In addition to `UncheckedExtrinsic`, there are two other types `SignedPayload` `CheckedExtrinsic` that are generic over the `SignedExtra` tuple.

pub type SignedExtra = (
	frame_system::CheckNonZeroSender<Runtime>,
	frame_system::CheckSpecVersion<Runtime>,
	frame_system::CheckTxVersion<Runtime>,
	frame_system::CheckGenesis<Runtime>,
	frame_system::CheckEra<Runtime>,
	frame_system::CheckNonce<Runtime>,
	frame_system::CheckWeight<Runtime>,
	pallet_asset_tx_payment::ChargeAssetTxPayment<Runtime>,
);

Going a bit lower down the stack,

### Weight refunding and transaction fitting from the Executive POV

The block weight calculation is performed when the block is executed by the Substrate node at runtime. The block execution is performed by something that implements the `ExecuteBlock` trait, which in the case of most Substrate runtime implementations is the `Executive` struct defined in the `frame_executive` module. In the following section we'll explore how the block weight is calculated based on the block's transactions and operations running on hooks such as `on_initialize` and `on_finalize`. We'll also check how the extrinsic return objects (`Dispatched` and `DispatchResultWithPostInfo`) are handled by the `Executive` to account for the final block weight.

#### Frame `Executive` and extrinsic weight bookeeping
- The Substrate [Executive module](https://paritytech.github.io/substrate/master/frame_executive/index.html) defines how the extrinsic dispatches are handled by the runtime. The weights play two important roles in the executive logic:
- 1. The executor checks if an incoming extrinsic call fits in the current block given its estimate default weight (pre-dispatch)
- 2. The executor "refunds" the weight based on the returned `DispatchResultWithPostInfo` of the extrinsic call (?)
- 3. The executor calculates the extrinsic fees based on the post-dispatch weight (?)

In the remaining of this article we'll check how the executive module uses the extrinsic weight information and post dispatch information internally at runtime.

> TODO(gpestana): the above may need to be revisited

- The `Executive` struct is the main entry point for the executive logic, namely the `fn execute_block()`. The `Executive` has a few generic parameters, which the most important one for us not is the `AllPalletsWithSystem`. This generic parameter is a tuple with all the active pallets in the runtime and the system's pallet. The `Executive` use the `AllPalletsWithSystem` trait implementor to call into the pallet's hooks, e.g. `on_initialize`, `on_finalize`, etc. (show `Executive` struct definition).
- The `Executive` implements `ExecuteBlock` trait, which allow it to execute all the extrinsics in a given block and checking for the validity of the final header.

The entry point for a new block is `Executive::execute_block(block: Block)`, which performs the following actions (roughly, I will gloss over some stuff that is not relevant to this discussion):

1. **Initialize the block**. `pub fn initialize_block(header: &System::Header)` is called in the beginning of every block execution. There are a few weights that are accounted at block intialization, namely weights related to a potential runtime upgrade, the weight associated with all the `on_initialize()` calls for all the pallets in the runtime and the `base_block` weight. All these initial weights are registered in the system pallet by calling `frame_system::Pallet<System>::register_extra_weight_unchecked(weight)`
- Note at `initialize_block`, the weights accounted for are not related to extrinsics yet, but it is important to reserve the weights required to run the `on_initialize` calls in all registered pallets and to account for the `base_block` weight. These weights are registered with the `DispatchClass::Mandatory`, which allows the weights to be registered even if if the total weigth surpasses the block max weight. If that's the case, there will be no more space for extrinsics to be added to the block. Similarly, the weight of transactions with `DispatchClass::Mandatory` are not checked by the weight signed extension that checks whether the a transaction queued in the mempool can fit in a block.

2. **Initial checks**: check if the block header is correct based on the parent hash and ensures that inherents are first to be executed in the block. (The default `Block` implementation used in Substrate is a struct with a header and a vector of extrinsics. Inherents should be at the head of the extrinsics vector).

3. **Execute extrinsics and weight bookeeping**: This step of the block execution is the most important when it comes to extrinsics weight management. The call to `execute_extrinsics_with_book_keeping(extrinsics, _);` will execute the extrinsics in the block and perform the post dispatch accounting of weights.

/// Execute given extrinsics and take care of post-extrinsics book-keeping.
fn execute_extrinsics_with_book_keeping(
    extrinsics: Vec<Block::Extrinsic>,
    block_number: NumberFor<Block>,
) {
    extrinsics.into_iter().for_each(|e| {
    if let Err(e) = Self::apply_extrinsic(e) {
        let err: &'static str = e.into();
            panic!("{}", err)
        }
    });

    // post-extrinsics book-keeping
    <frame_system::Pallet<System>>::note_finished_extrinsics();

    Self::idle_and_finalize_hook(block_number);
}

- There are a couple of corolaries from the fact that the extrinsics are applied sequentially and the fact that the post-extrinsic book-keeping is done only after all the extrinsics were executed *successfully*, namely:
- corolary 1.

4. **Final checks**: performs a few checks on the final header of the block.

#### Dispatchable flow and extrinsics weight bookkeeping

1. At block initialization, the dispatcher calculates how much weight each runtime's pallet will use `on_initialize`.
2. `on_intialize` weights are registered on the block (where/how?)
3. For every inherent, unsigned transaction and extrinsic in the block:
- Get the callable's `DispatchInfo`
- Apply the callable by running the computation encoded in the extrinsic;
- Calculate the actual weight of the extrinsic based on the associated `DispatchResult` and the `DispatchResultWithPostInfo` returning from a successful extrinsic apply
- The actual weight of the extrinsic is calculated by selecting the minimum between the `dispatch.weight` and `postInfo.weight`.

- ❓ [ ] The Executive's `initialize_block()` pre-emptively registers the weights of all the runtime pallet's `on_initialize` hooks (and the `base_block` weight). The `on_intialize` dispatch call is of type `Mandatory`, which means that the weights will be registered regardless even if it exceeds the `max_block` weight. What happens if one or more pallets fill the `max_block` on initialize? The runtime will never be able to execute any transactions, right?