# Transaction Payment \- Hold and Release # TL;DR Hardcoding the maximum deposit a contract can consume turned out to be impractical, because it is inflexible, results in too large gas estimates, and distorts the reconstituted weight when Eth Wallets adjusts the estimate. We can get rid of this while preserving all the other desired properties by changing how transactions are charged — by instead returning a gas estimation with the actual storage deposit to the user (i.e. lower and more accurate gas estimation), and during execution hold the transaction fee submitted and then release the remainder (instead of withdrawing and refunding it). This is a global change outside of pallet-revive but that it is worth it as it gives us accurate cost estimation. I also argue that we want to make this global change instead of having a different charging algorithm just for contracts. It would increase the maintenance burden to take care of different ways of doing it. As a bonus this will also allow paying transaction fees in different assets than the native currency (DOT/KSM). # Background/Context In [pallet\_revive Gas Mapping Design Doc](https://docs.google.com/document/d/1iGy0hHSZ9-LMlaXFZrHoyzqtLcK2nBgnn5JYmvoCFRU/edit?tab=t.0), we discussed how to map between Weight (ref\_time, proof\_size) \+ storage\_deposit into the one dimensional Ethereum gas. We do that so that we have an accurate display of fees in Ethereum wallets which only support gas. When a wallet estimates the required gas for a smart contract transaction, it will use pallet-revive to dry run the transaction. Dry running will determine the actually consumed weight and storage deposit. The weight and storage deposit will then be converted into a single gas amount that is sufficient to pay for both the transaction fee and the storage deposit. This gas amount will be returned to the wallet. The wallet will use this gas amount (plus some extra margin) as the gas limit of the actual Ethereum transaction it submits. Because we only receive the one dimensional gas inside the Ethereum transaction we cannot know what part is meant to be Weight (i.e. transaction fee \= block space) and what part is storage deposit (i.e. payment for disk space). But we need to know since we need to transform the Ethereum transaction into a substrate extrinsic with its proper Weight attached. We cannot just charge the whole gas as Weight because then it will be pre-charged as tx fee by pallet\_transaction\_payment and will potentially not be available when pallet\_revive tries to collect the deposit. This is why we originally decided to just hard code the storage deposit limit. This makes it a constant which allows us to know how much of the gas is Weight. However, after playing around with the constants we came to the conclusion that a limit is impractical: - A limit that is high enough to allow for every sensible transaction to succeed will dominate the fees for most transactions. We are always overestimating the costs massively or preventing some transactions from going through. - If the constant dominates the gas estimation it has a nasty interaction with wallets that add some extra gas: They add 50% extra gas. But this 50% also includes the big constant increasing the Weight part by much more than intended. - With the new upper limit of 1MiB contracts we would need to massively increase the max deposit to be able to accept such contracts. It will then affect everyone who deploys contracts and potentially every caller. - It is extremely difficult to tune the numbers correctly and makes the system less flexible **This is why in this document we propose a solution that does not require picking a constant but can return a gas estimation with the actual storage deposit to the user.** # Scope ## In-scope ### pallet-transaction-payment \+ pallet-asset-conversion-tx-payment Those will be changed to hold and then release the transaction fee instead of withdrawing and refunding it. The non refunded fee will be handled in exactly the same way as before (usually a split between burning and treasury funding). The latter pallet builds on top of the former in order to allow paying transaction fees in different assets than the native currency (DOT/KSM). ### pallet\_revive Will be changed to take the storage deposit from the balance put on hold by the transaction payment pallets. Currently, it just takes it from the free balance of the native balance. A neat side effect will be that the storage deposit will then also be paid in the fee paying asset. Hence `pallet_revive` contracts can be called without owning DOTs (USDT is enough). This is because `asset-conversion-tx-payment` is converting the user selected asset into the native asset for fee payment so that `pallet-revive` can keep only dealing with the native asset for deposits. ## Out-of-scope ### pallet-asset-tx-payment This seems to be the predecessor of pallet-asset-conversion-tx-payment. It is not in use on AssetHub. This pallet will not be supported to use with pallet\_revive. # Proposed Solution ### High-Level Architecture We change the transaction payment pallets to put the pre dispatch payment for the transaction into a named hold and then afterwards release the remainder. This allows extrinsics to progressively subtract fees from this hold during extrinsic execution. Essentially dynamically sharing the overall costs between deposit and transaction costs. After the extrinsic is run we know its exact weight and hence the refund (if any) the signer of the extrinsic should receive. We then apply the following logic inside `TransactionExtension::post_dispatch_details`: ```rust let refund = hold_leftover.saturating_sub(actual_weight_fee); if actual_weight_fee > hold_leftover { // Extrinsic used fees that were collected for tx payment as // deposit. We essentially did not collect enough fees. But // we cannot error out at this point. See explanation below. // pallet_revive will make sure that this never happens } // actual_weight_fee = Fee according to the post dispatch weight // hold_leftover = What is left inside the named hold ``` **An important point here is that we cannot roll back a transaction at this point. What can happen is that the extrinsic took so much deposit that we cannot pay for the actual weight anymore. An extrinsic essentially got the Weight for free in this case. However, the fees are always the maximum of the Weight fee and deposit fee.** **A pallet that withdraws balance from the hold needs to make sure that it always leaves enough balance in the hold to pay for the transaction fees. Ideally, this would be prevented by `pallet-transaction-payment` by rolling back if not enough balance is left. However, the current transaction extension design does not allow for that. It can only declare the transaction as invalid and not include it.** **This is why we will put this logic inside `pallet_revive`. This is technical debt we will get rid of later. It will work like this:** 1) **At the end of a contract execution pallet-revive calculates how much transaction fee was actually needed for the execution.** 2) **If there is not left enough in the hold it will fail (but include) the transaction. The storage changes (including the deposits taken from the hold) will be rolled back.** 3) **The signer is charged a tx fee according to the real weight.** The debt can later be removed by changing the `trait TransactionExtension` to be able to fail (but include) an extrinsic. It would require a larger refactoring and hence we are not doing it at this point. ### Detailed Design #### pallet-asset-conversion-tx-payment We only discuss changes to `pallet-asset-conversion-tx-payment` here. The changes to the less complex `pallet-transaction-payment` are analog. In case a runtime is using the latter, replace `SwapAssetAdapter` with `FungiblesAdapter`. The actual charging of the tx fee is not done by `pallet-asset-conversion-tx-payment` itself. It expects the runtime to implement a trait (`OnChargeAssetTransaction`) to achieve this. In practice, runtimes will not implement this themselves but use the `SwapAssetAdapter` provided by `pallet-asset-conversion-tx-payment.` This is where we do our actual changes. As of right now it uses `pallet-asset-conversion` in order to swap the asset the user wants to pay a fee into the native asset. It withdraws the asset and then gets a credit of the native asset but never deposits it. Post dispatch it swaps the leftover back into the asset the user originally paid in. We change the behavior to deposit the credit as a hold balance pre-dispatch. It is a named hold that cannot be mixed up with other held balances. This makes this held balance available to any pallets. Post dispatch we apply the logic outlined in the high level design. We don’t make any changes to the swapping logic. The sequence of events inside the `pallet-asset-conversion-tx-payment` will be: 1. Withdraw the fee paying asset from the transaction signer’s account. 2. Convert that to the native asset and add it as held balance onto the signer’s account. 3. Execute the extrinsic which might chip away from the held balance. 4. Convert the leftover (see code block above) back to the fee paying asset and refund it as free balance. #### pallet-revive Currently, we just collect the storage deposits from the free balance of the tx signer. We change this behaviour to collect it from the hold balance using the name (reason) given by the above trait. At whatever point there is not enough held balance left we just roll back the whole tx (just as we do now). This requires `pallet-revive` to learn about the name of the hold that those funds are held in. So we add a config knob that we expect the runtime to fill: ```rust type Config { ... type TxPaymentHoldReason: Get<T::RuntimeHoldReason>; } ``` Runtimes that use a custom `OnChargeAssetTransaction` implementation won’t work with `pallet-revive` unless they implement the same hold and release logic. It just fails at runtime when trying to collect the deposit. Nothing dangerous can happen. In practice none of the relevant runtimes do. Enforcing this at compile time will be hard as there is not a single pallet-transaction-payment we could add bounds on. With this solution we can get away with no additional tight coupling. This of course requires that the storage deposit was collected as a Weight fee. This means that the Weight is increased by the storage deposit so that the tx fee will cover the deposit too. The extrinsics no longer will take a storage\_deposit\_limit but operate in a “leftover” mode. They just take from the hold and roll back when they run out. We will do that for all extrinsics. Not only for the eth ones. Otherwise we won’t be able to profit from paying deposits in any fee paying asset. ### Risks - We change the behavior for all runtimes that use `SwapAssetAdapter` or the `FungiblesAdapter` for all transactions. Any flaw in the logic or implementation will have wide reaching effects. - The hold and release could add more overhead to transaction payment as it may add more storage accesses. But the holds should be stored inline with the free balance and no additional accounts are touched. Benchmarking will show though. - The pre-dispatch weight will include the deposit making the extrinsic appear heavier than it is because it includes the storage deposit as weight. This will be corrected post-dispatch though. Hence it will only slightly affect filling blocks up optimally. We will also exclude the code upload deposit from the storage deposit as those would blow up the weight too much. But only developers and not users are affected by this additional deposit not covered by the displayed fee. Something that can be documented. ## Rejected options ### Don’t charge a deposit at all Just don’t charge anything for deposits. Discarded because this will leave the chain vulnerable to storage bloat attacks. The weights are chosen to only account for computation. So filling up the disk space cheaply will be trivial. [Frontier already got attacked and only recovered because the attacker decided to stop.](https://forum.moonbeam.network/t/solving-the-xen-spamming-issue/1689) ### Charge deposit from the weight meter This is essentially what frontier is doing. But you will run into massive tuning problems when conflating resources: Charging additional computation for disk space. If you choose the number too low then you will be attackable (see above). If you choose it too high you are wasting block space. ### Don’t mention the storage deposit at all to the Ethereum wallets This is essentially what all the other pallets are doing. The tx fee of any other extrinsic just includes the weight. If the pallet is taking any deposit it is just taken and the user is left wondering why more balance disappeared from their free balance. For fixed functions this might be acceptable. But for contracts the users rightfully expect that there are no extra charges. Especially since Ethereum wallets only see free balance and nothing else. # Open Questions - Is it okay to make this change globally or should we only apply this to revive transactions? It would be complicated to special case. But it might be worth it if the performance impact is there. - Does the hold and release have a performance impact? # Future Work/Enhancements It might be worthwhile to make this behaviour more fool proof by encoding in the type system. For example, by sealing the `OnChargeAssetTransaction` trait.