Structure of the Stardust VM

# Structure of the Stardust VM ## Introduction *Stardust VM* is the next version of the ISC VM used in Wasp ver 0.3.x The *Stardust VM* brings significant changes to the previous version of the VM: * dust protection on L1 transaction. This brings about a lot of new dust deposit-related logic in the VM * gas, gas budgets, gas burning for SC calls. This caused a complete change of fee mechanism in the VM * it is based on new UTXO types: * *colored tokens* replaced with *native tokens*, *color* with *native token ID*. IOTA is not one of colors anymore * *foundries* introduced, minting of new tokens extended with concepts of supply control and token destruction function * *NFTs* as new request and asset type in the VM * a number of new validity bounds on transaction size etc The above results in significant changes of how we organize L1 ledger controlled by the chain, including getting rid of limitations on number of ledger elements, controlled by the chain, such as maximum number of different token IDs and maximum number of foundries. ## L1+L2 ledger *Stardust VM* calculates updates to L1 ledger, an *anchor transaction* and mutations of the key/value pairs of the L2 ledger. This way we can see it as one update of the ledger state of *L1+L2 ledger*. ### L1 assets *L1 assets* are **tokens**: iotas and native tokens (we leave aside special type of assets such as NFTs for now). In Stardust VM iotas and a vector of native tokens are treated as one data type *Assets*: <img src="https://i.imgur.com/gTDIj2W.jpg" width="350"> So, L1 assets is a pair of iotas balance plus mapping of tokenIDs to token balances: $(iotas, (tokenID \rightarrow {nativeTokens}))$ Each ISC chain keeps L1 assets in the *L1 account*, controlled by its *alias output*, i.e. in the *alias addres* (= *chain ID*) of the chain. These L1 assets we call **total assets of the chain**. We also call it *L1 account of the chain* and *L1 ledger of the chain*, as synonims. Each ISC chain also maintains *L2 ledger* of the chain in the key/value storage, the *state of the chain*. One may see *L2 ledger* as metadata of the *L1 account*. ### L2 accounts The *L2 ledger* consists *L2 accounts* (aka *on-chain accounts*) plus metadata of internal UTXO of the L1 account. Each *L2 account* contains iotas and native tokens belonging to the account. The account is identified by its *agent ID = (address, hname)*. One account in the *L2 ledger* is singled out and it is called *common account*. It's *agent ID = (chain ID, 0)*. The *common account* is used to collect chain-level funds, such as fees. ### L1+L2 ledger invariant L1 account of the chain is consistent with L2 ledger of the chain when: **assets on the L1 account = sum of assets on all accounts on the L2 ledger** or $$ assets(L1account) = \sum_{a \in L2accounts}{assets(a)} $$ The L2 ledger accounts contains split of all assets controlled by the chain among respective L2 accounts. This is a fundamental **ledger invariant** which cannot be broken under any circumstances. This makes **L1 account + L2 ledger = chain ledger** one ledger which is always be updated atomically to keep the ledger invariant valid. The consistency of L1 ledger of the chain with the L2 ledger of the chain (chain ledger invariant) is a fundamental property of the ISC chain. ### Maintaining consistency of L1+L2 The VM updates the chain ledger by **atomically** producing: * L1 transaction which updates L1 account. It reflects movements of tokens on L1 * update to the L2 ledger (mutations to the state), which reflects change of ownership of tokens among L2 accounts. The VM is responsible for producing those which always leads to the consistent state of L1+L2. The property of keeping commitment to the L2 ledger in each anchor transaction guarantees L1+L2 ledger remains consisent in subsequent updates of the ledger state. ### L1 account metadata in the L2 ledger state L2 ledger keeps all information which is necessary to reconstruct UTXOs of the L1 account of the chain. It means, anchor transactions which updates L1 account of the chain can be produced without querying UTXOs from the L1. ## Structure of the L2 ledger L2 ledger implements ledger of accounts. It is a mapping of *agentID* to *L1 assets*: $L2accounts = (agentID \rightarrow (iotas, (tokenID \rightarrow {nativeTokens})))$ of accounts. So, each account has: * *agentID*, an ID of the account. It is a key in the mapping * L1 assets: * number of iota tokens in the account * number of native tokens for each *tokenID* in the account The sender with the respective *agentID* controls all tokens in its account. There's special account on L2 ledger called **totalL2assets**. It always contains sum of all assets on L2 accounts. It is maintained by the VM internally. All key/value pairs which represents L2 accounts are kept in the *accounts* partition of the state (*accounts* core contract). The VM is updating L2 ledger with the following 3 internal functions in the *account* core contract (`accounts/internal.go`): * [CreditToAccount(agentID, assets)](https://github.com/iotaledger/wasp/blob/da6cde29f02e71102095573a2142990e508f57e9/packages/vm/core/accounts/internal.go#L135) adds tokens to the L2 ledger. Updates *totalL2Assets* accordingly * [DebitFromAccount(agentID, assets)](https://github.com/iotaledger/wasp/blob/da6cde29f02e71102095573a2142990e508f57e9/packages/vm/core/accounts/internal.go#L165) subtracts tokens to the L2 ledger. Rises exception if not enough tokens. Updates *totalL2Assets* accordingly. * [MoveBetweenAccounts(fromAgentID, toAgentID, assets)](https://github.com/iotaledger/wasp/blob/da6cde29f02e71102095573a2142990e508f57e9/packages/vm/core/accounts/internal.go#L214) moves Iotas and native tokens between accounts and *totalL2Assets* does not change. Rises exception if not enough balances <img src="https://i.imgur.com/izNnQTz.jpg" width="600"> In addition to accounts, the L2 ledger keeps data and ownership of all foundries owned by the chain. Each foundry is indentified in the ledger with its *serial number*. The *accounts* contracts maps ownership of each foundry in the ledger: $serialNumber \rightarrow agentID$ L2 ledger also maintains information of the internal UTXOs used to maintain L1 chain account (see below). ## Structure of the L1 chain account (L1 chain ledger) On L1 the chain keeps all tokens controlled by it in the UTXOs, controlled by its *alias address* (its *chain ID*). Note, that *alias address* may contain any other UTXOs, mostly sent by user as *on-ledger requests* to the chain. Those *other UTXO* comprise the *backlog of on-ledger requests*. UTXOs in the backlog are unlockable by the chain ID, however they are not a part of the *L1 chain account*. The L1 chain account consists only of UTXOs which are created by the chain itself, therefore all those outputs are known for the chain by its metadata on the L2. The chain *absorbs* funds brought by the UTXOs (requests) in the backlog and consolidates them into the the *L1 chain account*. The L1 chain account is a number of UTXOs, namely: * *AliasOutput* of the chain, also known as **anchor output**. The *anchor output* only contains iotas on it and never contains any of native tokens. Iotas in the *anchor output* are sum of *dust deposit* and *total iotas owned by L2 accounts*. The *dust deposit* constant for lifetime of the chain. **Deposit of iotas in the** *anchor output* **minus** *dust deposit* **is always equal to total iotas owned by the L2 accounts**. This is a ledger invariant. * one *ExtendedOutput* which keeps total of one native token ID accounted on the chain. It is known as **total token output**. Each *total token output* contains all the tokens of that *native token ID* on the chain plus minimum dust deposit in iotas on each such an output. **Number of tokens of specific** *tokenID* **contained in the total token output is always equal to the respective entry in the** *total assets* **in the L2 ledger**. This is a ledger invariant. * one *FoundryOutput* for each foundry owned by the chain. Chain creates those foundries in order to be able to mint and control supplies on native tokens on L1. Each such a foundry is owned by the L2 account. This fact is tracked by the L2 ledger. The foundry output always contains minimum dust deposit in iotas and does not contain any native tokens. * one *NFTOutput* per each NFT assets owned by some SC on the chain (**TBD**) <img src="https://i.imgur.com/4PPY2zM.png" width="600"> ## Token movement in L1+L2 ledger ### Requests and assets Each request to the smart contract has 3 properties (among others): **sender account**, **target account**, **assets**, **allowance** and **gas budget**. * **Sender account** in an *agent ID* of the originator of the request. It is always present in the request. * **Target account** is the target specified in the metadata of the request. If metadata does not allow correct parsing of the target, the target account is assumed to be the *common account*. * **Assets** are iotas and native tokens which are attached to the request and coming to the chain with the request. It is only possible if request is *on-ledger*, i.e. it is an UTXO. * *Assets* of the *off-ledger* request is always empty. **Off-ledger request do not modify total assets on the chain**. * In contrast, *on-ledger* request always contains at least minimum iotas of dust deposit, therefore **on-ledger request always modify total assets on the chain**. * **Allowance** is an optional specification of the budget of assets, which the target contract is alowed to withdraw from the `caller's` account. For the request, the *caller's* account is the `sender's` account. It is up to the target contract to withdraw allowed assets from the `caller's` account or not. The target contract can send allowed assets to any other account, including its own, but only within allowed budget. **Allowance does not modify total assets on the chain**. * **Gas budget** is a limit for the smart contract execution. It **has no impact on the total assets on the chain**. However, it regulates how fees are charged on the chain. <img src="https://i.imgur.com/74DzO0V.png" width="600"> ### New tokens coming to chain New L1 assets, iotas and native tokens, can only arrive to the chain as assets of the on-ledger request. **Assets of the on-ledger are always credited to the L2 account, controlled by the Sender of the request**. This is performed by [**creditAssetsToChain**](https://github.com/iotaledger/wasp/blob/3b22132ddae193e0c42dae647bf627e80f94f89b/packages/vm/vmcontext/runreq.go#L57) function of the VM. ### Tokens leaving the chain All L1 assets on the L1+L2 ledger are owned by some account. The L1 assets can leave the chain only by the authorisation of it's owner in the form of the *on-ledger* request to another chain or L1 address. This is performed by the [Send](https://github.com/iotaledger/wasp/blob/3b22132ddae193e0c42dae647bf627e80f94f89b/packages/iscp/sandbox.go#L87) function of the VM Sandbox. In that case assets are debited from the respective L2 account by [**accounts.DebitFromAccount**](https://github.com/iotaledger/wasp/blob/3b22132ddae193e0c42dae647bf627e80f94f89b/packages/vm/core/accounts/internal.go#L178) function. ### Minting/destroying native tokens Native tokens are just like iotas and other assets. However, new native tokens can be minted/destroyed by the foundries of the chain. Each foundry is owned by some smart contract. The smart contract owner can mint new tokens of the foundry and those new tokens are automatically credited to the L2 account of the foundry owner. The same applies to the destruction of native tokens: if smart contract has tokens of the foundry owned by it in its L2 account, it can destroy those tokens. This way the VM guarantees preservation of the L1+L2 ledger invariant. ### Dust deposit handling inside the VM Each UTXO in the L1 ledger must contain at least minimal dust deposit iotas. Therefore, the VM must take care about dust deposit for UTXOs created for the internal representation of the L1 chain account. The following rules are applied when handling dust deposits by the VM: * the dust deposit needed *total tokens outputs* is taken from the *common account* of the chain. So, whenever there's a need for dust deposit for a new native token, or the dust deposit is released (is not needed anymore), the "financing"is coming from the *common account*. As the *common account* is controled by the owner/governor of the chain, the owner has to ensure enough "operating capital" in the *common account*. If it is not enough iotas to create new total tokens output, the VM rises "not enough iotas for dust deposit" exception. * all other dust deposits for foundries, NFTs and outgoing on-ledger requests are taken from the sender's (requestor's) account * iotas in the dust deposits are excluded from the **totalL2Assets** and therefore are not accesible by smart contracts * to simplify the dust deposit handling logic, it is assumed that minimum dust deposit needed for the anchor and for one total tokens output is constant for the lifetime of the chain. Those values are calculated and stored in the chain's state upon initialization of the chain. The real requirements to the dust deposit can only decrease, but never increase. ### Gas concept in Stardust VM Each smart contract call (the request) burns gas. Each request sets a limit of how much gas can be burned during request processing at most: so-called **gas budget**. The *gas budget* is the property of every request. Gas and fee-related value is given a static (but configurable in the `governance` contract) price in iotas (or specified native tokens). All values are specified in the [gas fee policy](https://github.com/iotaledger/wasp/blob/3b22132ddae193e0c42dae647bf627e80f94f89b/packages/vm/gas/policy.go#L12). #### Pricing of gas units Gas is priced by specifying how many tokens (usually iotas) it is charged per *gas nominal unit*. Gas nominal unit is 1 or more gas units the price is given. For example if gas nominal unit is 100 and price of it is 100 iotas per gas nominal unit, it means 1 gas costs 1 iotas. This allows flexible pricing of gas units without using fractions, for example 1 iotas per 100 gas and 100 iotas per 1 gas (0.01 iotas per gas unit). #### Gas budget Gas budget is limit of how much gas can be burned in the request. Actual gas budget is set to minimum between gas budget specified in the request and the *affordable gas budget*. The *affordable gas budget* is what can be payed by the sender taking into account price of the gas and how much tokens the sender has on-chain in its account. The gas budget for the request is prepared by the [prepareGasBudget](https://github.com/iotaledger/wasp/blob/021f97c457ae2a4af2442d2cd712415a79f1b59d/packages/vm/vmcontext/runreq.go#L57) function of the VM. #### Charging for gas burned VM is burning gas while running the call in the [callTheContract](https://github.com/iotaledger/wasp/blob/021f97c457ae2a4af2442d2cd712415a79f1b59d/packages/vm/vmcontext/runreq.go#L59) function of the VM. There's always a minimal non-zero gas fee for each request. It means, if the sender's account is empty (this may happen for off-ledger senders), the request will fail. Upon normal completion of the request, the gas fee is charged from the **sender's** (requestor's) account by the [chargeGasFee](https://github.com/iotaledger/wasp/blob/021f97c457ae2a4af2442d2cd712415a79f1b59d/packages/vm/vmcontext/runreq.go#L155) function of the VM. The charged fee is split between owner of the chain and validator according to the gas fee policy in effect. If gas budget is exceeded, the VM rises `gas budget exeeded` exception immediately. In this case all changes to the state are discarded and sender is charged the whole gas budget (which is always affordable). Note that if sender's account is empty, the affordable gas budget will be 0, so the request won't be even starte tio process. ## Exception handling in the VM The Stadust VM must **always** produce a correct and deterministic output, i.e. the valid transaction and consistent mutation of the data state, **under any inputs**. The VM should **never** produce a transaction which could not be confirmed on the L2 due to its syntactical and semantical invalidity, for example because the transaction is not balanced, consumes wrong outputs or exceeds some limts set by protocol (dust deposit, number of outputs). The VM does not rely on the data of the state of the L1 account, the internally produced UTXOs. The state keeps all UTXOs in itself, so no need to query L1 and this is not a source of errors and non-determinism. <img src="https://i.imgur.com/ZeVGwte.png" width="450"> #### The outer ring of exception handling The VM [is invoked with `runvm.Run()` function]([https:/](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/runvm/runtask.go#L16)/). There is also panic catcher for the whole VM invocation. There are exactly 3 possible options how VM finishes its run: * normal finish (no exception). The valid anchor transaction and state mutation (block) is produced. * with `coreutil.ErrorStateInvalidated` exception. This exception is rised when state is invalidated by the *StateManager* writing a block to the database behind scenes. It means other validators finished the work without this node and assumption about the state is not valid anymore. So, there's no sense to continue with computations. The VM accesses the state with the optimistic assumption that state does not change in the background. If it changes, optimistic state reader [immediately rises the exception (panics)](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/vmcontext/stateaccess.go#L113). In case of optimistic reader failure VM just gracefully abandons the call without producing a result. * with fatal exception, like `DBError` or runtime error of the VM. Any panic or error in calls to the smart contracts and VM plugins (`wasmtime`) are not fatal, they are cought and processed inside and do not reach this point. #### Request level exceptions This is the next ring of exception handling. Each request in the batch is processed by running `RunTheRequest` function. The exceptions can happen by handling assets, gas and calling the smart contract (VM plugin). The exception [cought here](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/vmcontext/runreq.go#L52) are produced by the internal machinery of the VM and user is not responsible for it. Those exceptions results in skipping the request as if it never was included in the batch. It is **deterministic**. Next time (block) the request could be more lucky. The exceptions which results in skipping the request are: * transaction input limit is exceeded * transaction output limit is exceeded * number of maximum number of different native token IDs in the transaction is exceeded * not enough funds for internal dust deposit The last one can happen if the *common account* does not contain enough iotas for new internal output, which has to be created by the VM and which would keep total of a new native token. It is up to the chain's owner to provide enough iotas in the *common account*. #### Call level exceptions The request is processed by calling the smart contract. It happens in the [vmcontext.callTheContract()](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/vmcontext/runreq.go#L127) function. All exception which can be produced by the call are caught. Only the exceptions processed by the upper rings and fatal one are passed up. The rest of exceptions, including any panics in WM plugins and smart contracts, for example exceeding the gas budget and failing a `require` assertion are treated this way. The call exceptions results in: * [rolling back the state and](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/vmcontext/runreq.go#L152) the transaction builder before the call to the smart contract * [charging the gas burned](https://github.com/iotaledger/wasp/blob/502b5b07cbe02810bc24010b90e475fe8256a815/packages/vm/vmcontext/runreq.go#L155), the affordable budget from the sender. It means, the gas budget set by the user is never exceeded because the attempt to burn more results in the `gas budget exceeded` exception.