# [Suggestion]

# RFC-0046: Metadata for Offline Signers

| | |
| --------------- | -------------------------------------------------------------------------------------------- |
| **Start Date** | 2023-10-31 |
| **Description** | Include merkelized metadata hash in extrinsic signature for trust-less metadata verification.|
| **Authors** | Alzymologist Oy, Zondax LLC, Parity Technologies |

## Summary (NO CHANGES)

To interact with chains in the Polkadot ecosystem it is required to know how transactions are encoded and how to read state. For doing this Substrate, the framework used by most of the chains in the Polkadot ecosystem, exposes metadata about the runtime to the outside. UIs, wallets, signers, etc can use this metadata to interact with the chains. This makes the metadata a crucial piece of the transaction encoding as users are relying on the interacting software to encode the transactions in the correct format. It gets even more important when the user signs the transaction on an offline signer as the latter by its nature can not get access to the metadata without relying on the online wallet to provide it the metadata. So, the idea is that the metadata is chunked and these chunks are put into a merkle tree. The root hash of this merkle tree represents the metadata. Cryptographically digested together small number of extra-metadata information, it allows the offline signers and wallets to decode transactions by getting proofs for the individual chunks of the metadata. This digest is also included into the signed data of the transaction (but not sent as part of the transaction). The runtime is then including its known metadata root hash and extra-metadata constants when verifying the transaction. If the digest known by the runtime differs from the one that the offline signer used, it very likely means that the online wallet provided some invalid data and the verification of the transaction fails thus making metadata substitution impossible. The user is depending on the offline wallet showing them the correct decoded transaction before signing and with the merkelized metadata they can be sure that it was the correct transaction or the runtime will reject the transaction.

Thus we propose following:

Add a metadata digest value to signed data to supplement signer party with proof of correct extrinsic interpretation. This would ensure that hardware wallets always use correct metadata to decode the information for the user.

The digest value is generated once before release and is well-known and deterministic. The digest mechanism is designed to be modular and flexible. It also supports partial metadata transfer as needed by the signing party's extrinsic decoding mechanism. This considers signing devices potentially limited communication bandwidth and/or memory capacity.

## Motivation (NO CHANGES)

### Background

While all blockchain systems support (at least in some sense) offline signing used in air-gapped wallets and lightweight embedded devices, only few allow simultaneously complex upgradeable logic and full message decoding on the cold off-line signer side; Substrate is one of these heartening few, and therefore - we should build on this feature to greatly improve transaction security, and thus in general, network resilience.

As a starting point, it is important to recognise that prudence and due care are naturally required. As we build further reliance on this feature we should be very careful to make sure it works correctly every time so as not to create false sense of security.

In order to enable decoding that is small and optimized for chain storage transactions, a metadata entity is used, which is not at all small in itself (on the order of half-MB for most networks). This is a dynamic data chunk which completely describes chain interfaces and properties that could be made into a portable scale-encoded string for any given network version and passed along into an off-chain device to familiarize it with latest network updates. Of course, compromising this metadata anywhere in the path could result in differences between what user sees and signs, thus it is essential that we protect it.

Therefore, we have 2 problems to be solved:

1. Metadata is large, takes long time to be passed into a cold storage device with memory insufficient for its storage; metadata SHOULD be shortened and transmission SHOULD be optimized.
2. Metadata authenticity SHOULD be ensured.

As of now, there is no working solution for (1), as the whole metadata has to be passed to the device. On top of this, the solution for (2) heavily relies on a trusted party managing keys and ensuring metadata is indeed authentic: creating poorly decentralized points of potential failure.

### Solution requirements

#### Include metadata digest into signature

Some cryptographically strong digest of metadata MAY be included into signable blob. There SHALL NOT be storage overhead for this blob, nor computational overhead; thus MUST be a constant within a given runtime, deterministically defined by metadata.

- Metadata information that could be used in signable extrinsic decoding MAY be included in digest, its inclusion MUST be indicated in signed extensions;
- Digest MUST be deterministic with respect to metadata;
- Digest MUST be cryptographically strong against pre-image, both first (finding an input that results in given digest) and second (finding an input that results in same digest as some other input given);
- Extra-metadata information necessary for extrinsic decoding and constant within runtime version MUST be included in digest;
- It SHOULD be possible to quickly withdraw offline signing mechanism without access to cold signing devices;
- Digest format SHOULD be versioned.
- Work necessary for proving metadata authenticity MAY be omitted at discretion of signer device design (to support automation tools).

#### Reduce metadata size

Metadata should be stripped from parts that are not necessary to parse a signable extrinsic, then it should be separated into a finite set of self-descriptive chunks. Thus, a subset of chunks necessary for signable extrinsic decoding and rendering could be sent, possibly in small portions (ultimately - one at a time), to cold device together with proof.

- Single chunk with proof payload size SHOULD fit within few kB;
- Chunks handling mechanism SHOULD support chunks being sent in any order without memory utilization overhead;
- Unused enum variants MUST be stripped (this has great impact on transmitted metadata size; examples: era enum, enum with all calls for call batching).

## Stakeholders (NO CHANGES)

- Runtime implementors
- UI/wallet implementors
- Offline wallet implementors

All chain teams are stakeholders, as implementing this feature would require timely effort on their side and would impact compatibility with older tools.

This feature is essential for **all** offline signer tools; many regular signing tools might make use of it. In general, this RFC greatly improves security of any network implementing it, as many governing keys are used with offline signers.

Implementing this RFC would remove requirement to maintain metadata portals manually, as task of metadata verification would be effectively moved to consensus mechanism of the chain.

The idea for this RFC was brought up by runtime implementors and was extensively discussed with offline wallet implementors. It was designed in such a way that it can work easily with the existing offline wallet solutions in the Polkadot ecosystem.

## Explanation (CHANGES)

The FRAME metadata provides a lot of information about a FRAME-based runtime. It contains information about the pallets, the calls per pallet, the storage entries per pallet, information about runtime APIs and type information about most of the types used in the runtime. For decoding transactions on an offline wallet, we mainly require this type of information. Most of the other information in the FRAME metadata is actually not required for the decoding, and thus, we can remove them. Thus, we have come up with a custom representation of the metadata and how this custom metadata is chunked, ensuring that we only need to send the chunks required for decoding a certain transaction to the offline wallet.

### Protocol Definitions

### Metadata digest

The metadata digest is the compact representation of the metadata. The hash of this digest is the *metadata hash*. Below we have the type declaration of the `Hash` type that is being used and the `MetadatDigest` itself:

type Hash = [u8; 32];

enum MetadataDigest {
    #[index = 1]
    V1 {
        type_information_tree_root: Hash,
        extrinsic_metadata_hash: Hash,
        spec_version: u32,
        spec_name: String,
        base58_prefix: u16,
        decimals: u8,
        token_symbol: String,
    },
}

The `Hash` is 32 bytes long and we are using `blake3` for calculating it. The hash of the `MetadataDigest` is calculated by `blake3(SCALE(MetadataDigest))`. So, we first `SCALE` encode the `MetadataDigest` and then hash these bytes.

The `MetadataDigest` itself is represented as an `enum`. This is done to make it future proof, because a `SCALE` encoded `enum` is prefixed by the `index` of the variant. This `index` represents the version of the digest. As seen above, there is no `index` zero and we directly start with one. This proposal is for V1 only, however leaving the possibility of future implementations if need be.

Version one of the digest contains the following elements:

- `type_information_tree_root`: The root of the Merkelized type information tree.
- `extrinsic_metadata_hash`: The hash of the [Extrinsic metadata](https://docs.rs/frame-metadata/latest/frame_metadata/v15/struct.ExtrinsicMetadata.html).
- `spec_version`: The `spec_version` of the runtime as found in the `RuntimeVersion` when generating the metadata. While this information can also be found in the metadata, it is hidden in a big blob of data. To avoid transferring this big blob of data, we directly add this information here.
- `spec_name`: Similar to `spec_version`, but being the `spec_name` found in the `RuntimeVersion`.
- `base58_prefix`: The `base58` prefix used for addresses.
- `decimals`: The number of decimals for the token.
- `token_symbol`: The symbol of the token.

### Extrinsic metadata

For decoding an extrinsic, we require some information on what types are being used. The actual format of the extrinsic is the format as described in the [Polkadot specification](https://spec.polkadot.network/id-extrinsics). The metadata for an extrinsic is the following:

struct ExtrinsicMetadata {
    version: u8,
    address_ty: TypeRef,
    call_ty: TypeRef,
    signature_ty: TypeRef,
    signed_extensions: Vec<SignedExtensionMetadata>,
}

struct SignedExtensionMetadata {
    identifier: String,
    included_in_extrinsic: TypeRef,
    included_in_signed_data: TypeRef,
}

Let's start with the `TypeRef`. This is a unique identifier for a type as found in the type information.

The actual `ExtrinsicMetadata` contains the following information:

- `version`: The version of the extrinsic format. As of writing this, the latest version is `4`.
- `address_ty`: The address type used by the chain.
- `call_ty`: The `call` type used by the chain. The `call` in FRAME based runtimes represents the type of transaction being executed on chain. It references the actual function to execute and the parameters of this function.
- `signature_ty`: The signature type used by the chain.
- `signed_extensions`: FRAME based runtimes can extend the base extrinsic with extra information. This extra information that is put into an extrinsic is called "signed extensions". These extensions offer the runtime developer the possibility to include data directly into the extrinsic, like `nonce`, `tip`, etc. This means that the this data is sent alongside the extrinsic to the runtime. The other possibility these extensions offer is to include extra information only in the signed data that is signed by the sender. This means that this data needs to be known by both sides, the signing side and the verification side. An example for this kind of data is the *genesis hash* that ensures that extrinsics are unique per chain. Another example is the *metadata hash* itself that will also be included in the signed data. The offline wallets need to know which signed extensions are present in the chain and this is communicated to them using this field.

The `SignedExtensionMetadata` provides information about a signed extension:

- `identifier`: The `identifier` of the signed extension. An `identifier` is required to be unique in the Polkadot ecosystem as otherwise extrinsics are maybe built incorrectly.
- `included_in_extrinsic`: The type that will be included in the extrinsic by this signed extension.
- `included_in_signed_data`: The type that will be included in the signed data by this signed extension.

#### Type Registry
Decoders need to know the format of a type to be able to properly decoded. The [Metadata Digest](#Metadata-digest) previously detailed provides the information on the extrinsic, which can then be decoded with the utilization of these types.

struct Type {
  path: Path, // vector of strings
  type_params: Vec<TypeParams>,
  type_def: TypeDef, // enum of various types
  doc: Vec<String>,
}

struct TypeParams {
  name: String,
  ty: Option<Type>,
}

enum TypeDef {
    Composite(TypeDefComposite),
    Variant(TypeDefVariant),
    Sequence(TypeDefSequence),
    Array(TypeDefArray),
    Tuple(TypeDefTuple),
    Primitive(TypeDefPrimitive),
    Compact(TypeDefCompact),
    BitSequence(TypeDefBitSequence),
}

struct TypeDefComposite {
    fields: Vec<Field>,
}

pub struct Field {
    name: Option<String>,
    ty: Type,
    type_name: Option<String>,
    docs: Vec<String>,
}

struct TypeDefVariant {
    variants: Vec<Variant>,
}

struct Variant {
    name: String,
    fields: Vec<Field>,
    index: u8,
    docs: Vec<String>,
}

struct TypeDefSequence {
    type_param: Type,
}

struct TypeDefArray {
    len: u32,
    type_param: Type,
}

struct TypeDefTuple {
    fields: Vec<Type>,
}

enum TypeDefPrimitive {
    bool,
    char,
    String,
    u8,
    u16,
    u32,
    u64,
    u128,
    u256,
    i8,
    i16,
    i32,
    i64,
    i128,
    i256,
}

struct TypeDefCompact {
    type_param: Type,
}

struct TypeDefBitSequence {
    pub bit_store_type: Type,
    pub bit_order_type: Type,
}

The `Type` declares the structure of a type. The `type` has the following fields:

- `path`. A `path` declares the position of a type locally to the place where it is defined. The `path` is not globally unique, this means that there can be multiple types with the same `path`. Can be empty for built-in types.
- `type_def`. The high-level type definition, e.g. the type is a composition of fields where each field has a type, the type is a composition of different types as `tuple` etc.
- `type_params`. The generic type parameters of the type in use. Empty for non generic types.
- `doc`. The particularly added documentation.

The `TypeDef` variants have the following meaning:

- `Composite`. A composite type, consisting of either named (struct) or unnamed (tuple struct) fields.
- `Variant`. A Enum type (consisting of variants).
- `Sequence`. A `vector` like type wrapping the given type.
- `Array`. A fixed-length array of a specific type.
- `Tuple`. A composition of multiple types.
- `Primitive`. A primitive Rust type.
- `Compact`. A type wrapped in [Compact].
- `BitSequence`: A `vector` storing bits. `num_bytes` represents the size in bytes of the internal storage. If `least_significant_bit_first` is `true` the least significant bit is first, otherwise the most significant bit is first.

On top of these types, this RFC defines the `struct Chunk` to be used in the merkelization process of the metadata as detailed below. Chunks are scale-encoded and digested by blake3 algorithm into 32-byte CBMT leaf values.

struct Chunk {
  id: u32,
  type: Type,
}

Using the type information together with the [SCALE specification](https://spec.polkadot.network/id-cryptography-encoding#sect-scale-codec) provides enough information on how to decode types.

***Additional notes on `doc` and ` type_params`**: For the correct execution and implementation of Version 1 of this protocol, structure of types in shortened metadata need to resemble the structure of types in `scale-info` at MetadataV15 state. There are several reasons for this, particularly it requires less long-term mantainance overhead as the same work gets used for short metadata and full metadata and it allows for a faster ecosystem-wide integration, leveraging existin tools. However, to keep this performant, both `doc` and `type_params` are to be nullified as described in [Types Prunning](#Types-Prunning)*.

#### Types Prunning

To ensure objectives of this RFC are met, the following process for Types Prunning is suggested.

1. Types registry is stripped from `docs` fields.
2. Types records are separated into chunks, with enum variants being individual chunks differing by variant index; each chunk consisting of `id` (same as in full metadata registry) and SCALE-encoded 'Type' description (reduced to 1-variant enum for enum variants). Enums with 0 variants are treated as regular types.
3. Chunks are sorted by `id` in ascending order; chunks with same `id` are sorted by enum variant index in ascending order.

types_registry = metadataV15.types
modularized_registry = EmptyVector<Chunk>
for (id, type) in types.registry.iterate_enumerate {
  type.doc = empty_vector
  if (type is ReduceableEnum) { // false for 0-variant enums
    for variant in type.variants.iterate {
      variant_type = Type {
        path: type.path,
        type_params: empty_vector,
        type_def: TypeDef::Variant(variants: [variant]),
        docs: empty_vector,
      }
      modularized_registry.push(id, variant_type)
    }
  } else {
    modularized_registry.push(id, type)
  }
}

modularized_registry.sort(|a, b| {
    if a.id == b.id { //only possible for variants
      a.variant_index > b.variant_index
    } else { a.id > b.id }
  }
)

Chunks are scale-encoded and digested by `blake3` algorithm into 32-byte CBMT leaf values.

### Merkle Tree

The section below describes the protocol for developing the Metadata Merkle Tree. If any implementation where to calculate the Merkle Tree different from this proposal, the Root Hash would be different and hence the protocol would not work. It's imperative to have a common understanding and calculation process for the Metadata.

A **Complete Binary Merkle Tree** (**CBMT**) is proposed as digest structure.
- Every node of the proposed tree has a 32-bit value.
- A terminal node of the tree we call **leaf**. Its value is input for digest.
- The top node of the tree is refered to as **root**.
- All node values for non-leave nodes are not terminal are computed through non-commutative **merge** procedure of child nodes.
- In CBMT, all layers must be populated, except for the last one, that must have complete filling from the left.
- Nodes are numbered top-down and left-to-right starting with 0 at the top of tree.

Example 8-node tree

        0
     /     \
    1       2
   / \     / \
  3   4   5   6
 / \
7   8

Nodes 4, 5, 6, 7, 8 are leaves
Node 0 is root

**Complete Binary Merkle Tree construction protocol**

*Process*
1. Leaves are numbered in ascending order. Leaf index is associated with corresponding chunk.
2. Merge is performed using the leaf with highest index as right and leaf with second to highest index as left children; result is pushed to the end of nodes queue and leaves are discarded.
3. Step (2) is repeated until no leaves or just one leaf remains; in latter case, the last leaf is pushed to the front of the nodes queue.
4. Right node and then left node is popped from the front of the nodes queue and merged; the result is sent to the end of the queue.
5. Step (4) is repeated until only one node remains; this is tree root.

`blake3` transformation of concatenated child nodes (`blake3(left + right)`) is used as merge procedure.

queue = empty_queue

while leaves.length > 1 {
  right = leaves.pop_last
  left = leaves.pop_last
  queue.push_back(merge(left, right))
}

if leaves.length == 1 {
  queue.push_front(leaves.last)
}

while queue.len() > 1 {
  right = queue.pop_front
  left = queue.pop_front
  queue.push_back(merge(left, right))
}

return queue.pop

Visual representation of the results:

Resulting tree for metadata consisting of 5 nodes (numbered from 0 to 4):

       root
     /     \
    *       *
   / \     / \
  *   0   1   2
 / \
3   4

#### Digest

1. Blake3 hash is computed for each chunk of modular short metadata registry.
3. Complete Binary Merkle Tree is constructed as described above.
4. Root hash of this tree (left) is merged with metadata descriptor blake3 hash (right); this is metadata digest.

Version number and corresponding resulting metadata digest MUST be included into Signed Extensions as specified in Chain Verification section below.

### General Process Overview

To conclude this section, the following describes the steps any implementator has to go trough to achieve the same result. With the types described before, together with the Merkle Tree construction protocol, the following outlines the process implementors will go through.

1. The metadata is converted into lean modular form (vector of chunks)
2. A Merkle tree is constructed from the metadata chunks
3. A root of tree is merged with the hash of the `MetadataDescriptor`
4. Resulting value is a constant to be included in `additionalSigned` to prove that the metadata seen by cold device is genuine

### Chain verification (NO CHANGES)

The root of metadata computed by cold device MAY be included into Signed Extensions; if it is included, the transaction will pass as valid iff hash of metadata as seen by cold storage device is identical to consensus hash of metadata, ensuring fair signing protocol.

The Signed Extension representing metadata digest is a single byte representing both digest vaule inclusion and shortening protocol version; this MUST be included in Signed Extensions set. Depending on its value, a digest value is included as `additionalSigned` to signature computation according to following specification:

| signed extension value | digest value | comment |
|------------------------|----------------|------------------------------------|
| `0x00` | | digest is not included |
| `0x01` | 32-byte digest | this represents protocol version 1 |
| `0x02` - `0xFF` | *reserved* | reserved for future use |

## Drawbacks (NO CHANGES)

### Increased transaction size

A 1-byte increase in transaction size due to signed extension value. Digest is not included in transferred transaction, only in signing process.

### Transition overhead

Some slightly out of spec systems might experience breaking changes as new content of signed extensions is added - tools that delay their transition instead of preparing ahead of time would break for the duration of delay. It is important to note, that there is no real overhead in processing time nor complexity, as the metadata checking mechanism is voluntary.

## Testing, Security, and Privacy (NO CHANGES)

All implementations MUST follow the RFC to generate the metadata hash. This includes which hash function to use and how to construct the metadata types tree. So, all implementations are following the same security criteria. As the chains will calculate the metadata hash at compile time, the build process generally needs to be trusted. However, this is already a solved problem in the Polkadot ecosystem using reproducible builds. So, anyone can rebuild a chain runtime to ensure that a proposal is actually containing the changes as advertised.
Implementation can also be tested easily against each other by taking some metadata and ensuring that they all come to the same metadata hash.
Privacy of users should also not be impacted. This assumes that wallets will generate the metadata hash locally and don't leak any information to third party services about which chunks a user will send to its offline wallet. Besides that there is no leak of private information as getting the raw metadata from the chain is an operation that is done by almost everyone.

To be able to recall shortener protocol in case of vulnerability issues, a version byte is included.

## Performance, Ergonomics, and Compatibility (NO CHANGES)

### Performance

This is negligibly short pessimization during build time on the chain side. Cold wallets performance would improve mostly as metadata validity mechanism that was taking most of effort in cold wallet support would become trivial.

There should be no measurable impact on performance to Polkadot or any other chain using this feature. The metadata root hash is calculated at compile time and at runtime it is optionally used when checking the signature of a transaction. This means that at runtime no performance heavy operations are done.

### Ergonomics (NO CHANGES)

The proposal alters the way a transaction is build, signed and verified. So, this imposes some required changes to any kind of developer that wants to construct transactions for Polkadot or any chain using this feature. As the developer can pass 0 for disabling the verification of the metadata root hash, it can be easily ignored.

## Prior Art and References (CHANGES)

This project was developed upon a Polkadot Treasury grant; relevant development links are located in [metadata-offline-project](https://github.com/Alzymologist/metadata-offline-project) repository.

A reference implementation of this process is provided in [metadata-shortener](https://docs.rs/metadata-shortener/latest/metadata_shortener/) crate.

There doesn't seem to exist a similar solution to what the RFC proposes right now. However, there are other solutions to the problem of trusted signing. Cosmos for example has a standardized way of transforming a transaction into some textual representation and this textual representation is included in the signed data. Basically achieving the same as what the RFC proposes, but it requires that for every transaction applied in a block, every node in the network always has to generate this textual representation to ensure the transaction signature is valid. Furthermore, since rendering of this textual representation is potentially non-trivial (with use of non-printed glyphs, direction control, escape characters, etc.), inclusion of arbitrary textual representation increases attack surface and potentially results in subtly false sense of security.

## Unresolved Questions (NO CHANGES)

## Future Directions and Related Material

Changes to code of all cold signers to implement this mechanism SHOULD be done when this is enabled; non-cold signers may perform extra metadata check for better security. Ultimately, signing anything without decoding it with verifiable metadata should become discouraged in all situations where a decision-making mechanism is involved (that is, outside of fully automated blind signers like trade bots or staking rewards payout tools).

---
---
---
# Metadata Project - Difference between proposals
> Proposal 1: https://github.com/polkadot-fellows/RFCs/pull/46
> Proposal 2: https://hackmd.io/Za1oP9ezQra6XejlFKtP0w?both

## Metadata Structure

### Metadata `enum`
- Proposal 1 suggest building an `enum` of different `struct` for each version of what it calls Metadata Descriptor.
- Proposal 2 suggest building an `enum` with different variants prefixed by the version in line with the way scale encoding works for enums.

<table>
<tr>
<td> Proposal 1 </td> <td> Proposal 2 </td>
</tr>
<tr>
<td>

enum MetadataDescriptor {
  V0,
  V1(MetadataDescriptorV1),
}

</td>
<td>

enum MetadataDigest {
    #[index = 1]
    V1 { ... },
}

</td>
</tr>
</table>

Both proposals seem to treat this in a similar way.

### Metadata `variants`
The variants for the metadata `enum` present several differences.
- Both proposals agree on adding the following information, with just a difference on how to define the type of the `spec_version`:

        spec_version: u32 (P2) || String (P1),
        spec_name: String,
        base58_prefix: u16, 
        decimals: u8,
        token_symbol: String,

- Proposals disagree on how to represent the extrinsics within the metadata `variant`. Proposal 1 believes it's best to use the Id of type of the outermost Call enum (u32) as [described in the metadata docs](https://docs.rs/frame-metadata/latest/frame_metadata/v15/struct.ExtrinsicMetadata.html#structfield.call_ty), whereas proposal 2 suggests adding the hash of the exstrinsic metadata.

<table>
<tr>
<td> Proposal 1 </td> <td> Proposal 2 </td>
</tr>
<tr>
<td>

struct MetadataDescriptorV1 {
  call_ty: u32, // TypeId
  signed_extensions: Vec<SignedExtensionsMetadata>,
  spec_version: String,
  spec_name: String,
  ss58_prefix: u16,
  decimals: u8,
  token_symbol: String
}

</td>
<td>

enum MetadataDigest {
    #[index = 1]
    V1 {
        type_information_tree_root: Hash,
        extrinsic_metadata_hash: Hash,
        spec_version: u32,
        spec_name: String,
        base58_prefix: u16,
        decimals: u8,
        token_symbol: String,
    },
}

</td>
</tr>
</table>

- Proposals also disagree on adding inside the metadata variant, a vector representing `signed_extensions`. Proposal 1 proposes to add it, whereas Proposal 2 makes the vector `signed_extensions` part of the extrinsics hashing process.

### Open questions to solve
- How should the metadata `enum` be?
- Do we want to go with spec_version as `String` or as `u32`? Argument of P1 for going with a String is that not every substrate chain is using a `u32` on this. [Ref](https://github.com/polkadot-fellows/RFCs/pull/46#discussion_r1471624629).
- How should we treat extrinsics? Why is each deciding to treat it in the each way?

## Types

The main difference is that Proposal 1 sticks to the type definition of `scale-info` at Metadata v15, while Proposal 2 defines all the types by itself.

### Type `Struct`
Both proposal seem to be similar in terms of how types are being defined, however with some differences.

The definition of the `Type struct` shares in both proposals the need of having a `path: Vec<String>`, as well as a `type_def enum`. The difference lays on the fact that Proposal 1 adds two more keys to the struct: `type_params: Vec<TypeParams>` with `TypeParams` defined as `struct TypeParams { name: String, ty: Option<Type>,}` and `doc: Vec<String>`.

<table>
<tr>
<td> Proposal 1 </td>
<td> Proposal 2 </td>
</tr>
<tr>
<td>

struct Type {
    path: Path,
    type_params: Vec<TypeParams>,
    type_def: TypeDef,
    doc: Vec <String>,
}

</td>
<td>

struct Type {
    path: Vec<String>,
    type_def: TypeDef,
}

</td>
</tr>
</table>

### TypeDef `enum`

Both proposals share similarities in the definition of this enum. Difference lays in the fact that Proposal 1 adds `Primitive(TypeDefPrimitive)`, `Compact(TypeDefCompact)` and `Variant(TypeDefVariant)` as variants in the enum, while Proposal 2 is adding `Enumeration(Hash)`.

<table>
<tr>
<td> Proposal 1 </td> <td> Proposal 2 </td>
</tr>
<tr>
<td>

enum TypeDef {
    Composite(TypeDefComposite),
    Variant(TypeDefVariant),
    Sequence(TypeDefSequence),
    Array(TypeDefArray),
    Tuple(TypeDefTuple),
    Primitive(TypeDefPrimitive),
    Compact(TypeDefCompact),
    BitSequence(TypeDefBitSequence),
}

</td>
<td>

enum TypeDef {
    Composite(Vec<Field>),
    Enumeration(Hash),
    Sequence(TypeRef),
    Array(Array),
    Tuple(Vec<TypeRef>),
    BitSequence(BitSequence),
}

</td>
</tr>
</table>

When looking at the each of the different structs that are variants of `enum TypeDef`, some differences arise although probably solvable. Best is to discuss the overall approach first, and then dig into the specifics if need be.

### Open questions to solve
- What are the advantages of the specific type definition? And what about adopting what's already there? What if the types used were to be exactly those of `scale-info` at Metadata v15, but re-defined for this specific RFC? What do we do if metadata or `scale-info` changes, how does this evolve? Maybe defining it just as it what we need, helps future-proof it too (going in this direction means justifying the choice too).
- Related to the above: if there is a modularization process to remove docs for example, then that's an implementation and not a speficication. It's how we go from what we get to what we need.

## Merkle tree Building
To my naked eye view, process seem very similar. It's a binary tree in both cases, don't see much of a difference there.

---
---
---

#### Metadata Descriptor

The Metadata descriptor is the compact representation of the metadata. It's composed of an `enum` of different implementation versions for future proofing, however this proposal focuses in the implementation of V1. Other RFCs could follow in the future with updated implementations if needed.

enum MetadataDescriptor {
  V0,
  V1(MetadataDescriptorV1),
}

The Metadata Descriptor V1 is a `struct`, defined as:

struct MetadataDescriptorV1 {
  call_ty: u32, // TypeId
  signed_extensions: Vec<SignedExtensionsMetadata>,
  spec_version: String,
  spec_name: String,
  ss58_prefix: u16,
  decimals: u8,
  token_symbol: String
}

This name fields represent:
- `call_ty`: The `Id` of type of the outermost Call enum (u32). Currently this is described [here](https://docs.rs/frame-metadata/latest/frame_metadata/v15/struct.ExtrinsicMetadata.html#structfield.call_ty). This is of high utility for this protocol as it ensures the proper decoding of Extrinsics.
- `signed_extensions`: FRAME based runtimes can extend the base extrinsic with extra information. This extra information that is put into an extrinsic is called "signed extensions". These extensions offer the runtime developer the possibility to include data directly into the extrinsic, like nonce, tip, etc. This means that the this data is sent alongside the extrinsic to the runtime. The other possibility these extensions offer is to include extra information only in the signed data that is signed by the sender. This means that this data needs to be known by both sides, the signing side and the verification side. An example for this kind of data is the genesis hash that ensures that extrinsics are unique per chain. Another example is the metadata hash itself that will also be included in the signed data. The offline wallets need to know which signed extensions are present in the chain and this is communicated to them using this field. For this RFC, this is represented as a:

struct SignedExtensionMetadata {
    identifier: String,
    ty: u32, // TypeId
    additional_signed: u32, // TypeId
}

This matches with the current definition at MetadataV15.

- `spec_version`. As found in the `RuntimeVersion` at generation of the metadata. While this information can also be found in the metadata, it is hidden in a big blob of data. To avoid transferring this big blob of data, we directly add these values here. It is defined as a `String` as implementation on different runtimes is not of a consistent Type.
- `spec_name`. Similar to spec_version, but being the spec_name found in the RuntimeVersion.
- `ss58_prefix`. The base58 prefix used for addresses.
- `decimals`. The number of decimals for the token. If there's a decima value it will be of tpy `u8`, otherwise if no unites are defined, it will be `0u8`.
- `tokenSymbol`. The symbol of the token.

All the above constitute Metadata Descriptor. Together with types registry described below, this is minimal information that is sufficiently needed to decode any signable transaction.

---
---
---

Previously added:

***Additiona notes on `spec_version`**: this type is described in metadata; it is indeed u32 in Polkadot and Kusama and most other networks and thus could be defined in a typesafe manner, but theoretically, it could be anything, as long as it is serializable. We witnessed several networks using u16 there in the past. Not that it's all that important, but as long as our protocol would be the only limitation - it would be a limitation for our protocol; it would be unwise to enforce it here without first enforcing it upstream.
The version is not exactly an algebraic number - it has comparison operation, but no other algebra is defined; the primitive variable with these properties is really a `String`, not an integer. We've argued a lot about this internally, but with a naive JS-style solution, we have all the flexibility we might want and no real downsides (as from the point of view of our protocol, it's just an opaque constant - key for database search).*