---
tags: substrate
---

# Meta type system for Substrate & ink!

### Issues

Substrate currently has a powerful metadata system that provides a lot of information about modules in a runtime. However it does not includes all the necessary information for the SDK to correctly interact with the runtime. Those information currently needs to be distributed off-chain and out-out-sync information will cause hard to diagnostic bugs and potentially cause security issues. It also increase maintenance burden as any change of the types that exposed in metadata will become backward incompatible change and requires SDK and all the dApps to be upgraded.

Ink! is currently able to generate ABI file which includes metadata out the contract and contains a limited set of the type information. However currently adding new types are non-trivial and need to be done on both ink! side and polkadot.js side. It also has some of the issues Substrate metadata having, such as shared namespace for all type names and not able to handle generic types properly.

### Goal

Have a unified way to represent type information and able to consume it in SDKs. This information can have two representations: Human readable JSON format, or compact binary format (using SCALE codec). The human readable one can includes document comments in the source code and intended to be distributed off-chain. The compact binary format should only store the only absolute minimal necessary information to represent a type and intended to be able to stored on-chain efficiently.

### Previous discussion & work

Issue: https://github.com/paritytech/substrate/issues/917

Substrate PoC:

- Substrate: https://github.com/paritytech/substrate/pull/1328
- polkadot.js: https://github.com/polkadot-js/api/pull/678 https://github.com/polkadot-js/api/pull/712

ink: https://github.com/paritytech/ink/pull/104

### Proposal

New crate `meta-type` (please suggest a better name) which is based on the [PoC substrate-metadata](https://github.com/xlc/substrate/blob/metadata-reflection-poc/core/metadata/src/lib.rs) crate. It proves a way to represent Rust type information (and possibly a super-set of be language agnostic) and able to serialized the representation into JSON (via optional serde feature) or SCALE (via optional parity-codec feature).

New crate `meta-type-derive` which is based on the [PoC substrate-metadata-derive](https://github.com/xlc/substrate/blob/metadata-reflection-poc/core/metadata-derive/src/lib.rs) create. It proves Rust macro `MetaType` (name suggestion?) to derive the implementation to generate type information for `meta-type`.

#### Integration with Ink!

Ink! will be refactored to use `meta-type` to generate the type information part of the ABI file.

#### Integration with Substrate

New create `substrate-codec` that offers `Encode` and `Decode` derive macro which generates both `parity_codec::{Encode, Decode}` and `meta-type::MetaType` implementation. This should minimized the code change on Substrate.

New metadata version is required to include all the necessary type information.

#### Integration with polkadot.js

@polkadot/types add support of parsing type information generated by `meta-type` and generate the `Codec` confirming types at runtime.

Optionally generate `.d.ts` file at built time to offer better IDE support and compile time check.

Then this can be used by both Substrate metadata handling and Smart Contract SDK generation.

## Design

### Types

General representation of a type in an language agnostic way

- TypeIdentifier
- Uniquely identify a type
- Struct
- namespace: String
- or `String[]` for better nested namespace support?
- name: `IdentKind`
- type_args: TypeIdentifier[]
- generic arguments
- empty array for types does not have generic arguments
- TypeDefinition
- Contains the definition of a type
- Enum
- Primitive
- This is a primitive type, it is expected the support of this type is natively implemented
- Struct(StructDefinition)
- A struct / class, contains fields
- Enum(EnumDefinition)
- An enum, contains variants
- TypeRegistry
- Contains the `TypeDefinition`s of all the types. This is needed to to avoid circular dependency and duplicate type definitions
- Struct
- `types: BTreeMap<TypeIdentifier, TypeDefinition>`
- `fn register(&mut self, type_id: TypeIdentifier, f: Fn(& mut TypeRegistry) -> TypeDefinition)`
- f is closure because we may not need to call it if `type_id` is already registered
- StructDefinition
- Type alias of `Field[]`, where `Field` is a struct with:
- `name`: name of the field, a `FieldName` enum:
- `FieldName::Unnamed(u16)`
- `FieldName::Named(String)`
- `ident`: `TypeIdentifier` of the field
- EnumDefinition
- Type alias of `Variant[]`, where `Variant` is a struct with:
- `name`: name of this variant.
- `index`: index of this variant.
- `fields`: associated data, which is a `Field[]`.

### JSON format

The default JSON format generated by serde-json should be enough?

This should be guarded via feature flag so we can disable it for Substrate usage.

### Binary format

The default parity-codec may be enough?

It won't be the most compact format, so may need to have custom encode/decode logic.

Have a new set of `CompactTypeRegistry`, `CompactTypeIndentifier`, `CompactTypeDefinition` etc?

This should be guarded via feature flag.