# Cross-Consensus Query Language (XCQ)

## Overview

Cross-Consensus Query Language (XCQ) is designed to provide a universal way to query onchain data. It can be integrated into XCM or used as a standalone runtime API to allow offchain components such as client SDK to query onchain state in a standardized way.

## Example Use Cases

- Query balances of an account
- Query the weight and fee of a particular XCM
- Query the weight and fee of a particular extrinsic
- Query the formula to convert weight to fee
- Query the price of a particular asset
- Query the liquidity of a swap trading pair
- Query estimated slippage of a given trade
- Query if a certain feature / pallet is available

## Integrated Use Cases

### Onchain Use Cases

- Implement decentralized staking pool from a parachain
- Use XCQ to query staking rewards from relaychain
- Use XCQ to query weights and fees for related operations
- Automatically purchase coretime from broker chain
- Use XCQ to query market information
- Use XCQ to query weights and fees for related operations
- Crosschain treasury management
- Use XCQ to query price information and trading result from different chains
- Use XCQ to query weights and fees for related operations from different chains

### Offchain Use cases

- Improve XCM bridge UI
- Query XCM weight and fee from hop and dest chain
- Wallet
- Use XCQ to query asset balances
- Use XCQ to query weights and fees for related operations from different chains
- Universal dApp that supports all the parachains
- Use XCQ to perform feature discovery
- Use XCQ to query pallet specific feature
- Use XCQ to help construct extrinsic by querying pallet index, call index, etc

## Design Goal

- Simple
- Flexible
- Performant
- Generic
- Extensible

Other properties should also be considered when designing XCQ

- Future proof / Versioned
- Platform agnostic
- Size efficient
## Design Decisions

Some design decisions made so far:

- Extension based design and no centralized registry
- Instructions are imported via extensions. This ensures maximum flexibility as everyone can just make their own extension to extend XCQ without the needs of a centralized process.
- All but the core extensions are optional to not make assumptions about what features a chain should have
- Hash based identifier
- Something similar to Solidity method selector
- No need centralized register
- We can still have a register to collect all the extensions and standardize the process of creating an extension. However this is an optional step and anyone can start implementing and using an extension without being blocked
- Compact encoded
- A way to encode XCQ compactly to reduce the encoded length. This is needed to reduce onchain storage footprint and be XCM friendly
- e.g. Use SCALE-Compact unsigned integer to encode unsigned integer
- Powered by PVM
- be able to reuse existing tools and build on top of a mature VM stack

## PolkaVM Questions

- minimal program size
- 22 bytes for empty program
- 85 bytes for an adder
- estimated average program size: 150 bytes
- stills some rooms for optimization
- how to generate program by helper library
- may not be possible to do it in runtime in a efficient way
- but maybe we don't need to dynamically generate program in runtime
## Entry Point

### Runtime API

All XCQ compatible chain MUST implement the XCQ runtime API:

type Xcq = // TBD
type XcqResponse = // TBD
type XcqError = // TBD
type XcqResult = Result<XcqResponse, XcqError>;
type SizeLimitedXcq = WithMaxSize<Xcq, GetUsize<1024 * 512>>; // max size is chain specific and for WithMaxSize see https://github.com/paritytech/polkadot-sdk/issues/4092
pub trait XcqApi {
  /// Execute the provided XCQ with an optional weight limit
  /// Return the result and consumed weight
  /// When weight_limit is none, the chain specific upper weight is used
  fn execute_query(query: SizeLimitedXcq, weight_limit: Option<Weight>) -> (XcqResult, Weight);
}

All XCQ compatible chain MUST implement the new XCM instruction:

ReportQuery {
  query: SizeLimitedXcq,
  weight_limit: Option<Weight>,
  info: QueryResponseInfo,
}

It should execute the query and send a [`QueryResponse`](https://github.com/paritytech/polkadot/blob/962bc21352f5f80a580db5a28d05154ede4a9f86/xcm/src/v3/mod.rs#L426) back with the `XcqResult` in the response.

## XCQ Execution Environment

- PolkaVM program

#[polkavm_derive::polkavm_import]
extern "C" {
    fn query(id: ExtendionId, method: u8, args: &[u8]) -> Vec<u8>;
}

#[polkavm_derive::polkavm_export]
extern "C" fn extensions() -> Vec<ExtensionId> {
	// return the extensions list
}

#[polkavm_derive::polkavm_export]
extern "C" fn execute(args: &[u8]) -> Vec<u8> {
	// The program body
}

### ExtensionId

/// blake2_64 hash of the extension signature
type ExtensionId = [u8; 8];

Extensions are recognized by its id, which is a blake2_64 hash of the extension signature.

TODO: describe how to standardize the extension signature and generate hash of it. Could be something like

`<extension_name> ++ extension_methods.map(|m| m.name ++ '(' ++ m.params.join(',') ++ ')' ).join()`

### Extension Lookup Table

- Each XCQ includes an Extension Lookup Table. This table is used to map the global 8 byte `ExtensionId` with a local 1 byte `ExtensionIndex`. For the rest of the XCQ, only the `ExtensionIndex` is used to minimize the encoding size.
- This means each XCQ can only use up to 255 non-core extensions.
- We can also scan this table to check if a XCQ is supported without checking the whole XCQ

type ExtensionLookupTable = Vec<ExtensionId>;

## Extensions

XCQ is implemented with multiple extensions. Each runtime implements a set of extensions based on the features the runtime offers. The core extension offers the ability to perform extension discovery.

### Versioning

Extensions are not versioned. A new version is a new extension. However, for majority of the cases, we could just have the new features on a new extension and keep the existing extension intact. For the use case that requires the new version, it simply check if both the original and the additional extension are available.

### Generic and Typing

Ideally we want XCQ API to be strongly typed but it is not really possible so we will have it weakly typed instead. The typing information can be discovered and described with a meta type description format. XCQ offers the ability to query the meta type to help SDK to generate type definitions file or perform runtime type validations.

### Core
- Feature discovery instructions
- HasExtension(id: ExtensionId) -> bool
- Crypto functions
- blake2_64
- blake2_128
- blake2_256
- ...

### Fungibles

- AssetId: Type
- balanceOf(asset_id: Self::AssetId) -> Unsigned

### Fees

- Weight: Type
- weightToFee(weight: Self:Weight) -> Unsigned
- weightToFeeFormula() -> Xcq

### XCM

- Weight: Type
- weight(xcm: Vec\<u8>) -> Self:Weight

### Swap

- AssetId: Type
- PoolId: Type
- Liquidity: Type
- estimateSwap(inAsset: Self:AssetId, inAssetAmount: Unsigned, outAsset: Self:AssetId) -> Unsigned
- poolLiquidity(pool: Self:PoolId) -> Self:Liquidity

## Meta Types

enum PrimitiveType {
  /// Unsigned Integer up to 128 bits
  Unsigned,
  /// Signed Integer up to 128 bigs
  Signed,
  U256,
  I256,
  Bool,
  /// Vec<u8>
  Bytes,
  /// [u8; 32]
  H256,
}

struct Field {
  iden: Vec<u8>,
  type: Type,
}

struct StructType {
  iden: Vec<u8>,
  fields: Vec<Field>,
}

enum VariantField {
  Unnamed(Vec<Type>),
  Named(Vec<Field>),
}

struct Variant {
  iden: Vec<u8>,
  fields: Vec<VariantField>
}

struct EnumType {
  iden: Vec<u8>,
  variants: Vec<Variant>
}

enum Type {
  Primitive(PrimitiveType),
  Struct(StructType),
  Enum(EnumType),
}