# QMK Firmware XAP Specs

---

### **Comments made may disappear -- they're not removed -- you can turn on hidden comments in the settings on the top right which should restore them.**

### **Doc generation is now handled from code:**
* [XAP defs](https://github.com/tzarc/qmk_firmware/tree/xap/data/xap) + [output](https://github.com/tzarc/qmk_firmware/blob/xap/docs/xap_protocol.md)

---

# QMK Firmware XAP Specs

This document aims to describe the requirements of the new QMK XAP ("extensible application protocol") API. The intention is to provide access to everything currently exposed through the VIA interface, as well as additional subsystems that VIA does not support.

Ultimately the intent is that XAP replaces the current VIA protocol, and the VIA host app (as well as others such as Vial, remap.keys) can leverage XAP going forward. Once sufficient critical mass is reached, support for the VIA protocol can be removed from QMK Firmware.

Comments by **@tzarc** look like this:

>> ...and should not be considered as part of the final document.

## Types

see [Types](https://github.com/qmk/qmk_firmware/blob/xap/docs/xap_0.2.0.md#types)

## Definitions

see [Definitions](https://github.com/qmk/qmk_firmware/blob/xap/docs/xap_0.2.0.md#definitions)

## Key requirements

* Superset of VIA functionality
* Once an externally-visible _ID_ is present in the `master` branch, it can never be changed, only marked as deprecated
* Deprecation ensures the "reordering" issue previously struck with keycode numbers does not occur
* XAP defines the protocol definitions only, and is transport-layer independent
* RAW HID implementation likely to be initially provided
* Pick another usage page and usage ID for XAP
* Future support may be provided for VIRTSER/SysEx etc.
* Some _routes_ may be marked as _secure_, requiring an _unlock sequence_ to be initiated by the user before allowing execution
* _Secure routes_ may be executed without an _unlock sequence_ at the user's decision, through a `#define` in their user/keymap config.h.
* At no point should QMK accept a PR with the insecure flag enabled in base keyboard or default keymap.
* The _unlock sequence_ should automatically timeout after a period of inactivity with XAP.
* Default to `90000ms`.
* Allow override through `#define` in user/keymap config.h
* Also disallow in keyboard config.h

## High-level Suggestions

* Protocol document to be defined using hjson
* Allows for comments and matches existing usage within QMK
* Allows for schema validation
* All IDs need to be defined in the protocol document
* Includes things such as:
* Subsystem IDs
* Keycodes
* RGB animation IDs
* etc. etc.
* All IDs should be exported to header file during build, much like info.json etc.
* Strongly suggested that routing is handled through codegen.
* Possibly look at embedding a compressed copy of info.json into the firmware, to ensure valid information at the time of build is available.
* Requires proper info.json to be present for all builds including XAP.
* Alternatively, provide only the things that matter for the current build -- key positions and the like.

## Requests and Responses

see [Requests and Responses](https://github.com/qmk/qmk_firmware/blob/xap/docs/xap_0.2.0.md#requests-and-responses)

### Example "conversation":

see [Example conversation](https://github.com/qmk/qmk_firmware/blob/xap/docs/xap_0.2.0.md#example-conversation)

## Broadcast messages

see [Broadcast messages](https://github.com/qmk/qmk_firmware/blob/xap/docs/xap_0.2.0.md#broadcast-messages)

## XAP Versions

* 0.0.1: Initial XAP version. Intentionally only includes XAP subsystem, with only the version query route. Can be used as a fallback to ensure version query can succeed.
* 0.1.0: Fleshes out the rest of the "base" XAP functionality -- adds unlock capability, as well as subsystem query, QMK, keyboard, and user subsystems.
* 1.0.0: Currently undefined -- should include Dynamic Keymap, Encoder, and lighting configuration.

## Subsystems

>> This list is not "locked in" and is non-exhaustive -- it is present mainly to cater for initial feature parity.

* `0x00`: XAP (always present, introduced XAP version 0.0.1)
* `0x01`: QMK (always present, introduced XAP version 0.1.0)
* `0x02`: Keyboard-specific passthrough (introduced XAP version 0.1.0) -- `xap_handler_kb()`
* `0x03`: User-specific passthrough (introduced XAP version 0.1.0) -- `xap_handler_user()`
* `0x04`: Dynamic Keymaps
* `0x05`: Dynamic Encoders
* `0x06`: rgblight
* `0x07`: rgb_matrix
* Others TBD

Subsystem validity should be queried through the _"Enabled-in-firmware subsystem query"_ under the QMK subsystem (route=`0x00,0x01`). This is the primary method for determining if a subsystem has been enabled in the running firmware. Example pseudo-C-code running on the host application side:

// elsewhere...
#define XAP_SUBSYSTEM_ID 0x00
#define XAP_SUBSYSTEM_ENABLED_SUBSYSTEMS_QUERY_ID 0x01

// check if subsystem is enabled
bool is_subsystem_enabled(uint8_t subsystem) {
    uint8_t query[] = { XAP_SUBSYSTEM_ID, XAP_SUBSYSTEM_ENABLED_SUBSYSTEMS_QUERY_ID };
    uint8_t response[32] = {0};
    xap_execute(query, sizeof(query), response, sizeof(response));
    return (response[subsystem / 8] & (1 << (subsystem % 8))) ? true : false;
}

// elsewhere...
#define RGB_MATRIX_SUBSYSTEM_ID 0x07
bool is_rgb_matrix_enabled = is_subsystem_enabled(RGB_MATRIX_SUBSYSTEM_ID);

>> Currently unsure if we want to make a distinction between rgblight and rgb_matrix, or whether or not we just provide a flag. There's considerable overlap between the two subsystems, but also enough difference that it could go either way. Should we just call it "lighting", and add support for backlight too?

>> The addition of _dynamic encoders_ is subject to decisions regarding how to persist their functionality in other in-flight PRs. If they end up being part of the matrix, then we may not need another subsystem.

### XAP Subsystem `0x00` (**always present**, introduced XAP version 0.0.1)

This subsystem is always present if XAP is enabled, and provides the ability to query information about the XAP protocol of the connected device.

#### Routes

* `0x00`: XAP protocol version (**always present**, introduced XAP version 0.0.1)
* Returns the BCD-encoded version in the format of XX.YY.ZZZZ => `0xXXYYZZZZ`
* e.g. 1.1.15 will match `0x01010015`.
* Response:
* `u32` value.
* `0x01`: Enabled-in-firmware subsystem query (**always present**, introduced XAP version 0.1.0)
* Allows for querying which top-level subsystem routes are valid within the running firmware.
* Response:
* `u8[16]` = 128 bits = 128 total subsystems.
* Each bit is the equivalent of `(1ul << SUBSYSTEM_ID)`.
* For example, a firmware with dynamic keymaps(`0x04`), and rgb_matrix(`0x07`) will have the first entry in the array of `0b10011111`. (Note: XAP, QMK, keyboard, and user, will always be present.)
* Any matching bit in the response signifies that there is a corresponding handler able to receive a request made to that subsystem.
* e.g.
* `0x02`: Query secure route status (**always present**, introduced XAP version 0.1.0)
* Allows checking if secure routes are allowed.
* Response:
* `u8`:
* `0` means secure routes are disabled
* `1` means unlock sequence initiated but incomplete
* `2` means secure routes are allowed
* any other value should be interpreted as disabled
* `0x03`: Initiate secure route unlock sequence (**always present**, introduced XAP version 0.1.0)
* Tells the firmware that the unlocking sequence should be performed by the user.
* Normal keypresses should be disabled until the sequence is completed, or fails through pressing the wrong combination, or through timeout.
* Host apps should query secure route status to determine when the unlock sequence has completed.
* All other subsystems are disabled during unlock.
* Response flags are used to determine success/fail.
* `0x04`: Disable secure routes (**always present**, introduced XAP version 0.1.0)
* Tells the firmware that any previous unlock sequence should be disregarded, thereby preventing secure routes from executing until another unlock sequence is completed.
* This also serves as a "cancel" for the unlocking sequence.
* Response:
* `u8`: `1` means secure routes are disabled, any other value signifies failure. If secure routes were already disabled, a `1` will be returned.

### QMK Subsystem `0x01` (**always present**, introduced XAP version 0.1.0)

This subsystem is always present, and provides routes that are specific to QMK, but not tied to any specific subsystem within QMK, such as jumping to bootloader.

#### Routes

* `0x00`: QMK subsystem capabilties query (**always present**, introduced XAP version 0.1.0)
* Allows for querying which QMK subsystem functionality is present.
* Capabilities (`QMK_CAPABILITY_ID`):
* `0x01`: QMK version (always present)
* `0x02`: Keyboard VID/PID/UniqueID (always present)
* `0x03`: Jump to bootloader (secure)
* Response:
* `u8[8]` = 64 bits
* Each bit is the equivalent of `(1ul << QMK_CAPABILITY_ID)`.
* For example, a firmware with base keyboard functionality will have the first entry in the array of `0b00011111`.
* `0x01`: QMK version (**always present**, introduced XAP version 0.1.0)
* Returns the BCD-encoded version in the format of XX.YY.ZZZZ => `0xXXYYZZZZ`
* e.g. 0.10.135 will match `0x00100135`.
* Response:
* `u32`
* `0x02`: Keyboard VID/PID/UniqueID/UserID (**always present**, introduced XAP version 0.1.0)
* Contains the USB VID:PID pair, as well as a UniqueID defined within QMK, and a UserID for determining if a specific user has had their userspace loaded.
* Allows for identification of the connected keyboard, as opposed to QMK version.
* Response:
* ```c
struct {
u16 VendorID;
u16 ProductID;
u64 UniqueID; // Format TBD, see below.
u16 UserID;
}

    * UniqueID allows for identifying a specific keyboard without relying on the VID:PID pair.
        * Alleviates the need for finding a unique pairing of VID:PID
        * Needs to be deterministic at time of build, and not dependent on build time or environment setup
        * Format is currently TBD
        * Do we need to go to 128-bit? Or do we drop down, is 32-bit enough?
        * Perhaps something like:

% f=$(echo -n planck/rev6 | sha256sum)
% echo ${f:0:8} # 32-bit
c654a00b
% echo ${f:0:16} # 64-bit
c654a00b59e9b67e
% echo ${f:0:32} # 128-bit
c654a00b59e9b67e0c433aaca7437f37

    * UserID allows for users to define an identifier that can be queried to determine if their firmware is loaded.
        * For example, @tzarc could add `#define XAP_USER_ID 0x1337` to his keymap's config.h
        * If the UserID matches, then you know the _User-specific Subsystem `0x03`_ below can accept data associated with @tzarc's _handlers_.
* `0x03`: Jump to bootloader (secure, introduced XAP version 0.1.0)
    * May not be present -- if QMK capabilities query returns "true", then jump to bootloader is supported
    * Secure operation -- will fail if XAP secure routes are not allowed
    * Response:
        * `u8`:
            * `0` means secure routes are disabled, and should be considered as a failure
            * `1` means successful, board will jump to bootloader
            
### Keyboard-specific Subsystem `0x02`

* Any keyboard-specific XAP functionality is to be implemented under this route.
* No restrictions on payload data or format.
* Invokes `xap_receive_kb()`.

### User-specific Subsystem `0x03`

* Any user-specific XAP functionality is to be implemented under this route.
* No restrictions on payload data or format.
* Invokes `xap_receive_user()`.