# SSZ-QL: Proof-of-Concept

> [!Important]
> This document is based on this version: https://github.com/syjn99/prysm/tree/eac60374f13dcdcf7d35febdfefb5f912ae886de. Some links might be updated while I'm working on it continuously.

---

{%preview https://github.com/syjn99/prysm/pull/1 %}

> [!Note]
> Following up the [Option Discovery Phase](https://hackmd.io/@junsong/Byr3_lfPxl), this document will contain a part of implementation details. You can also see the diff in this [PR](https://github.com/syjn99/prysm/pull/1).

## Introduction

There was some discussion on my last write-up, and we all synced with each other that ["Option 2. Marshal everything, and treat as string of bytes"](https://hackmd.io/@junsong/Byr3_lfPxl#Option-2-Marshal-everything-and-treat-as-string-of-bytes--Byte-Parsing-Engine) is better than the Option 1. One thing I worried about the Option 2 was the **overhead** in marshalling the full `BeaconState` (almost) every time. (You might wonder how `BeaconState` is fetched in Prysm, then you can check the [very first section](https://hackmd.io/@junsong/Byr3_lfPxl#Background-Fetching-BeaconState) of my last write-up.)

In this point, Radek provided a concrete time ([~120ms](https://discord.com/channels/476244492043812875/1387734369527136297/1399399388345925785)), and we think that it is not that bad. As Bastin [mentioned](https://discord.com/channels/476244492043812875/1387734369527136297/1399400934517244096), a significant proportion of API call would be about the head state, so I [thought](https://discord.com/channels/476244492043812875/1387734369527136297/1399418641698525316) it was quite early to discuss caching mechanism due to the low cache hit-rate.

So it's time to show the **feasibility** of this approach. Let's start with the actual test of round trip. You can see the full diff [here](https://github.com/syjn99/prysm/pull/1).

## Annotated process with a concrete example: Get `.data.target.root` from `IndexedAttestationElectra`

// Start with a pointer to empty object and calculate SSZ info of `IndexedAttestationElectra`.
info, err := sszquery.PreCalculateSSZInfo(&ethpb.IndexedAttestationElectra{})
require.NoError(t, err, "PreCalculateSSZInfo should not return an error")

First, we need to *precalculate* the target SSZ type. `PreCalculateSSZInfo` analyzes the received value **recursively** and returns `sszInfo` object that has all information needed.

// sszInfo holds the pre-calculated SSZ data for a struct type.
// TODO: maybe we should another field for which type? (e.g., "Container", "List", etc.)
type sszInfo struct {
	// isVariable is true if the struct contains any variable-size fields.
	isVariable bool
	// fixedSize is the total size of the struct's fixed part.
	fixedSize uint64

	// For structs, additional information is stored:
	//
	// fieldOffsets maps a field's JSON name to its offset within the struct's fixed part.
	fieldOffsets map[string]uint64
	// goFieldNames maps a field's JSON name to its Go struct field name (e.g., "attesting_indices" -> "AttestingIndices").
	// TODO: do we need this?
	goFieldNames map[string]string
	// fieldInfos maps a field's JSON name to its SSZ info (for nested structs).
	fieldInfos map[string]*sszInfo
}

`sszInfo` is not yet finalized (b/c it's PoC) but contains enough data to calculate our example path.

Struct (fixedSize: 228, isVariable: true)
├─ attesting_indices (offset: 0) Struct (fixedSize: 4, isVariable: true)
├─ data (offset: 4) Struct (fixedSize: 128, isVariable: false)
│  ├─ beacon_block_root (offset: 16) Struct (fixedSize: 32, isVariable: false)
│  ├─ committee_index (offset: 8) Struct (fixedSize: 8, isVariable: false)
│  ├─ slot (offset: 0) Struct (fixedSize: 8, isVariable: false)
│  ├─ source (offset: 48) Struct (fixedSize: 40, isVariable: false)
│  │  ├─ epoch (offset: 0) Struct (fixedSize: 8, isVariable: false)
│  │  └─ root (offset: 8) Struct (fixedSize: 32, isVariable: false)
│  └─ target (offset: 88) Struct (fixedSize: 40, isVariable: false)
│     ├─ epoch (offset: 0) Struct (fixedSize: 8, isVariable: false)
│     └─ root (offset: 8) Struct (fixedSize: 32, isVariable: false)
└─ signature (offset: 132) Struct (fixedSize: 96, isVariable: false)

Printing the `info` looks like above. As `PreCalculateSSZInfo` runs in recursive manner, you can also see `Checkpoint` container is analyzed as well.

I'd like to note we *might* cache the result for the SSZ types that are often used (e.g. `BeaconState`).

// Calculate the offset and length for the path ".data.target.root".
path, err := sszquery.ParsePath(".data.target.root")
require.NoError(t, err, "ParsePath should not return an error")

Given `sszInfo`, we can move on to parse the path. Currently, I defined `PathElement` only to include the (field) name, and parsing the accessor for array element is out of the scope of this PoC.

offset, length, err := sszquery.CalculateOffsetAndLength(info, path)
require.NoError(t, err, "CalculateOffsetAndLength should not return an error")

`CalculateOffsetAndLength` returns **1) the offset and 2) length to read in bytes**. As we treat `BeaconState` as serialized, those are enough. `CalculateOffsetAndLength` doesn't support the various sized type yet, but it seems it is a low-hanging fruit if we include that information in `sszInfo`.

// Marshal the real object to SSZ.
realObj := &ethpb.IndexedAttestationElectra{
    // ...
}

realObjMarshaled, err := realObj.MarshalSSZ()
require.NoError(t, err)

// With the offset and length, extract the target value from the marshaled data.
targetValue := realObjMarshaled[offset : offset+length]

assert.DeepEqual(t, expectedTargetRoot, targetValue, "Extracted target value should match expected")

Alright, we now have all. Extract only needed part from bytes slice, and then unmarshal it if it is needed. In PoC, I just compared two byte slices (`assert.DeepEqual`), but it is trivial to unmarshal a bytes slice into wanted struct.

## Limitation & Future Works

- The analyzer heavily relies on the metadata of the protoc-generated struct like `json`, `ssz-size`, and `ssz-max`. Indeed, this relies on `fastssz` library which Prysm uses it for generating the structs based on `*.proto` file. You can check out the encoding logic in `fastssz` repository [here](https://github.com/ferranbt/fastssz/blob/f0b0ed3a7f4cfae1c169cbdc66533a2665e215e8/sszgen/generator/generator.go#L852).
- `sszInfo` can be discussed more. I think it must include more details to deal with the various sizeed types like `List`. Also if it's homogenous collection (e.g. `List`, `Vector`), it SHOULD contain the `sszInfo` of the element.
- Can this be compatible with merkle proof generation?