ㅈ# SSZ Query Language Task Lists

> [!Note]
> This document aims to suggest a **concrete task lists** for implementing [SSZ-QL](https://github.com/eth-protocol-fellows/cohort-six/blob/master/projects/ssz-ql-with-merkle-proofs.md) in Prysm. Those lists **will be included** in the meta issue ([SSZ Query Language #15343](https://github.com/OffchainLabs/prysm/issues/15343)) for project management.
>
> This document follows the structure of the Github issue for the previous EPF project ([Light Client Support #12991](https://github.com/OffchainLabs/prysm/issues/12991)).

## Must have

> Items are sorted by **chronological order**.

**Implement the SSZ-QL engine.**

- [ ] Initialize the `ssz_query` package (NOTE: the package name can be modified.) with the following features. Mostly migrating from my previous [PoC work](https://github.com/syjn99/prysm/pull/1).
- [ ] Basic path parser (e.g., `.data.target.root`)
- [ ] Define `sszInfo` struct that contains all necessary information about the corresponding SSZ type.
- [ ] Precalculate the given Go struct, **only supporting [fixed-size types](https://github.com/ethereum/consensus-specs/blob/master/ssz/simple-serialize.md#variable-size-and-fixed-size)**.
- [ ] Calculate offset and length in bytes to read in marshalled (= serialized) SSZ object.
- [ ] Unmarshal (= deserialize) a requested value.
- [ ] Implement various-sized types (e.g., `List`, `Bitlist`).
- [ ] Add more features on the path parser (e.g., access the n-th element in `List`/`Vector`).

**Expose SSZ-QL as a new endpoint.**

- [ ] Add endpoints (`/prysm/v1/beacon/states/{state_id}/query` and `/prysm/v1/beacon/blocks/{block_id}/query`) without proof generation.
- [ ] Integrate Merkle Proofs feature. Related issue: https://github.com/OffchainLabs/prysm/issues/15344
- [ ] Benchmark the SSZ-QL engine (and Optimize it if it shows the worst performance.).

## Nice to have

> There is no specific sort order here.

- [ ] **Propose these new endpoints to [ethereum/beacon-APIs](https://github.com/ethereum/beacon-APIs)**.
- [ ] Parse `len()` for `List` and `Bitlist`.
- [ ] Implement [more advanced features](https://hackmd.io/@fernantho/rkjsksrIxg) for V2.
- [ ] Determine to introduce a cache or not: `sszInfo` of `BeaconState` from the current fork. ([Related discussion in Discord](https://discord.com/channels/476244492043812875/1387734369527136297/1399402459524370484))
- [ ] Use encoded bytes of `BeaconState` if it is in the DB. ([Related write-up](https://hackmd.io/@junsong/Byr3_lfPxl#Background-Fetching-BeaconState))
- [ ] Support more SSZ data types like `Union`, `ProgressiveList`.
- [ ] Use [SSZ fuzzer](https://github.com/KindKillerwhale/sszfuzzer) to test more crazier cases.

## Fixes

TBD

## Other

TBD

## References

### Specification

- [`postStateQuery` & `postBlockQuery` API Specification](https://hackmd.io/@junsong/rkAN9lIIxx): This specification contains the very first version of SSZ-QL API, as well as our **primary** goal for the EPF6.
- [SSZ-QL: Full Specification](https://hackmd.io/@fernantho/rkjsksrIxg): This specification contains the endgame of SSZ-QL, including advanced features for fine-grained queries.

### Write-ups

- [Navigating Options for Implementing SSZ-QL on Prysm](https://hackmd.io/@junsong/Byr3_lfPxl)
- [SSZ-QL: Proof-of-Concept](https://hackmd.io/@junsong/B1v3x4wwle)
- [Struct Tags used in `sszgen`](https://hackmd.io/@junsong/H101DKnwxl)

---

## Questions

> [!Note]
> I have some structural questions for the general design decisions. Thanks in advance!

1. **How do we merge our work?** I guess we need a separate branch in Prysm (say, `ssz-ql`) and target the branch during this project.
- NOTE: Works for Merkle Proof (leading by Nando) also can be managed in the same branch.
2. Where should `ssz_query` package locate? In [PoC](https://github.com/syjn99/prysm/tree/077bc23fedcced4a0d963266ffdedf5042adeaa4/ssz-query), I placed it in the root. Or do you have any structural advice here?