# EPF6 - Week 18 Updates

## tl;dr

- Such an intensive week, lots of progress achieved:
- Discussed with Nando about our priority.
- Decided to focus on MVP: Supporting only single query, and returning the result as SSZ-encoded bytes
- Merged two PRs: [#15864](https://github.com/OffchainLabs/prysm/pull/15864), [#15767](https://github.com/OffchainLabs/prysm/pull/15767)
- Reviewed Nando's PRs: [#15805](https://github.com/OffchainLabs/prysm/pull/15805), [#15873](https://github.com/OffchainLabs/prysm/pull/15873)
- Opened one PR that fully implements SSZ-QL endpoint without proof inclusion: [#15888](https://github.com/OffchainLabs/prysm/pull/15888)

## Details

### Storing a pointer of the original object in `sszInfo` (and learnt about the fundamental for defining an interface)

> But fastssz already provides functions `SizeSSZ()` and `HashTreeRoot()` for all SSZ objects. So my first question is: can we just use them instead of manually calculating things?
> By Radek, September 30th ([Message link](https://discord.com/channels/476244492043812875/1387734369527136297/1422579267639709887))

Last week ([Week 17](https://hackmd.io/@junsong/B1r4rrq6eg)), Nando and I reached a consensus on leveraging `fastssz` API with our mentor, Radek. The discussion was started from Radek's comment on our work that I added above.

{%preview https://github.com/OffchainLabs/prysm/pull/15805 %}

type SSZObject interface {
	HashTreeRoot() ([32]byte, error)
}

The PR from Nando first initialized to use the existing API of `fastssz` by adding a Go interface, `SSZObject`. His PR also adds `source` field in `sszInfo` which holds a pointer that implements `SSZObject`. I reviewed this PR and left [this comment](https://github.com/OffchainLabs/prysm/pull/15805#discussion_r2426577668):

type SSZObject interface {
    ssz.Marshaler
    ssz.Unmarshaler
    ssz.HashRoot
}

> As `SSZObject` must be referring to a struct from `*.pb.go` file, I think inheriting all interfaces would be fine.

However, Radek [pointed me out](https://discord.com/channels/476244492043812875/1387734369527136297/1427403244908445737) that *"it(an interface) should contain only the functionality that we require, nothing less and nothing more."*. Even if most of protobuf structs implement `ssz.*` interfaces, we don't have to inherit all the interfaces because **we don't need them at this moment**. There are possible futures that Prysm moves protobufs out from its codebase, and if I added all those interfaces, that task would be super hard. It was great for me to correct myself when defining Go interfaces.

In the meantime, Nando and I had a short call before EPF standup call, and concluded our highest priority as **storing a pointer of the object in `sszInfo`**. This goal was **already** achieved at this week, with one PR from Nando([#15805](https://github.com/OffchainLabs/prysm/pull/15805)) and me([#15864](https://github.com/OffchainLabs/prysm/pull/15864)) each!

### Thinking of MVP: A couple of decisions made this week.


> So my question is: Is there any fancy way to convert arbitrary protobuf struct to JSON-er, by using the existing converters?
> [Message Link](https://discord.com/channels/476244492043812875/1387734369527136297/1427624078356906107)

After our SSZ-QL engine can successfully query any SSZ object for any arbitrary path in the form of **SSZ-encoded** bytes, the only question mark from my side is: *So how can I convert a raw bytes into JSON format?* Prysm has lots of functions to convert from protobuf into JSON format in `api/server/structs` package ([Link](https://github.com/OffchainLabs/prysm/tree/develop/api/server/structs)).

func SignedBeaconBlockHeaderFromConsensus(src *eth.SignedBeaconBlockHeader) *SignedBeaconBlockHeader {
	return &SignedBeaconBlockHeader{
		Message: &BeaconBlockHeader{
			Slot:          fmt.Sprintf("%d", src.Header.Slot),
			ProposerIndex: fmt.Sprintf("%d", src.Header.ProposerIndex),
			ParentRoot:    hexutil.Encode(src.Header.ParentRoot),
			StateRoot:     hexutil.Encode(src.Header.StateRoot),
			BodyRoot:      hexutil.Encode(src.Header.BodyRoot),
		},
		Signature: hexutil.Encode(src.Signature),
	}
}

For example, `SignedBeaconBlockHeaderFromConsensus` converts from `eth.SignedBeaconBlockHeader` (protobuf) into a new struct `SignedBeaconBlockHeader` that satisfies [JSON mapping of SSZ](https://github.com/ethereum/consensus-specs/blob/master/ssz/simple-serialize.md#json-mapping). My big question was the way that enables me use a *generic* version of the `*FromConsensus` functions.

> OK, so here is the other idea. **Let's return SSZ from the Beacon API endpoints instead of JSON**.
> [Message Link](https://discord.com/channels/476244492043812875/1387734369527136297/1428019908708663346)

However, Radek suggested a great breakthrough that is totally reasonable. First, light clients would mostly consume the outputs from the endpoint, so that it doesn't have to be human-readable format like JSON. Second, SSZ is becoming the standard of Beacon APIs, so it makes sense to implement these newer endpoints to support SSZ. This breakthrough makes my life much easier, as the SSZ-QL engine **already** returns SSZ-encoded bytes, so we can simply put the query result in the response.

{
    "query": ".genesis_validators_root",
    "include_proof": false
}

Also, we decided **supporting a single query** instead of multiple queries, as it is important to show feasibility, rather than perfect features at once. Thus, the request body would be much simple like above.

{%preview https://hackmd.io/@junsong/rkAN9lIIxx %}

To apply those, I need to fix the endpoint specification document. (it was nice to have a separate HackMD file for spec 😂) Note that we also have our endgame specification, and it remains same as before.

{%preview https://github.com/OffchainLabs/prysm/pull/15888 %}

- `/prysm/v1/beacon/states/{state_id}/query`
- `/prysm/v1/beacon/blocks/{block_id}/query`

This PR **finally** adds a couple of endpoints which implements SSZ-QL without proof inclusion. My colleague Nando is working really hard to work on proof generation, starting with [calculating the generalized indices](https://github.com/OffchainLabs/prysm/pull/15873) from `sszInfo`. I'm quite confident with our project status, and really thrilled to share this at Devconnect.