EPF C4 - Tomás Arjovsky - Week 7

These are the updates for the week of 2023/08/28.

First update

• Finished the project proposal.
• Worked on the slides for the project presentation.

Second update

Storage Solutions

I did some research on state persistence. Any node needs to save:

• Beacon blocks gathered from peers
• Beacon state calculated from genesis or checkpoint and using the blocks as diffs.

These need to be persisted so that if the node crashes or is stopeed the values can be restored and it won't be necessary to get them all again from gossip and request-response protocols.

Key value stores are typically used to store blocks and states, as they are very hierarchical structures that are always queried together and would require many joins if stored in a SQL db. Other consensus clients chose:

• LevelDB: kv store implemented by google. Great concurrent access and great for write-heavy usage.
• BoltDB: kv store great for read-heavy usage and crash-recovery.

All of these solutions use binary keys and binary values. Specially for the values, we'll mostly use SSZ as a serialization mechanism for easiness of use, and as we already have the values encoded.

We chose LevelDB for now, but we'll be building an abstraction so that it's easier to move from one to the other if necessary.

Storage will be a key part of the first milestones, which consists of getting blocks from gossipsub and saving them locally for queries.

U256 ssz decoding

When coding an elixir test runner for the spec tests, the team encountered an issue: we're using a rust library (through NIF) for SSZ encoding, NIFs don't support u256 integers. One of the team members was trying to handle the u256 as decimal strings in both languages, and I helped them figure out a way to handle this with integers in elixir and binaries in NIF+rust.

I came up with a description of the required process:

1. [elixir] We parse the yaml containing the expected number after deserializing. This contains decimal strings in for U256. We should turn the decimal strings into actual integers here, as elixir can handle big integers out of the box.
2. [rust] We parse the yaml containing the encoded ssz and send it to rust for deserialization. This contains some U256 structs/types.
3. [rust] we transform the U256 into the little endian (binary) representation.
4. [NIF] We send the binary representation to elixir.
5. [elixir] We turn the little endian representation to a regular integer
6. We compare the two integers in the test.

I also provided elixir tools for easy little endian representation.