# EPF5 Week 13 & 14 Updates

I made some prgress this week on a few fronts:
- I finally solved that `map_while` issue from the last few weeks. Lists can now be decoded with sizably less memory allocation which should have an effect on latency, although more benchmarking will be required to say for sure. The resulting code change will be posted at the end of the post.
- I'm trying to migrate the benchmarking suite to [`divan`](https://crates.io/crates/divan) since it comes with support for allocation profiling.
- I've continued working on my ssz implementation, a little behind but not by much. The rest fo the post will go into considerations made while designing this package.

## Components of a Rust SSZ Implementation

An ssz implementation can be described in three parts, varying in difficulty:

- Trait design (medium)
- Base types implementation (easy)
- Derive macro implementation (hard)

### Trait Design
Rust has support for [traits](https://doc.rust-lang.org/book/ch10-02-traits.html) (similar to interfaces) and it allows us to describe how types should implement ssz serialization and deserialization. Defining how encoding/decoding works is the first step in shipping an ssz crate, and trait design will have major downstream effects on performance and ease of use.

I rated it as a medium because, while not actually difficult, it requires some consideration. Defining how the functions work and what inputs they accept will have major effects on performance as each type is expected to make many subcalls to other encode/decode functions. Any added latency as a consequence of how you defined your traits will compound.

Here's what I'm working with so far on the encoding side, which is a bit different to how other crates define it. The `SszDecode` trait is defined similarly to the lighthouse crate since I can't think of how a different trait definition can affect performance.

pub trait SszEncode {
    fn is_ssz_fixed_len() -> bool;
    fn ssz_fixed_len() -> usize;
    fn ssz_bytes_len(&self) -> usize;
    fn ssz_max_len() -> usize;

    fn ssz_write_fixed(&self, offset: &mut usize, buf: &mut [u8]);
    fn ssz_write_variable(&self, buf: &mut [u8]);

    fn ssz_write(&self, buf: &mut [u8]) {
        let offset = Self::ssz_fixed_len();
        let mut_offset = &mut offset.clone();

        self.ssz_write_fixed(mut_offset, &mut buf[..offset]);
        
        self.ssz_write_variable(&mut buf[offset..]);
    }

    fn to_ssz(&self) -> Vec<u8> {
        let mut buf = Vec::with_capacity(self.ssz_bytes_len());
        self.ssz_write(&mut buf);

        buf
    }

    fn to_ssz_with_vec(&self, buf: &mut Vec<u8>) {
        buf.reserve_exact(self.ssz_bytes_len());
        self.ssz_write(buf.as_mut_slice());
    }
}

The first thing to notice is the `to_ssz` and `to_ssz_with_vec` are the dev-facing trait functions. `ssz_write` can be called directly but it's meant to be called internally by the successive encoding subcalls.

A `Vec` buffer is only allocated once with enough capacity to encode the entire type without extra allocations. The internal functions only pass aroubd mutable *slices* which should be way more efficient than passing around vecs (the effect is slight for one call but I expect the savings to compound).

### Implementing Base Types
Next, encoding and decoding implementations of basic types should be provided for developers using this package. This makes devs' work easier as they don't need to rewrite a bunch of boilerplate code to get started. Trait implementations should be provided for types like unsigned integers, booleans, options, and common ethereum types like `Address` and `H256`.

Any type a dev would go on to create is likely to be a composite of all those base types, so providing implementations should have the dev ready to hit the ground running after installing our crate.

I still have a few base types left to implement before I can move on to the last part.

### `#[derive(SszEncode, SszDecode)]` Macro

As stated before, different structs in Ethereum are just composite types of the base types we talked about above. Encoding and decoding structs is just making successive calls to the encode/decode functions for each field in the struct. It's easy, but tedious and hinders dev productivity while cluttering your codebase. Luckily, rust has good support macros which allow us to *generate code*. Using a derive macro, a developer could generate an implementation of the encoding and decoding traits at *compile time*. They don't have to write another line of code.

This looks like the code below. Note that `A`, `B`, `C` either have derived or custom encode/decode implementations.

#[derive(SszEncode, SszDecode)]
struct MyStruct {
    a: A,
    b: B,
    c: C
}

I rated this hard because I've never written a macro before, and they're considered fairly advanced and error-prone.

---

The code for processing lists of decoded results went from:

process_results(
    bytes
        .chunks(<T as Decode>::ssz_fixed_len())
        .map(T::from_ssz_bytes),
    |iter| {
        List::try_from_iter(iter).map_err(|e| {
            ssz::DecodeError::BytesInvalid(format!("Error building ssz List: {:?}", e))
        })
    },
)?

to

let mut result: Result<(), ssz::DecodeError> = Ok(());

// we need to consume iter first to remove the mutable borrow of `result`
let iter = bytes
    .chunks(<T as Decode>::ssz_fixed_len())
    .map(T::from_ssz_bytes)
    .inspect(|res| match res {
        Ok(_) => (),
        Err(e) => result = Err(e.clone()),
    })
    .map_while(|res| res.ok());

match List::try_from_iter(iter) {
    Ok(list) => match result {
        Ok(_) => Ok(list),
        Err(e) => Err(e),
    },
    Err(err) => match result {
        Ok(_) => Err(ssz::DecodeError::BytesInvalid(format!(
            "Error building ssz List: {:?}",
            err
        ))),
        Err(e) => Err(e),
    },
}

which is less succinct, but hopefully performs better both in lighthouse and grandine.