# Week 15

## Recap
My work this week was just a continuation of my work from Week 14. Which was to implement SSZ serialization and deserialization into the Portal Client. In terms of the actual implementation I worked on and completed the serialization for each Portal Wire packet type. From there I decided to move on to implementing each case of Portal Wire vector test for both serialization and deserialization in the form of unit tests. For deserialization this was just done by checking the newly created PortalWireMessage packets values against the wire vectors tests inputted information. The serialization tests were done by taking the constructing a corresponding packet with the vector tests respective information and then checking if the packet bytes lined up. This turned out to reveal quite a few problems in the serialization and deserialization values being outputted by the client. After searching for a possible cause in my implementation I found that it actually had to do with the way the library was encoding certain SSZ types. Specifically the Bytelist type. In SSZ it is specified that all collections are preceeded by a four byte offset detailing the length of the bytes to follow. However after going through what the library was producing compared to what the test vectors should be outputted, it looked as though the only issue was how long the Tuweni library specified the offset to be. Essentially:

1. The test payload for the PING packet had a value enclosed in an SSZ container that was 32 bytes long
2. The offset of the Tuweni library gave value 0x20000000 (little-endian) which makes out to 32, which was seemingly a correct length
3. The wire test vectors however show that the value should be 0x0c000000 (little-endian) which makes out to 24

On the deserialization side the Tuweni library was taking into accoun the 0x0c000000 value and truncating the payload to be 24 bytes instead of 32, which would make sense, but then would imply that the test vectors were wrong. I decided to double check by contacting a member of the Portal Client Trin team where we went over the vectors and looked for an issue. Both of us could not find out what was going wrong in terms of the SSZ processing done by the Tuweni SSZ library, until it was realized that the library had been archived by Apache. The last change to SSZ specs happened before the time that the library was archived, but may not have been reflected in implementation. So that left me with a task of finding a new SSZ library. Which is when I decided that just using the Teku internal SSZ would be in my best interest. The use of the library itself is less clear and it might be slightly overkill for just doing packet serialization/deserialization, but at least it is actively maintained. Fortunately all of the outside logic that the client uses is the same and only the internal decoder/packet SSZ implementation needs to be swapped over to use the Teku SSZ library.

## Learning
There wasn't too much learning this week outside of refering to the Portal Wire vector tests to implement each test as a unit test. I looked through the Tuweni SSZ library as well to see if there was anything I could be missing that might fix the issue I was experiencing.

### Concepts

#### Tuweni SSZ
Most of my research on the SSZ library just had to do with attempting to figure out why I was experiencing the issues that I was having. Which was mainly how the 4 byte offset for containers with bytelists are generated.

##### Resources
1. https://github.com/apache/incubator-tuweni/tree/main/ssz

#### Portal Wire Specifications
I looked through and referred to the Portal Wire vector tests to implement the tests as unit tests into the client. As well as to better understand the SSZ encoded packet values provided to learn how the encoding should work in comparison with the Tuweni SSZ library.

##### Resources
1. https://github.com/ethereum/portal-network-specs/blob/master/portal-wire-protocol.md
2. https://github.com/ethereum/portal-network-specs/blob/master/portal-wire-test-vectors.md

## Tasks
As stated previously, most of my implementation work was just adding the serialization process to each packet and then creating unit tests for both serialization and deserialization. Once the unit tests were created I primarily worked on figuring out the issue I had been experiencing. Ultimately I came to the conclusion that it would be best to switch from the Tuweni SSZ library to the Teku internal SSZ library. The original inspiration for use of the Tuweni library had actually come from Teku as they use the byte wrapper provided by Tuweni. It turns out that as the byte wrapper is not subject to change from any specifications, its use is ok even if Tuweni itself is archived. This of course doesn't hold for the SSZ implementation as it is subject to specification updates, which is where the problem I had stems from.

https://github.com/meldsun0/samba/commit/a52d0a248e79c1d7caf49637391d9f66c185435d
https://github.com/meldsun0/samba/commit/e76d774b90d04ec9e1b2a5dee8b6589c586721c6

## Week 16 Plan
Heading into Week 16 I am hoping to completely finish with all things SSZ related. As all of the correct logic for encoding and decoding is set-up it should hopefully not be too large of a task to migrate libraries from Apache's Tuweni to the Teku internal SSZ library. In general I am looking to:

- Migrate from the Tuweni SSZ library to the Teku internal library
- Pass all SSZ unit tests and Portal Wire vector tests
- Continue working on the clients outgoing request process
- Finish all Portal Wire PING/PONG logic so the client can officially make real connections with other Portal Clients (Outside of already functioning POC operations)
- If time allows start design of DHT node storage handler, this will be my first large step towards creating a functioning History Network but should also then function with any other sub-networks

The biggest unknown will be just understanding the use of the Teku SSZ library as it has a differing design, but once learned the implementation should quite quick. My work this week will also be in close communication with Meldsun as we look to get the official PING/PONG connections functioning. From there much of the logic for setting up the clients remaing packets to attain official inbound/outbound function should follow a similar pattern.