# Development Update #4
This week, I made good progress in my goals I had established, and in the upcoming week, I foresee that I will largely continue these goals, with the ultimate goal of making PRs for them and merging them into the Trin codebase.

## Previous Goals
- Write a content-id derivation function & open a PR (there will be some details to consider here, with regards to the now many-to-one mapping of content-key to id)
- Look into Python DDHT implementation for how to possibly rearchitect the way Trin handles requests, and how it can handle responses
- Review my previous notes on discv4 and discv5 and write a draft explanation of them as part of my guidebook

## Progress
### Content-Id Derivation Function
It turns out this was not so much an "easy fish" as I had expected. After some discussion on where we should define this function, we ended up planning on including it in the `trin-state` and `trin-history` packages. Actually, we split the derivation function into 2, one for state and history, and thus it makes sense for each to be defined in its respective package. The reason we split it is so we don't need to deal with figuring out whether the content-key came from a message from the state network or the history network. This makes for better encapsulation.

Then, after writing out some psuedocode, and then formatting it to match Rust best practices, I then needed to solve 2 major problems--finding an implementation for the hashing function, and figuring out how to use SSZ sedes to properly decode the content-keys. The first part was easy, but the second more time consuming.

I have looked into the SSZ spec and read articles explaining it--while I have yet to grasp how SSZ serialization and deserialization algorithms work in full, I have a good general understanding, and have now been looking into the sigp crate, aka the Rust implementation of SSZ, and figuring out how to use it.

Now, Trin already uses this crate to decode protocol messages. Since there are many types of content-keys, each which have different format, the current approach I am taking is creating an `enum` which has the various content-key types, and then each content-key gets its own `struct`, containing the fields as specified in the spec. This is just a first stab however, and may change as I learn more about SSZ.

### Architecture for handling responses & requests
Following my meeting with Piper, I then took some time to look deeper into the Python DDHT implementation, and how it used a subscription and dispatcher architecture to handle incoming requests and responses. It seemed to me the architecture was like a publisher-subscriber type thing (except not defined over a network, but instead within a node, where the publishers and subscribers are perhaps just the transmitter and receiver ends of a channel). This seemed a clean way to handle things, since the dispatcher would just send messages to the appropriate handlers, thus avoiding any blocking issues, and also not having to duplicate the logic of handling.

This opened up a larger discussion with the Trin team, and there were two broad ways proposed on how to handle this: something I have termed "pub-sub" model, and a more basic event loop model. For now, we are choosing the event loop model, mostly because it is simpler to implement and fulfills our needs for now.

However, a recent update is that the overlay subprotocol has now been implemented for Trin, and this changes the architecture again. So we will have to redesign the current event handlers to process events concurrently from multiple networks. I have been looking into this issue and designing how we can do this.

### Devp2p Guidebook
This week, I started writing again for my guidebook, and also sent it out as a resource to some new CDAP participants, who wanted to see it even though it's still very much a WIP. I was not able to get to writing a draft of Discv5; however, I began a draft for discv4, reviewed the spec, and wrote a draft for the overview of devp2p (tying everything together). I have been trying to take a "design first" explanation, that is, from the POV of the designers, the problems they encountered, and how they solved it, as I think this makes a lot more intuitive sense to understand.

### Bonus: OFFER/ACCEPT
These are back in session baby! With uTP implementation done and merged, the next step was to integrate these with uTP, since after sending ACCEPT, the actual data is sent over a uTP stream. Thus, I need to write response handlers to initiate a uTP connection and send data, as well as have the receiving node be listening for an incoming connection. I looked over the uTP code and I think I have a general idea of how I will interface with it, but still have some questions. Furthermore, this now ties into the issue of "architecture for handling requests and response".

## Next Week's Goals (9/25/21)
By next week, I hope to have completed a "reviewable" implementation of the derivation functions, that is, it is close enough to be merged, but needs a look over. With the event response handlers, this is a larger issue, and another developer is also working on it.

1. Figure out SSZ sedes & sigp implementation crate to successfully design a scheme to decode all sorts of `content-keys` & open a PR for review
2. Brainstorm & design how to rewrite the event handlers to create an event loop(s) to handle requests and responses. This includes studying how the current architecture in Trin works. Take a first stab at implementing this, if necessary.
3. Finish the discv4 draft and write an outline for discv5.

The OFFER/ACCEPT and uTP integration is dependent on goal 2, so I will see how much I can get done for that this upcoming week.

## Summary
In summary, this week the main things I learned were about SSZ sedes, Trin architecture & more generally the possible architectures and designs one can take of handling requests and responses, and got a good review of discv4. On the side, I learned more about Merkle trees, specifically how the proofs work, and some about concurrent programming.

One thing I've learned is to just dive in and start implementing/programming. I had been working through a Rust book, which was beneficial, but actual coding in Rust showed me the gaps in my knowledge, and also taught me "hand-on".