# Zerocopy Ecosystem Overview

## Issue tracking
- fxbug.dev
- cs.opensource.google/fuchsia
- Components:
- Connectivity>Libraries>net_types
- Connectivity>Libraries>packet
- Connectivity>Libraries>packet-formats

## Omnibus Build of Packet Dependencies
Add to this.

fx format-code && fx build src/connectivity/network/netstack3/core:{netstack3-core-instrumented,netstack3-core,netstack3-core-test-package,netstack3-core-testutils,netstack3-core-benchmarks,fuzz-single-device}

## Things We're Doing
- [ ] Remove `nonzero!` macro.
- [ ] Change `outer` argument of `Serialier` to `PacketConstraints`.
- [ ]

## Things we need
- update documentation
- long tail of little sharp edges
- open q: when to publish
- a: bias for publishing earlier than later
- fuchsia-specific stuff that needs to be disentangled
- most important crates:
- net_types
- packet
- packet-formats

## Miscellany
- https://fuchsia-docs.firebaseapp.com/rust/fuchsia_component/index.html

$ fx set core.x64 --no-ccache --cargo-toml-gen
$ fx build src/lib/network/packet:packet
# Might not need to do this line every time
$ fx build build/rust:cargo_toml_gen
$ fx gen-cargo //src/lib/network/packet:packet

## TODOs by crate

### Packet

### Patchsets
- ~~[[packet] Simplify execution of Serializers](https://fuchsia-review.googlesource.com/c/fuchsia/+/861377)~~
- ~~[[packet] Rename and simplify `SerializeBuffer](https://fuchsia-review.googlesource.com/c/fuchsia/+/862010)~~
- ~~[[packet] Simplify buffer trait hierarchy](https://fuchsia-review.googlesource.com/c/fuchsia/+/862776)~~
- ~~[[packet] Merge SerializeBuffer into GrowBufferMut](https://fuchsia-review.googlesource.com/c/fuchsia/+/874198)~~

#### Clean up trait hierarchy

- Rename `FragmentedBuffer` to `MaybeFragmentedBuffer`?
- Move `Buffer`/`BufferMut` into N3
- Add `testutil` module which provides tests that validate buffer implementations; mention this module in the docs for all buffer traits

#### Improve docs

- `#![deny(missing_docs)]`
- In `ParsablePacket::parse`'s `Padding` docs section, the phrase "consume any post-packet padding from the buffer's suffix" is misleading - it should be something more like "from the body and added to the suffix"
- Consider renaming `encapsulate` to `encapsulate_in` or `nest_in` or similar
- Consider renaming `Nested` now that it is only used for `Serializers`
- In `GrowBuffer`:
- This sentence is misleading: "The body of the buffer can shrink or grow as allowed by the capacity as packets are parsed or serialized." It is not guaranteed that a `GrowBuffer` can shrink.
- Clarify "guarantees never to discard"
- `undo_parse` can panic, but the docs claim it can't
- Add doc comment to `Serializer`

#### Impelemntation cleanup

- In `alloc_no_reuse`, change the following line to use checked arithmetic?

  let need_capacity = prefix + body + suffix;

#### Miscellaneous
- Move `BufferProvider::alloc_no_reuse` to `BufferAlloc`? It was introduced by Zeling in https://fxrev.dev/786703
- Consider verifying behavior using [flux](https://github.com/flux-rs/flux)
- Make functions and methods `const`
- Merge N3's `transport::tcp::buffer::SendPayload` into `packet`?
- There should be an extension trait on slices that provides the ability to read and write zerocopy objects (e.g., like `write_obj_front`)
- Remove getters on `ParseMetadata` now that its fields are `pub`?
- Make sure naming is consistent (traits use full words and types use abbreviated words)
- Consolodate `zero` and `zero_iter`
- Consider using `either` crate instead of custom `Either` buffer type
- Consider replacing `ByteSliceInnerPacketBuilder` with a blanket impl
- Combine `Either` and `EitherSerializer`?
- Make `PacketConstraints: Copy`

## Hints that Josh gave Jack for implementing UDP
- Start with parsing; it's easier than serializing
- Implement the `ParsablePacket` trait
- For parsing, take a look at what the `BufferView` trait gives you
- For ease of implementation, start with `ParseArgs = ()`
- For generating the body, use `buffer.into_rest()`
- For `ParsablePacket::parse_metadata`, the body length field is the framework's notion of body length, not (necessarily) the length field from the UDP header
- "...", the body length is `self.body.len()`, not 0
- After Jack started implementing `PacketBuilder` for `UdpPacket`: Your `PacketBuilder` type does not need to be the same as your `ParsablePacket` type
- Think of a `PacketBuilder` as a builder - just whatever information is necessary to perform serialization
- Just introduce a new `UdpPacketBuilder` type which contains only those fields which you can't compute on the fly
- After Jack suggested putting a body field in the `UdpPacketBuilder` type: Look at the `SerializeBuffer` docs (my thinking was to dissuade him from this and point him towards the fact that the `SerializeBuffer` already has access to the body in the buffer)
- Remember that it is the caller's responsibility to provide the body - `PacketBuilder` doesn't know about it until `serialize` is called

## Potential future directions
- If we use GATs to support a type parameter that represents mutability, we could do something like:

  struct UdpPacket<'a, M: Mutability> {
      header: Ref<'a, M, UdpHeader>,
      body: Ref<'a, M, [u8]>,
  }

- Make `Serializer` methods which take a `PacketBuilder`/`PacketConstraints` `#[doc(hidden)]` (unless we think consumers would ever want to implement `Serializer`?)

## Concepts

### Buffers

A sequence of bytes logically segmented into a:
- prefix
- body
- suffix

The total size of these segments is the `capacity` of the buffer.

The size of the body is the `length` of the buffer.

## Before

digraph G {
    Buffer [tooltip="A byte buffer used for parsing that can grow back to its original size."];
    Buffer -> GrowBuffer, ParseBuffer [dir=back];

    BufferMut [tooltip="A byte buffer used for parsing and serialization."];
    BufferMut -> TargetBuffer, ParseBufferMut, Buffer [dir=back];
    
    //BufferView [tooltip="A view into a ShrinkBuffer."];
    //BufferView -> Sized, "AsRef<[u8]>" [dir=back];
    
    //BufferViewMut [tooltip="A mutable view into a Buffer."];
    //BufferViewMut -> BufferView, "AsMut<[u8]>" [dir=back];
    
    ContiguousBuffer [tooltip="A buffer that is contiguous in memory."];
    ContiguousBuffer -> "AsRef<[u8]>" [dir=back];
    
    ContiguousBufferImpl [tooltip="A helper trait to implement FragmentedBuffer and ContiguousBuffer for any type that is AsRef<[u8]>."];
    ContiguousBufferImpl -> "AsRef<[u8]>" [dir=back];
    
    ContiguousBufferMut [tooltip="A mutable buffer that is contiguous in memory."];
    ContiguousBufferMut -> ContiguousBuffer, "AsMut<[u8]>" [dir=back];
    
    ContiguousBufferMutImpl [tooltip="A helper trait to implement FragmentedBufferMut and ContiguousBufferMut for any type that is AsMut<[u8]>."];
    ContiguousBufferMutImpl -> ContiguousBuffer, "AsMut<[u8]>" [dir=back];
    
    //Fragment [tooltip="A single byte slice fragment in a FragmentedByteSlice."];
    //Fragment -> ByteSlice [dir=back];
    
    FragmentedBuffer [tooltip="A buffer that may be fragmented in multiple parts which are discontiguous in memory."];
    
    FragmentedBufferMut [tooltip="A FragmentedBuffer with mutable access to its contents."];
    FragmentedBufferMut -> FragmentedBuffer [dir=back];
    
    //FromRaw [tooltip="A packet that can be created from a raw form."];
    //FromRaw -> Sized [dir=back];
    
    GrowBuffer [tooltip="A buffer that can grow its body by taking space from its prefix and suffix."];
    GrowBuffer -> FragmentedBuffer [dir=back];
    
    GrowBufferMut [tooltip="A GrowBuffer which provides mutable access to its contents."];
    GrowBufferMut -> GrowBuffer, FragmentedBufferMut [dir=back];
    
    //ParsablePacket [tooltip="A packet which can be parsed from a buffer."];
    //ParsablePacket -> Sized [dir=back];
    
    ParseBuffer [tooltip="A byte buffer used for parsing."];
    ParseBuffer -> ShrinkBuffer, ContiguousBuffer [dir=back];
    
    ParseBufferMut [tooltip="A ParseBuffer which provides mutable access to its contents."];
    ParseBufferMut -> ParseBuffer, FragmentedBufferMut, ContiguousBufferMut [dir=back];
    
    ReusableBuffer [tooltip="A byte buffer that can be reused."];
    ReusableBuffer -> TargetBuffer, ShrinkBuffer [dir=back];
    
    ShrinkBuffer [tooltip="A buffer that can reduce its size."];
    ShrinkBuffer -> FragmentedBuffer [dir=back];
    
    TargetBuffer [tooltip="A buffer that can be serialized into."];
    TargetBuffer -> GrowBufferMut [dir=back];
}

## After

digraph G {    
    
    //BufferView [tooltip="A view into a ShrinkBuffer."];
    //BufferView -> Sized, "AsRef<[u8]>" [dir=back];
    
    //BufferViewMut [tooltip="A mutable view into a Buffer."];
    //BufferViewMut -> BufferView, "AsMut<[u8]>" [dir=back];
    
    ContiguousBuffer [tooltip="A buffer that is contiguous in memory."];
    ContiguousBuffer -> FragmentedBuffer, "AsRef<[u8]>" [dir=back];
    
    ContiguousBufferMut [tooltip="A mutable buffer that is contiguous in memory."];
    ContiguousBufferMut -> ContiguousBuffer, FragmentedBufferMut, "AsMut<[u8]>" [dir=back];
    
    //Fragment [tooltip="A single byte slice fragment in a FragmentedByteSlice."];
    //Fragment -> ByteSlice [dir=back];
    
    FragmentedBuffer [tooltip="A buffer that may be fragmented in multiple parts which are discontiguous in memory."];
    
    GrowBuffer [tooltip="A buffer that can grow its body by taking space from its prefix and suffix."];
    GrowBuffer -> FragmentedBuffer [dir=back];
    
    FragmentedBufferMut [tooltip="A FragmentedBuffer with mutable access to its contents."];
    FragmentedBufferMut -> FragmentedBuffer [dir=back];
    
    //FromRaw [tooltip="A packet that can be created from a raw form."];
    //FromRaw -> Sized [dir=back];
    
    
    GrowBufferMut [tooltip="A GrowBuffer which provides mutable access to its contents."];
    GrowBufferMut -> GrowBuffer, FragmentedBufferMut [dir=back];
    
    //ParsablePacket [tooltip="A packet which can be parsed from a buffer."];
    //ParsablePacket -> Sized [dir=back];
    
    ParseBuffer [tooltip="A byte buffer used for parsing."];
    ParseBuffer -> ShrinkBuffer, ContiguousBuffer [dir=back];
    
    ParseBufferMut [tooltip="A ParseBuffer which provides mutable access to its contents."];
    ParseBufferMut -> ParseBuffer, ContiguousBufferMut [dir=back];
    
    ReusableBuffer [tooltip="A byte buffer that can be reused."];
    ReusableBuffer -> SerializeBuffer, ShrinkBuffer [dir=back];
    
    ShrinkBuffer [tooltip="A buffer that can reduce its size."];
    ShrinkBuffer -> FragmentedBuffer [dir=back];
    
    SerializeBuffer [tooltip="A buffer that can be serialized into."];
    SerializeBuffer -> GrowBufferMut [dir=back];
}

## Naming `packet`

### `buffer`
pros:
- boop

cons:
- moop