# 5G Aloha
[TOC]

## Objective
Compare different wireless communication technologies for real-time virtual concerts.
ALOHA boards:
- Raspberry pi with DAC
- custom linux distro
- sushi: mixer
- aloha: vst plugin for streaming audio over UDP, protocol very similar to RTP

## Setup hardware


### Time offset network
NTP is not bad, but simply collecting the offset over time is better.
The traffic needs to be on a separate network to avoid interfering with the ALOHA traffic and to have a more precise measurement of the time offset.

### ALOHA network
Using lan, 4g, 5g or wifi the "audio" traffic generated by ELK's ALOHA protocol has a **star topology**:
- each board sends the recorded audio to the MITM
- the MITM mixes and sends back the result to each board

## Setup software
### ALOHA boards
1. Downgrade to version 0.9.5-rc3 and add/move alohavst.so to `plugins/` (already done for 3 boards)
2. Setup IPs on both networks (dongle and ethernet)
3. copy simple aloha config json
4. disable NTP: best thing would be to update the clock with NTP after boot and then disable it, to avoid being out of sync for too much time and accumulate too much time error
5. setup core isolation:
- isolate 1 thread for tcpdump
- isolate 1 thread for time traffic
- aloha/sushi runs at super high priority, no need to isolate for that

### MITM
1. Either .net runtime, or compile to binary
2. realtime kernel (need low-latency sockets and mixing)
3. core isolation:
- 1 thread for mixer
- 1 thread for tcpdump aloha
- 1 thread for tcpdump timediff

## GOULASH mixer
Written in .net 5

### Queueing packets for mixing
When UDP packet arrives:
1. Check that length is 876 bytes (ALOHA audio packet)
2. Add packet to the queue of the corresponding "Board":
1. Calculate difference between packet header timestamp and current timestamp (subject to clock drift, but used only for jitter estimate)
2. Store this "delay"
3. Add packet to queue at correct position (queue is a sorted dict, where key is the timestamp). If the expected timestamp is lower than the arrived one, make sure that the queue has all values from expected to current/arrived. If some packets are not there, add null as value. This ensures that the queue length is equal to `last timestamp - first timestamp + 1`
4. if the last delay/jitter adjustement was more than 2 secs ago:
1. calculate jitter as the standard deviation $\sigma$ of the delays in ms
2. calculate desired queue length as $l = \frac{m\sigma}{p}$, where $p=\frac{1000}{375}=2.\overline{6}$ is the period in ms of aloha packets (same as the duration of the audio sample contained in 1 packet). $m$ is a factor of trade-off between packet drops and latency: if the jitter is high, a smaller queue means that more packets will arrive too late for mixing. A good value found experimentally is $m=4$.
3. if the difference between the desired length and the current length is bigger than 1 (can lower this threshold, but adjustements can be heard so they should be not so frequent), shift the queue in order to match the desired length. Shortening means removing the first packets and changing the expected value, while to lengthen means lowering the expected value and adding "null" packets at the front.
3. Trigger mixing

### Mixing and sending
If last mix was more than $2.\overline{6}$ ms ago, mix the first packet of each queue (null packets count as a stream with volume 0).
For each board, mix other streams and send to it:
1. Normalize volume of each packet according to gain specified in its header
2. average the normalized streams
3. create aloha packet and send in UDP

## Metrics pipeline

### Files generated by a test

**Aloha dumps**: tcpdump on each board and mitm, to get information about which aloha packet (indicated by its ID) is sent and received by each device.

**Offset packets:** the boards send through the time-offset network the packets containing their timestamp, while the MITM dumps all the received packets into a .pcap.
The offset dump is used is the processing phase together with the dumps containing the aloha traffic in order to correct the clock differencies between the boards.

**Mappings file:** generated by the Mixer, links the IDs of the arrived packets to the IDs of the mixed ones. Needed to recontruct the end-to-end flows only if the network uses a mixer.
- Example: first column are the IDs of the result mixed packets, the other columns contain the IDs of the input packets.

### Pipeline to merge and collect statistics

Configure our `exec_pipeline.sh` script, or manually as follows.
**Pipeline:**
1. **collect** packets (eg: *Tcpdump*): `tcpdump -i <interface> udp -w my_dump`
- alternative: `tshark -i "Ethernet 2" -f "udp" -w my_dump.pcap`
2. **export** csv (eg: *Tshark*): `tshark -r my_dump -T fields -e ip.proto -e udp.srcport -e frame.time -e data.len -e data.data -E header=y > my_dump.csv`
3. **parse** (eg: *Python*): the packet fields such as source and destination ip are readily available from the previous point, but the body of the aloha packet (counter, timestamp) has to be manually parsed.
- Example of parsed dump for a single board 
- Overview of the header of an aloha packet 
4. **process** (eg: *Python*): all the parsed dumps are joined together and the different packet flows are extracted.
6. **plot** (eg: *R*)

### NetEm to simulate network situations
**`netem`** for simulating packet loss and latency. [manual](https://man7.org/linux/man-pages/man8/tc-netem.8.html). Examples:
- Introduce 30% packet loss: `tc qdisc add dev eth0 root netem loss random 30%`
- Introduce Gilbert-Elliot loss model `tc qdisc add dev eth0 root netem loss gemodel (Pe/b)/(1-Pe) 1/b`
- Undo all for selected interface: `tc qdisc del dev eth0 root netem`
- Overwrite simulation `tc qdisc change dev eth0 root netem delay 10ms 10ms`

### Audio interface delay
To measure the total delay that a user would perceive, we should add the audio processing delay to the network+mixing one. We can measure it using jack_delay with sushi running on a single board (simple audio loopback). The following example is only to compare different audio loops and not to measure the actual delay of audio processing.
| Linux, 48Khz, 1024f/p, 2p/b | Avg (ms) | Min | Max | mDev |
| -------- | -------- | -------- | -------- | -------- |
| ONLY AUDIO INTERFACE | $43.47$ | | | |
| BOARD IN LOOP | $46.705$ (+3) | | | |
| **P2P** --> BASELINE | $58$ (+12) | | | |
| MITM | $62$ (+4) | $61$ | $67$ | |

To calculate the actual delay of audio processing, the parameters need to be tuned (frame and buffer sizes).

## Results

Example of processed csv:
- we compute metrics for each directional stream (eg: packets `A->MITM->B` and packets `B->MITM->A` are different streams)