# Real World Experiments: Continued

## Force Prediction, 'd' only
First, try learning a model which predicts force as a function of the drones' relative position and velocity. That is to say, $\hat{\mathbf{f}}_{\text{ext}}^{(i)} = f_{\theta}(\mathbf{x}^{(j)} - \mathbf{x}^{(i)})$, where $\mathbf{x}^{(i)} = [\mathbf{p}^{(i)}, \mathbf{v}^{(i)}]^T$. For now, we will focus on the downward force only. Use the mean-squared error normalized by the norm of the labels.

The first dataset ("stage 0") is collected at $\Delta p_d = -1.75m$. These are the corresponding force labels:

Now, evaluate the model at a fixed relative velocity vector and relative 'd' coordinate. Vary the relative 'n' and 'e' coordinate.





The magnitudes in the last slice appear to be incorrect. Looking at the plot of the raw velocity data, we may not have seen this combination of velocities before.

Each of the plots above have fixed $\Delta v_d = 0$. At the first stage, the drone has only ever seen velocities in the 'd' direction between approximately $-0.3 m/s$ and $+0.3 m/s$. When we try to make predictions outside these values, they are not reasonable.

Bottom drone is moving down at $0.2 m/s$:

Bottom drone is moving down at $0.5 m/s$:

At stage 1, suppose we wish to perform a random walk in the 'd' direction between $\Delta p_d = -1.75m$ and $\Delta p_d = -1.5m$ (in addition to performing a random walk in the 'n' and 'e' directions). If the downward velocity of the bottom drone exceeds $0.2 m/s$ in magnitude, we would not expect the force predictions to be reasonable. Perhaps we should begin our data collection with a random walk in the 'd' direction so that we get downwards velocities outside of this range.

## Signal Denoising
Last week, we saw there is an issue with the measured accelerations when the drones aproach zero in the 'n' and 'e' directions. Perhaps there is an interaction between the ground effect of the bottom drone and the downwash effect of the top drone.

Hypothesis: This effect is concentrated on the high-frequency nodes of the measured acceleration. If we can filter these nodes out, we will get the desired signals (i.e. forces).

No filter:

Gaussian filter $\sigma = 2$:

Gaussian filter $\sigma = 10$:

Notwithstanding, the measured forces are comparable in magnitude to the noise in our force estimates.

In Neural-Swarm, the measured forces are more than three times greater than the magnitude of the noise in the force labels.

### Force Prediction, All Three Directions
Now, try training a neural network which predicts all three forces.
| 'n' | 'e' | 'd' |
| -------- | -------- |-------- |
|  | | |
|| | 
|

### Force Prediction, All Three Directions, With Gaussian Filter ($\sigma = 10$)
| 'n' | 'e' | 'd' |
| -------- | -------- |-------- |
| || |
|||
|

The filtered predictions are not necessarily better than the unfiltered ones. In fact, in the 'd' direction, our filter seems to lessen the magnitude the predicted forces.

# Next Steps
1. Henry: Develop a more rigorous learning framework.
* How can we certify that at stage $z$, we have learned the true force function within $\epsilon$ accuracy? Use a validation dataset to estimate the population risk.
* Trajectory tracking to assess model correctness
* Cerify that the stage is "successful" when the test error is lower than some threshold. Otherwise, we need to collect more data points.
2. Jennifer and Ajay: Outdoor flights.
* Can we replicate the force prediction results from indoor training when we move outdoors?
* Collect an outdoor dataset with the same procedure we used indoors.
3. Validating our Procedure:
* Reduce the biases in the controller using Ajay and Peter's thrust stand.
* Validate our force estimates using Jennifer's mount for the sufferer drone.

# Future Research: Temporal Learning for Graphical Models
Our problem: we have a sequence of points $\{\mathbf{x}_t\}_{t=1}^N$ and corresponding labels $\{ \mathbf{y}_t\}_{t=1}^N$, where each $\mathbf{x}_t = \mathcal{G}(\mathcal{V}, \mathcal{E}_t)$ is a graph of $K$ vertices $\mathcal{V}$ and edges $\mathcal{E}_t$ and each $\mathbf{y}_t$ is a $3K$-dimensional vector of forces. Notice that here, the edge set can change as a function of time, while the number of vertices (robots) is assumed to be constant. We want some model that takes into account the temporal nature of our data (i.e. uses $(\mathbf{x}_t, \mathbf{y}_t)$ to make predictions for $\mathbf{y}_{t+1}$).

The naïve way to do this: just shove $\mathbf{f}_{t-1}, \ldots, \mathbf{f}_{t-T}$ into the model.

Another way: recurrent neural networks.
Recent work on memory and graph convolutional networks ([Pareja, et al](https://arxiv.org/pdf/1902.10191.pdf) and [Kipf and Welling](https://arxiv.org/pdf/1609.02907.pdf)):
Can we apply something similar to our existing graph neural network architecture?
Right now, we predict our forces according to:
\begin{align*}
\mathbf{f}^{(i)} = \rho\left( \sum_{j \in \mathcal{N}_i} \phi(\Delta \mathbf{x}^{(ij)} , \theta_{\phi}), \theta_{\rho}\right).
\end{align*} [Wu, Nikolentzos, and Vazirgiannis](https://arxiv.org/pdf/2003.00842.pdf) first use a message-passing GNN to output an encoding vector, and then use a RNN to decode this vector into a prediction. The RNN has a memory of the previous encoding vectors it has seen.

Jan (from last meeting): Training an RNN requires much more data than a simple MLP. Is this too burdensome in terms of the amount of flight data we would need?
Ryan: Would just involve adding a GRU to the existing GNN. Theoretically, this should not result in worse performance since the GRU could learn to ignore the previous hidden states.

Proposed model:
Note that the [number of parameters in the GRU does not vary with the length of the input sequence](https://pytorch.org/docs/stable/generated/torch.nn.GRU.html), $T$.