---
# System prepended metadata

title: F-formation research - UET AI lab

---

---
title: F-formation research - UET AI lab
---

**Available data**.
* Upstream task data
* Personality data
* 3D position and pose data
* F-formation labels

**Problems**.
* The amount of data is limited
* Occluded people still have position, leading to the fact that their position may be mapped to wrong people

**Survey**.
* Multitask learning.
    * *Possible tasks*. Detection, tracking, human pose estimation, and head pose estimation
    * *Ideas*.
        * Train upstream tasks then downstream task
        * Train upstream tasks and downstream task simultaneously
        * Multi-dataset training
    * *Model*. Model will output positional encoding, and object embedding, then close object embeddings will correspond to a F-formation group, i.e. KMeans
* Embedding learning.
    * *Task-specific embedding*. Train with downstream tasks
    * *Task-invariant embedding model*. Self-supervised learning
    * *Hybrid model*. Hybrid of two approaches above
* Generative modeling.

**Ideas**.
* To be robust and deployable, we must not use 3D-related data, e.g. 3D position, 3D pose, camera calibration coefficients, etc.

    $\to$ All of the given data of SALSA is useless
* Include temporal information via LSTM, attention, or 3D convolution

    $\to$ No data for this
* Apply some trick to have "more data"

**Final**.
* *Step 1*. Use generative model to generate Gaussians for pairwise distance and pairwise angular distance
* *Step 2*. For each vertex pair, derive two Gaussians for in-Fformation or not, then compare two probabilities
    * *Contrastive learning*. Input triples of (anchor, positive, negative)
        * *Input*. 
            * *Edge*. Distance, angular distance
            * *Vertex*. 
        * *Sampling method*. Use hard sampling
* *Step 3*. Some additional optimization algorithm
    * *Loss function for F-formation*. 