---
# System prepended metadata

title: YODO notes
tags: [Research]

---

---
tags: Research
---


# YODO notes

${^{kpt}T_{ee}}$
${\Omega(\cdot) \overline{s}}$

where ${\overline{s}}$ and ${\overline{P}_{\mathbb{C}}}$ are ground-truth labels

${Q \in \mathbb{Q}}$ are equivalent symmetric transformation

### Dataset

The data generation pipeline is developed with Blender 
- partial point cloud: from the rendered 2D depth image 
- ${\overline{P}_{\mathbb{C}}}$ and ${\overline{s}}$ and pose: retrieved from CAD models
- In one trajectory: **neighboring sub-goals are at least 2𝑚𝑚 or 2°**

### BundleTrack
${\text{Input}}$

- ${I_{\tau}}$,  ${\tau \in {0,...,t}}$: a sequence of RGB-D data
- ${M_0}$: a binary mask on the first image ${I_0}$, indicating the target object region to track in the image space
- ${T_{0}^{C}}$ (optional): the initial pose in the camera’s frame ${C}$ for computing object’s absolute pose in ${C}$

${\text{Output}}$
- ${T_{0 \rightarrow \tau} \in SE(3), \tau \in {1,2,...,t}}$ 

### NUNOCS

NUNOCS net : ${\phi(P_\mathcal{O}) = (P_{\mathbb{C}, s})}$
- ${\phi}$ : PointNet like architecture
- Input: a scanned partial point cloud
    - ${P_\mathcal{O} \in \text{R}^{N \times 3}}$
- Output: point-wise coordinates and scales
    - ${P_\mathbb{C} \in \text{R}^{N \times 3}}$ => ${ \text{R}^{N \times 3 \times B}}$ where ${B = 100}$
    - ${s = (1, \alpha, \beta) \in \text{R}^{3}}$ (for compactness)
- Inference during online execution
    - ${s \circ P_{\mathbb{C}}}$
    - establishes a dense correspondence between the NUNOCS representation of ${\mathcal{O}}$ and the NUNOCS shape of ${\mathcal{O}_\mathcal{D}}$ by finding nearest neighbor
- Loss


### Jocobian

Basically, a Jacobian defines the dynamic relationship between two different representations of a system

${f : \text{R}^n \rightarrow \text{R}^m}$ ,where ${f(x) \in \text{R}^m}$, ${x \in \text{R}^n}$

${J_{ij} = \frac{\partial{f_i}}{\partial x_j}}$
- ![](https://i.imgur.com/kDVWKTv.png)
- ${f(x + \Delta x) 	\approx f(x) + J(x)\cdot(\Delta x)}$

${\begin{split}
J &= \frac{\partial x}{\partial q} &= \frac{\partial x}{\partial t} \cdot \frac{\partial t}{\partial q} 
\end{split}}$

- ${\dot{x} = J \cdot \dot{q}}$  or 
- ${\Delta{x} = J \cdot \Delta{q}}$



### CatBC
- ${\mathcal{J}}$: demonstration trajectory
- ${\bar{\xi}_i}$ : object ${SE(3)}$ pose in demo
- ${\xi_i}$ : object ${SE(3)}$ pose in testing
- ${q}$ : joint configuration


### pose graph optimization

${G = \{V, E\}}$
- ${V}$: i'th frame and object pose ${T_i}$, where ${|V| = k+1}$ 
- ${E}$: two types of energies
    ![](https://i.imgur.com/EJIPN62.png)
    - residuals from fearure correspondense
        ![](https://i.imgur.com/AN679YC.png)
    - dense pixel-wise point-to-plane distance
        ![](https://i.imgur.com/rb5wYad.png)
- Target
    ![](https://i.imgur.com/ipB7V84.png)

### Method: NUNOCS net
Given the single visual demonstration for object ${\mathcal{O}_{\mathcal{D}} \in \mathbb{O}_{\text{train}}}$ and in order to “project” the trajectory so it works for a novel object ${\mathcal{O}}$ during online execution, **category-level data association between** ${\mathcal{O}_{\mathcal{D}}}$ **and** ${\mathcal{O}}$ **is required**. To do so, this work **establishes dense correspondence in a 9-dim**


### true pose of BundleTrack

${T_{\tau}=T_{0\rightarrow \tau} T_0^C \in SE(3)}$





_