---
tags: Research
---

# YODO notes

${^{kpt}T_{ee}}$
${\Omega(\cdot) \overline{s}}$

where ${\overline{s}}$ and ${\overline{P}_{\mathbb{C}}}$ are ground-truth labels

${Q \in \mathbb{Q}}$ are equivalent symmetric transformation

### Dataset

The data generation pipeline is developed with Blender
- partial point cloud: from the rendered 2D depth image
- ${\overline{P}_{\mathbb{C}}}$ and ${\overline{s}}$ and pose: retrieved from CAD models
- In one trajectory: **neighboring sub-goals are at least 2𝑚𝑚 or 2°**

### BundleTrack
${\text{Input}}$

- ${I_{\tau}}$, ${\tau \in {0,...,t}}$: a sequence of RGB-D data
- ${M_0}$: a binary mask on the first image ${I_0}$, indicating the target object region to track in the image space
- ${T_{0}^{C}}$ (optional): the initial pose in the camera’s frame ${C}$ for computing object’s absolute pose in ${C}$

${\text{Output}}$
- ${T_{0 \rightarrow \tau} \in SE(3), \tau \in {1,2,...,t}}$

### NUNOCS

NUNOCS net : ${\phi(P_\mathcal{O}) = (P_{\mathbb{C}, s})}$
- ${\phi}$ : PointNet like architecture
- Input: a scanned partial point cloud
- ${P_\mathcal{O} \in \text{R}^{N \times 3}}$
- Output: point-wise coordinates and scales
- ${P_\mathbb{C} \in \text{R}^{N \times 3}}$ => ${ \text{R}^{N \times 3 \times B}}$ where ${B = 100}$
- ${s = (1, \alpha, \beta) \in \text{R}^{3}}$ (for compactness)
- Inference during online execution
- ${s \circ P_{\mathbb{C}}}$
- establishes a dense correspondence between the NUNOCS representation of ${\mathcal{O}}$ and the NUNOCS shape of ${\mathcal{O}_\mathcal{D}}$ by finding nearest neighbor
- Loss

### Jocobian

Basically, a Jacobian defines the dynamic relationship between two different representations of a system

${f : \text{R}^n \rightarrow \text{R}^m}$ ,where ${f(x) \in \text{R}^m}$, ${x \in \text{R}^n}$

${J_{ij} = \frac{\partial{f_i}}{\partial x_j}}$
- 
- ${f(x + \Delta x) \approx f(x) + J(x)\cdot(\Delta x)}$

${\begin{split}
J &= \frac{\partial x}{\partial q} &= \frac{\partial x}{\partial t} \cdot \frac{\partial t}{\partial q}
\end{split}}$

- ${\dot{x} = J \cdot \dot{q}}$ or
- ${\Delta{x} = J \cdot \Delta{q}}$

### CatBC
- ${\mathcal{J}}$: demonstration trajectory
- ${\bar{\xi}_i}$ : object ${SE(3)}$ pose in demo
- ${\xi_i}$ : object ${SE(3)}$ pose in testing
- ${q}$ : joint configuration

### pose graph optimization

${G = \{V, E\}}$
- ${V}$: i'th frame and object pose ${T_i}$, where ${|V| = k+1}$
- ${E}$: two types of energies
- residuals from fearure correspondense
- dense pixel-wise point-to-plane distance
- Target

### Method: NUNOCS net
Given the single visual demonstration for object ${\mathcal{O}_{\mathcal{D}} \in \mathbb{O}_{\text{train}}}$ and in order to “project” the trajectory so it works for a novel object ${\mathcal{O}}$ during online execution, **category-level data association between** ${\mathcal{O}_{\mathcal{D}}}$ **and** ${\mathcal{O}}$ **is required**. To do so, this work **establishes dense correspondence in a 9-dim**

### true pose of BundleTrack

${T_{\tau}=T_{0\rightarrow \tau} T_0^C \in SE(3)}$

_