# Energy function, partial differential and math of BodyFusion

[Out of update] Fusion uses numeric differential now.

## 1. Introduction
In this topic, the energy function and partial differential are described here. The needed properties of matrix, optimization and calculus are gathered here. We will describe how to do the partial differential with the detailed explanation. The derivation can give engineers a guidance to write/implement the code.

## 2. Energy function

The energy function of BodyFusion is
\begin{aligned}
E_t = λ_nE_{nonrigid} +λ_sE_{skeleton} +λ_gE_{graph} +λ_bE_{binding}
\end{aligned}

There are 4 terms. we will talk about each of them in the following sections and derive the partial differential w.r.t $\alpha$. This is a non-linear least-squares problem which can be solved by Gauss-Newton method [1].
Here is the notations table showing the notations used in the energy function and derivation process.

| Notation | Description |
| -------- | -------- |
| $\mathcal{D}$ | Depth sequence |
| $\mathcal{N()}$ | The neighboring vertices. |
| $\mathcal{P}$ | The domain of valid pairs. |
| $\text{v}$ | vertex coordinate |
| $\hat{\text{v}}$ | vertex coordinate warped by **nodes** |
| $\hat{\text{n}}$ | normal vector warped by nodes |
| $\hat{\text{u}}$ | corresponding vertex on the depth map |
| $\text{T}_{nj}$ | SE(3) deformation associated with the *j*th node |
| $\omega_{i,j}$ | weight of the *j*th node influencing $\text{v}$ |
| $\tilde{\text{v}}$ | vertex coordinate warped by the **skeleton** |
| $\tilde{\text{n}}$ | normal vector warped by the **skeleton** |
| $\text{T}_{bj}$ | SE(3) accumulated motion of the $j$th bone |
| $\eta_{i,j}$ | The attachment weight with which the $j$th bone influences $v_i$ |
| $\mathcal{H}_{n_i}$ | The skinning weights of node $n_i, \mathcal{H}_{n_i}=[\eta_{i,1},\cdots,\eta_{i,m}]$ |
| $m$ | The number of bones |
| $\text{x}_i$ | The coordinate of the $i$th node |
| $\tilde{\text{x}}_i$ | Coordinate of the $i$th node warped by the skeleton kinematic |

Table 1. Notations

| Notation | Description | Notation | Description |
| -------- | ------------- | -------- | ----------- |
| $\alpha$ | x rotation | $t_x$ | x translation |
| $\beta$ | y rotation | $t_y$ | x translation |
| $\gamma$ | z rotation | $t_z$ | x translation |

Table 2. Variables to be solve in each node

---
### 2.1 Data term
Nonrigid energy and skeleton energy are classified to the data term.
### 2.1.1 Nonrigid energy $E_{nonrigid}$
* Energy formula:
\begin{aligned}
E_{nonrigid} = \sum\limits_{(v_i,u_i)∈\mathcal{P}}
|\hat{\text{n}}_{v_{i}}(\hat{\text{v}}_i-{\text{u}_i})|^2,
\end{aligned}

\begin{aligned}
\text{where}\ \ \hat{\text{v}}_i = \sum\limits_{j∈\mathcal{N}(v_i)} ω_{i,j}\text{T}_{n_j}\text{v}_i\ ,\
\hat{\text{n}}_{v_i}=\sum\limits_{j∈\mathcal{N}(v_i)} ω_{i,j}\text{T}_{n_j}\text{n}_{v_i}\
\end{aligned}

So, the complete nonrigid energy formula can be written as
\begin{aligned}
E_{nonrigid} = \sum\limits_{(v_i,u_i)∈\mathcal{P}}
|(\sum\limits_{j∈\mathcal{N}(v_i)} ω_{i,j}\text{T}_{n_j}\text{n}_{v_i})\ ((\sum\limits_{j∈\mathcal{N}(v_i)} ω_{i,j}\text{T}_{n_j}\text{v}_i)-\text{u}_i)|^2,
\end{aligned}

* Partial differential:
Please look at the partial differential in skeleton energy. These 2 energy terms follow the same derivation process. The difference is that they use different vertex, normal (nonrigid uses [$\hat{\text{v}}$, $\hat{\text{n}}$] and skeleton use [$\tilde{\text{v}}$, $\tilde{\text{n}}$]) and weightings ($\omega, \eta$).

---
### 2.1.2 Skeleton energy $E_{skeleton}$
**Energy formula:**
\begin{aligned}
E_{skeleton} = \sum\limits_{(v_i,u_i)∈\mathcal{P}}
|\tilde{\text{n}}_{v_{i}}(\tilde{\text{v}}_i-{\text{u}_i})|^2,
\end{aligned}

\begin{equation}
\text{where}\ \ \tilde{\text{v}}_i = \sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j}\text{v}_i\ ,\
\tilde{\text{n}}_{v_i}=\sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j}\text{n}_{v_i}, \\
\text{where}\ \eta_{i,j}\begin{cases}
1 & \text{if}\ j\text{th bone is most closely related to node }n_i \\
0 & \text{otherwise}\
\end{cases} \\
\mathcal{H_{n_i} =[\eta_{i,1},...,\eta_{i,m}]} \\
\eta_{i,j} \in \mathcal{H} =
\begin{pmatrix}
\eta_{11} & \eta_{21} & \cdots & \eta_{n1}\\
\eta_{12} & \eta_{22} & \cdots & \eta_{n2}\\
\vdots & \vdots & \ddots & \vdots\\
\eta_{1m} & \cdots & \cdots & \eta_{nm}
\end{pmatrix} \in \mathcal{R}^{m\ \text{bones x}\ n\ \text{nodes}}
\end{equation}

Smooth the node attachments by averaging $\mathcal{H}_{n_i}$ over the node graph using 8 neighbors. Perform this step iteratively until attachment weights have been assigned to all nodes.

So, the complete skeleton energy formula can be written as
\begin{equation}
E_{skeleton} = \sum\limits_{(v_i,u_i)∈\mathcal{P}}
|(\sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j}\text{n}_{v_i})((\sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j}\text{v}_i)-{\text{u}_i})|^2, \\
where\ \mathcal{H}_{v_i} = \frac{1}{Z}\sum_{k\in\mathcal{N}(v_i)}\lambda_{i,k}\mathcal{H}_{nk}, \\
\end{equation}

**Partial differential $\frac{\partial E_{skeleton}}
{\partial \alpha}$:**
Let $f = \tilde{\text{n}}(\tilde{\text{v}}-{\text{u}})$

\begin{aligned}
\frac{\partial E_{skeleton}}{\partial \alpha} =
\frac{\partial f}{\partial \alpha}
\end{aligned}

Let $T =\sum\limits_{j∈\mathcal{N}(v_i)}η_{i,j}\text{T}_{b_j}$

\begin{equation}
\frac{\partial f}{\partial \alpha} = tr(
\frac{\partial f}{\partial T} \cdot
\frac{\partial T}{\partial \alpha}),\ tr\text{ is trace()}
\end{equation}

Since $\tilde{\text{v}}_i = (\sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j})\text{v}_i\ ,\
\tilde{\text{n}}_{v_i}= (\sum\limits_{j∈\mathcal{N}(v_i)} η_{i,j}\text{T}_{b_j})\text{n}_{v_i}$

\begin{equation}
\frac{\partial f}{\partial T} = T(nv^T + vn^T) - v_ln^T,[7]\\
tr(AB) = \text{vec}(A)^T \cdot \text{vec}(B)
\end{equation}

Continue on $\frac{\partial T}{\partial \alpha}$, denote dual quaternion $q$ is composed by $(q^0,q^1,q^2,q^3,q^4,q^5,q^6,q^7)$ which are identical to $(w_0,x_0,y_0,z_0,w_1,x_1,y_1,z_1)$

\begin{equation}
\frac{\partial T}{\partial \alpha} =
\sum\limits_{j∈\mathcal{N}(v_i)}\frac{\partial SE3(\hat{q})}{\partial \hat{q}^{i}} \cdot
\frac{\partial\hat{q}^i}{\partial \alpha}
\end{equation}

where $SE3(\hat{q})$ is the transformation matrix of dual quaternion [2]
\begin{equation}
SE3(\hat{q}) =
\begin{bmatrix}
1-2(\hat{y}_0^2+\hat{z}_0^2) && 2(\hat{x}_0\hat{y}_0-\hat{z}_0\hat{w}_0) && 2(\hat{x}_0\hat{z}_0+\hat{y}_0\hat{w}_0) && 2(-\hat{w}_1\hat{x}_0+\hat{x}_1\hat{w}_0-\hat{y}_1\hat{z}_0+\hat{z}_1\hat{y}_0) \\
2(\hat{x}_0\hat{y}_0+\hat{w}_0\hat{z}_0) && 1-2(\hat{x}_0^2+\hat{z}_0^2) && 2(\hat{y}_0\hat{z}_0-\hat{x}_0\hat{w}_0) && 2(-\hat{w}_1\hat{y}_0+\hat{x}_1\hat{z}_0+\hat{y}_1\hat{w}_0-\hat{z}_1\hat{x}_0) \\
2(\hat{x}_0\hat{z}_0-\hat{y}_0\hat{w}_0) && 2(\hat{y}_0\hat{z}_0+\hat{x}_0\hat{w}_0) && 1-2(\hat{x}_0^2+\hat{y}_0^2) && 2(-\hat{w}_1\hat{z}_0-\hat{x}_1\hat{y}_0+\hat{y}_1\hat{x}_0+\hat{z}_1\hat{w}_0) \\
0 && 0 && 0 && 1
\end{bmatrix}
\end{equation}

The partial differential $SE3(\hat{q})$ to the 8 components of dual quaternion are

\begin{equation}
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(0)} \equiv \hat{w}_0)} = 2
\begin{bmatrix}
0 & -\hat{z}_0 & \hat{y}_0 & \hat{x}_1 \\
\hat{z}_0 & 0 & -\hat{x}_0 & \hat{y}_1 \\
-\hat{y}_0 & \hat{x}_0 & 0 & \hat{z}_1 \\
0 & 0 & 0 & 0
\end{bmatrix}
\ \ \ \ \ \
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(1)} \equiv \hat{x}_0)} = 2
\begin{bmatrix}
0 & \hat{y}_0 & \hat{z}_0 & -\hat{w}_1 \\
\hat{y}_0 &-2\hat{x}_0 & -\hat{w}_0 & -\hat{z}_1 \\
\hat{z}_0 & \hat{w}_0 & -2\hat{x}_0 & \hat{y}_1 \\
0 & 0 & 0 & 0
\end{bmatrix}
\end{equation}

\begin{equation}
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(2)} \equiv \hat{y}_0)} = 2
\begin{bmatrix}
-2\hat{y}_0 & \hat{x}_0 & \hat{w}_0 & \hat{z}_1 \\
\hat{x}_0 & 0 & \hat{z}_0 & -\hat{w}_1 \\
-\hat{w}_0 & \hat{z}_0 & -2\hat{y}_0 & -\hat{x}_1 \\
0 & 0 & 0 & 0
\end{bmatrix}
\ \ \ \ \ \
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(3)} \equiv \hat{z}_0)} = 2
\begin{bmatrix}
-2\hat{z}_0 & -\hat{w}_0 & \hat{x}_0 & -\hat{y}_1 \\
\hat{w}_0 &-2\hat{z}_0 & \hat{y}_0 & \hat{x}_1 \\
\hat{x}_0 & \hat{y}_0 & 0 & -\hat{w}_1 \\
0 & 0 & 0 & 0
\end{bmatrix}
\end{equation}

\begin{equation}
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(4)} \equiv \hat{w}_1)} = 2
\begin{bmatrix}
0 & 0 & 0 & -\hat{x}_0 \\
0 & 0 & 0 & -\hat{y}_0 \\
0 & 0 & 0 & -\hat{z}_0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix}
\ \ \ \ \ \
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(5)} \equiv \hat{x}_1)} = 2
\begin{bmatrix}
0 & 0 & 0 & \hat{w}_0 \\
0 & 0 & 0 & \hat{z}_0 \\
0 & 0 & 0 & -\hat{y}_0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix}
\end{equation}

\begin{equation}
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(6)} \equiv \hat{y}_1)} = 2
\begin{bmatrix}
0 & 0 & 0 & -\hat{z}_0 \\
0 & 0 & 0 & \hat{w}_0 \\
0 & 0 & 0 & \hat{x}_0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix}
\ \ \ \ \ \
\frac{\partial SE3(\hat{q})}{\partial (\hat{q}^{(7)} \equiv \hat{z}_1)} = 2
\begin{bmatrix}
0 & 0 & 0 & \hat{y}_0 \\
0 & 0 & 0 & -\hat{x}_0 \\
0 & 0 & 0 & \hat{w}_0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix}
\end{equation}

Now, The derivative of dual-quaternion-blending (DQB) term$\frac{\partial\hat{q}^i}{\partial \alpha}$:
Due to DQB $\hat{q} = \sum_kw_k\hat{q}_k$ and the length of $\hat{q}$ is 1. Using the chains rule:

\begin{equation}
\frac{\partial\hat{q}^i}{\partial \alpha_k} =
\sum_{j=0}^7 \frac{\partial\hat{q}^i}{\partial\hat{q}_k^j}
\cdot
\frac{\partial\hat{q}^j_k}{\partial\alpha_k} \\
\frac{\partial\hat{q}^i}{\partial\hat{q}_k^j} =
\begin{cases} \frac{w_k}{\begin{Vmatrix}\hat{q}\end{Vmatrix}} \cdot
\delta_{ij} - \frac{\hat{q}^i\hat{q}^jw_k}{||\hat{q}||^3}
, & \text{if}\ 0\leq j\leq 3 \\
\frac{w_k}{\begin{Vmatrix}\hat{q}\end{Vmatrix}} \cdot
\delta_{ij} , & \text{if}\ 4\leq j\leq 7
\end{cases}
\end{equation}

Here $\delta_{ij}$ is the response function $i == j?1 : 0$

The last item is the dual-quaternion-exponential term $\frac{\partial\hat{q}^j_k}{\partial\alpha_k}$. We use twist (exponential) to update the energy function but using dual quaternion to do DQB. So need to convert to twist.
Denote $\theta_k = (\alpha_k, \beta_k, \gamma_k)^T
, t_k = (t_{xk}, t_{yk}, t_{zk})^T$, then using the exponential map
\begin{cases}
\hat{q}^0_k = \cos||\theta_k|| \\
\hat{q}^1_k = \frac{\alpha_k}{||\theta_k||}\sin||\theta_k|| \\
\hat{q}^2_k = \frac{\beta_k}{||\theta_k||}\sin||\theta_k|| \\
\hat{q}^3_k = \frac{\gamma_k}{||\theta_k||}\sin||\theta_k|| \\
\hat{q}^4_k = -\frac{1}{2}(+t_{xk}\hat{q}_k^1+t_{yk}\hat{q}_k^2+t_{zk}\hat{q}_k^3) \\
\hat{q}^5_k = +\frac{1}{2}(+t_{xk}\hat{q}_k^0+t_{yk}\hat{q}_k^3-t_{zk}\hat{q}_k^2) \\
\hat{q}^6_k = +\frac{1}{2}(-t_{xk}\hat{q}_k^3+t_{yk}\hat{q}_k^0+t_{zk}\hat{q}_k^1) \\
\hat{q}^7_k = +\frac{1}{2}(+t_{xk}\hat{q}_k^2-t_{yk}\hat{q}_k^1+t_{zk}\hat{q}_k^0) \\
\end{cases}

Then do the partial differential of $(\alpha_k, \beta_k, \gamma_k,t_{xk}, t_{yk}, t_{zk})$.
If $||\theta||\neq0$, let $b=\frac{\sin||\theta_k||}{||\theta_k||}, c = \frac{\cos||\theta_k||}{||\theta||^2} - \frac{\sin||\theta_k||}{||\theta||^3}$
\begin{cases}
\frac{\partial\hat{q}_k}{\partial\alpha_k} = (-\alpha_kb,\ b+\alpha_kc,\ \alpha_k\beta_kc,\ \alpha_k\gamma_kc, ......)^T \\
\frac{\partial\hat{q}_k}{\partial\beta_k} = (-\beta_kb,\ \beta_k\alpha_kc,\ b+\beta_k\beta_kc,\ \beta_k\gamma_kc ......)^T \\
\frac{\partial\hat{q}_k}{\partial\gamma_k} = (-\gamma_kb,\ \gamma_k\alpha_kc,\ \gamma_k\beta_kc,\ b+\gamma_k\gamma_kc, ......)^T \\
\frac{\partial\hat{q}_k}{\partial t_{xk}} = \frac{1}{2}(0,0,0,0,-\hat{q}^1_k,\hat{q}^0_k,-\hat{q}^3_k,\hat{q}^2_k)^T \\
\frac{\partial\hat{q}_k}{\partial t_{yk}} = \frac{1}{2}(0,0,0,0,-\hat{q}^2_k,\hat{q}^3_k,\hat{q}^0_k,-\hat{q}^1_k)^T \\
\frac{\partial\hat{q}_k}{\partial t_{xk}} = \frac{1}{2}(0,0,0,0,-\hat{q}^3_k,-\hat{q}^2_k,\hat{q}^1_k,\hat{q}^0_k)^T
\end{cases}

If $||\theta||=0$, according to [3], $b=1$,
\begin{cases}
\frac{\partial\hat{q}_k}{\partial\alpha_k} = (0,1,0,0, ......)^T \\
\frac{\partial\hat{q}_k}{\partial\beta_k} = (0,0,1,0 ......)^T \\
\frac{\partial\hat{q}_k}{\partial\gamma_k} = (0,0,0,1, ......)^T \\
\frac{\partial\hat{q}_k}{\partial t_{xk}} = \frac{1}{2}(0,0,0,0,0,1,0,0)^T \\
\frac{\partial\hat{q}_k}{\partial t_{yk}} = \frac{1}{2}(0,0,0,0,0,0,1,0)^T \\
\frac{\partial\hat{q}_k}{\partial t_{xk}} = \frac{1}{2}(0,0,0,0,0,0,0,1)^T
\end{cases}

By using the chain rule, the skeleton energy is decomposed clearly.

---
### 2.2 Smooth term $E_{graph}$
* Energy formula:
\begin{equation}
E_{graph} = \sum\limits_i \sum\limits_{j∈\mathcal{N}(i)}\psi_{reg}(\mathcal{H}_{n_i}, \mathcal{H}_{n_j})||\text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j||^2_2, \\
\text{where}\ \psi(\mathcal{H}_{n_i}, \mathcal{H}_{n_j}) = \rho(||\mathcal{H}_{ni}-\mathcal{H}_{nj}||^2_2), \text{(see norm types at [6])}\\
\rho(\cdot) \text{is a Huber kernel with threshold 0.2}
\end{equation}

Huber kernel is basically a combination of squared loss and absolute loss.
\begin{equation}
\text{Huber}(y) =
\begin{cases}
\frac{1}{2}(y)^2 & |y|\le\delta \\
\delta(|y|-\frac{1}{2}\delta) & \text{otherwise}
\end{cases}
\end{equation}

Here is the Huber function plot.

So, the complete graph energy formula can be written as
\begin{equation}
E_{graph} = \sum\limits_i \sum\limits_{j∈\mathcal{N}(i)}\rho(||\mathcal{H}_{ni}-\mathcal{H}_{nj}||^2_2)||\text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j||^2_2
\end{equation}

* Partial differential:
What we want to solve in the graph energy is the 6 variables ($\alpha, \beta, \gamma, t_x, t_y, t_z$) in the node transforamtion $\text{T}$.
So, $\rho(||\mathcal{H}_{ni}-\mathcal{H}_{nj}||^2_2)$ can be seen as a constant. Let's focus on the $||\text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j||^2_2$. More specifically, is $\text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j$

Denote $h = \text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j$, take $\alpha_i$ and $\alpha_j$ for the example,

\begin{equation}
\begin{cases}
\frac{\partial h^k}{\partial \alpha_i} =
tr((\frac{\partial h^k}{\partial \text{T}_{ni}})^T \frac{\partial \text{T}_{ni}}{\partial \alpha_i}) \\
\frac{\partial h^k}{\partial \alpha_j} =
tr((\frac{\partial h^k}{\partial \text{T}_{nj}})^T \frac{\partial \text{T}_{nj}}{\partial \alpha_j}) \\
\end{cases} \\
\text{k indicates an element of the vector, $h$ =}
\begin{bmatrix}
h^{(0)} \\
h^{(1)} \\
h^{(2)} \\
1 \\
\end{bmatrix} =
\begin{bmatrix}
h_x \\
h_y \\
h_z \\
1 \\
\end{bmatrix} =
\end{equation}

\begin{equation}
h = \text{T}_{n_i}\text{x}_j − \text{T}_{n_j}\text{x}_j =
\begin{bmatrix}
a & b & c & d \\
e & f & g & h \\
i & j & k & l \\
0 & 0 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix}
\text{x}_{jx} \\
\text{x}_{jy} \\
\text{x}_{jz} \\
1 \\
\end{bmatrix} -
\begin{bmatrix}
m & n & o & p \\
q & r & s & t \\
u & v & w & x \\
0 & 0 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix}
\text{x}_{jx} \\
\text{x}_{jy} \\
\text{x}_{jz} \\
1 \\
\end{bmatrix} =
\begin{bmatrix}
h_x \\
h_y \\
h_z \\
1 \\
\end{bmatrix} \\
h_x = h^{(0)} = (a-m)\text{x}_{jx} + (b-n)\text{x}_{jy} + (c-o)\text{x}_{jz} + (d-p)×1
\end{equation}

Then , \begin{equation}
\frac{\partial h^{(0)}}{\partial \text{T}_{ni}} =
\begin{bmatrix}
\text{x}_{jx} & \text{x}_{jy} & \text{x}_{jz} & 1 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix}, \\
\text{we also can get}
\frac{\partial h^{(1)}}{\partial \text{T}_{ni}} =
\begin{bmatrix}
0 & 0 & 0 & 0 \\
\text{x}_{jx} & \text{x}_{jy} & \text{x}_{jz} & 1 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
\end{bmatrix} \text{and} \
\frac{\partial h^{(2)}}{\partial \text{T}_{ni}} =
\begin{bmatrix}
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
\text{x}_{jx} & \text{x}_{jy} & \text{x}_{jz} & 1 \\
0 & 0 & 0 & 0 \\
\end{bmatrix} \\
\end{equation}
We can derive $\frac{\partial h^{(k)}}{\partial \text{T}_{nj}}$ in the same way.

In the data term of DynamicFusion,
\begin{equation}
\frac{\partial \hat{T}}{\partial \alpha_k} = T_{lw}\sum_{i=0}^7
\frac{\partial SE3(\hat{q})}{\partial \hat{q}^{(i)}} \cdot
\sum_{j=0}^7
\frac{\partial\hat{q}^{(i)}}{\partial\hat{q}_k^{(j)}} \cdot
\frac{\partial\hat{q}_k^{(j)}}{\partial\alpha_k}
\end{equation}

For $\frac{\partial \text{T}_{ni}}{\partial \alpha_i}$, $\text{T}_{ni}$ is converted from the single dual qutaternion rather than the 8 dual quaternions.
So,
\begin{equation}
\frac{\partial T_{ni}}{\partial \alpha_i} = \sum_{k=0}^7
\frac{\partial SE3(\hat{q})}{\partial \hat{q}^{(k)}} \cdot
\frac{\partial\hat{q}^{(k)}}{\partial\alpha_i}
\end{equation}
The computation is similar with previous described.

23 is similar with previous described.
---
### 2.3 Binding term $E_{binding}$
* Energy formula:
\begin{equation}
E_{binding} = \sum_{i=1}^{N}||\text{T}_{n_i}\text{x}_i − \tilde{\text{x}}_i||^2_2 \ \ \\
\text{where}\ \tilde{\text{x}}_i = \text{T}_{bj}\text{x}_i
\end{equation}
$N$ is the number of node.

So, the complete binding energy formula can be written as
\begin{equation}
E_{binding} = \sum_{i=1}^{N}||\text{T}_{n_i}\text{x}_i − \text{T}_{bj}\text{x}_i||^2_2
\end{equation}

* Partial differential:
Look into the binding formula, $\text{T}_{bj}$ is given by the sensor body tracking SDK and servered as a constant transformation matrix in each frame. $\text{x}_i$ is the coordinate of $i$th node and $\text{T}_{n_i}$ is the correpsonding transformation matrix which consists of a dual-quaternion .
We try to solve the dual-quaternion in twist representation $\alpha, \beta, \gamma, t_x, t_y\ \text{and}\ t_z$.

Denote $h = \text{T}_{n_i}\text{x}_i − \text{T}_{bj}\text{x}_i$, take $\alpha_i$ for the example,
\begin{equation}
\frac{\partial h^k}{\partial \alpha_i} =
tr((\frac{\partial h^k}{\partial \text{T}_{n_i}})^T \frac{\partial \text{T}_{n_i}}{\partial \alpha_i})
\end{equation}

Again, $\frac{\partial \text{T}_{n_i}}{\partial \alpha_i}$ is similar with previous described.

## 3. Supplemental math characteristic
* [1] The normal euation of Gauss-Newton is $J^TJx = J^Te$, where $J$ is the Jacobian matrix, $x$ is the variable to be solved and $e$ is the residual.
* [2] The formula is on page 6. [Skinning with Dual Quaternions](https://www.cs.utah.edu/~ladislav/kavan07skinning/kavan07skinning.pdf)
* [3] Assume matrix $A$ is $n×n$ but of rank r < n. In that case, the system $Ax = b$ is solved by $x = A^+b$ where A+ is the pseudo-inverse of the rank-deficient matrix.
* [4] $\lim\limits_{x\rightarrow0}\frac{\sin(x)}{x} = 1$
* [5] $\frac{d|x|}{dx} = \frac{x}{|x|}, x \neq 0$
* [6] For the math symbol of norm $||\cdot||_p^n$, giving some examples below as the explanation. For the superscript $n$, it is just the $n$ power of the value p-norm, $\text{p-norm}^n$.
\begin{equation}
\text{norm}
\begin{cases}
||v||_1 = \sum_{i=1}^n|v_i|, & \text{1-norm} \\
||v||_2 = \sqrt[]{\sum_{i=1}^n|v_i|^2} = \sqrt[]{v^Tv} & \text{2-norm, also called Euclidean norm, L}^2 \text{ norm, or square norm} \\
||v||_p = \sqrt[p]{\sum_{i=1}^n|v_i|^p} & \text{p-norm} \\
||v||_\infty = \max|v_i|, & \text{Infinity norm}
\end{cases} \\
\end{equation}
* [7] According to the [link](https://jonathan-hui.medium.com/machine-learning-linear-algebra-a5b1658f0151)
$\frac{n^TT^TTv}{dT} = T(nv^T+vn^T)$, $\frac{n^TT^Tv_l}{dT} = v_ln^T$ where $v,n\ and\ v_l \in \mathbb{R}^{\text{4x1}}, T \in \mathbb{R}^{\text{4x4}}$

* [8] Let $U$ and $V$ be 2 nxn matrices, and the inner product symbol is $<,>$. $<U,V>=tr(U^TV)$

* [9] The Jacobian matrix of DynamicFusion $\in \mathbb{R}^\text{(nPixel+node*8*3)×(node×6)}$. The data term is in the upper part and the reg term is in the lower part.
In the data term, number 1~8 represent the derivatives of $(\alpha, \beta, \gamma, x, y, z)$ from the specific 8 node.
In the reg term, number 0 represent the original node, number 1~8 represent the 8 KNN nodes. For each knn node, there are 3 rows because $h$ is a vector of $(h_x,h_y,h_z)$.

\begin{equation}
\left[
\begin{array} \\
&\cdots &111111 &\cdots &222222 &\cdots &333333 &\cdots &444444 &\cdots &555555 &\cdots &666666 &\cdots &777777 &\cdots &888888 &\cdots \\
&\cdots &&111111 &\cdots & &222222 &\cdots &333333 &\cdots &444444 &\cdots &555555 &\cdots &666666 &\cdots &777777 &\cdots &888888 &\cdots \\
&&&&&&&&&\vdots \\
\hline
&000000 &\cdots &&&&111111 \\
&000000 &\cdots &&&&111111 \\
&000000 &\cdots &&&&111111 \\
&000000 &\cdots &222222 \\
&000000 &\cdots &222222 \\
&000000 &\cdots &222222 \\
&&&&&&&&&\vdots \\
&000000 &\cdots &&&&&&888888 \\
&000000 &\cdots &&&&&&888888 \\
&000000 &\cdots &&&&&&888888 \\
&\cdots &&&000000 &\cdots &111111 \\
&\cdots &&&000000 &\cdots &111111 \\
&\cdots &&&000000 &\cdots &111111 \\
&&&&&&&&&\vdots \\
\end{array}
\right]
\end{equation}