{%hackmd @themes/dracula %}


Goal: controllable GAN

Face GAN
original GAN: can't control the feature of image
focus on how to generate the high-quality images

==controllable GAN==

3DMM
https://arxiv.org/abs/2004.11660
issue: synthetic-to-real domain gap
在3DMM model 的範圍有可能會不夠

proposed method

==Disentanglement by contrastive learning==
the latent space is divided into sub-spaces, each encoding a different image property.

**latent space**
original batch
|latent space
|-
|$z_1$
|$z_2$
|$z_3$
|$z_4$
|$z_5$
|$z_6$

cut original batch
|pose|expression|color
|-|-|-
|$z_1^1$|$z_1^2$|$z_1^3$
|$z_2^1$|$z_2^2$|$z_2^3$
|$z_3^1$|$z_3^2$|$z_3^3$
|$z_4^1$|$z_4^2$|$z_4^3$
|$z_5^1$|$z_5^2$|$z_5^3$
|$z_6^1$|$z_6^2$|$z_6^3$

modify to
|pose|expression|color
|-|-|-
|$z_1^1$|$z_1^2$|$z_1^3$
|**$z_1^1$**|$z_2^2$|$z_2^3$
|$z_3^1$|$z_3^2$|$z_3^3$
|$z_4^1$|**$z_3^2$**|$z_4^3$
|$z_5^1$|$z_5^2$|$z_5^3$
|$z_6^1$|$z_6^2$|**$z_5^3$**

**attribute distance**
$d_k(I_i, I_j) = dist(M_k(I_i), M_k(I_j))$
$I$: output of generator
:::success
$M_k: I -> \mathbb{R}^{D_k}$
ID: ArcFace [14]
head-pose: HopeNet [18]
Expression: RSR [20]
Illumination: R-Net [9]
Age: Dex [17]
Hair color: PSPNet [21]
:::

:::success
$dist$
L1
L2
consine-distance
:::

**contrastive loss component for attribute k**
$l_k(z_i, z_j) = \begin{cases} \frac{1}{C_k^+}max(d_k(I_i, I_j) - \tau_k^+, 0), & z_i^k = z_j^k \\\frac{1}{C_k^-}max(\tau_k^--d_k(I_i, I_j), 0), & otherwise \end{cases}$

$C_k^+ = \sum_{i,j}\mathbb{I}\{z_i^k = z_j^k\}$
$C_k^- = \sum_{i,j}\mathbb{I}\{z_i^k = z_j^k\}$

**contrastive loss**

==Interpretable explicit control==
dataset {y, w}

==Inference==

## experiment

FHHQ dataset [29] downsample to 512x512 resolution
**FID**

**control precision**

**Qualitative evaluation**