---
# System prepended metadata

title: 'GAN-Control: Explicitly Controllable GANs'
tags: [paper]

---

{%hackmd @themes/dracula %}

![](https://hackmd.io/_uploads/BkpaatMeT.png)

Goal: controllable GAN

Face GAN
original GAN: can't control the feature of image
focus on how to generate the high-quality images
![](https://hackmd.io/_uploads/S1zNdcMga.png)

==controllable GAN==

3DMM
![](https://hackmd.io/_uploads/rJMXIczgp.png)
https://arxiv.org/abs/2004.11660
issue: synthetic-to-real domain gap
在3DMM model 的範圍有可能會不夠

proposed method
![](https://hackmd.io/_uploads/Hy_ylBBla.png)

==Disentanglement by contrastive learning==
![](https://hackmd.io/_uploads/HJjFo5Gg6.png)
the latent space is divided into sub-spaces, each encoding a different image property.

**latent space**
original batch
|latent space
|-
|$z_1$
|$z_2$
|$z_3$
|$z_4$
|$z_5$
|$z_6$

cut original batch
|pose|expression|color
|-|-|-
|$z_1^1$|$z_1^2$|$z_1^3$
|$z_2^1$|$z_2^2$|$z_2^3$
|$z_3^1$|$z_3^2$|$z_3^3$
|$z_4^1$|$z_4^2$|$z_4^3$
|$z_5^1$|$z_5^2$|$z_5^3$
|$z_6^1$|$z_6^2$|$z_6^3$

modify to
|pose|expression|color
|-|-|-
|$z_1^1$|$z_1^2$|$z_1^3$
|**$z_1^1$**|$z_2^2$|$z_2^3$
|$z_3^1$|$z_3^2$|$z_3^3$
|$z_4^1$|**$z_3^2$**|$z_4^3$
|$z_5^1$|$z_5^2$|$z_5^3$
|$z_6^1$|$z_6^2$|**$z_5^3$**

**attribute distance**
$d_k(I_i, I_j) = dist(M_k(I_i), M_k(I_j))$
$I$: output of generator
:::success
$M_k: I -> \mathbb{R}^{D_k}$
ID: ArcFace [14]
head-pose: HopeNet [18]
Expression: RSR [20]
Illumination: R-Net [9]
Age: Dex [17]
Hair color: PSPNet [21]
:::

:::success
$dist$
L1
L2
consine-distance
:::

**contrastive loss component for attribute k**
$l_k(z_i, z_j) = \begin{cases} \frac{1}{C_k^+}max(d_k(I_i, I_j) - \tau_k^+, 0), & z_i^k = z_j^k \\\frac{1}{C_k^-}max(\tau_k^--d_k(I_i, I_j), 0), & otherwise \end{cases}$ 

$C_k^+ = \sum_{i,j}\mathbb{I}\{z_i^k = z_j^k\}$
$C_k^- = \sum_{i,j}\mathbb{I}\{z_i^k = z_j^k\}$

**contrastive loss**
![](https://hackmd.io/_uploads/BJz43EHg6.png)


==Interpretable explicit control==
![](https://hackmd.io/_uploads/S1MBSVHea.png)
![](https://hackmd.io/_uploads/HJ8xZrBl6.png)
dataset {y, w}

==Inference==
![](https://hackmd.io/_uploads/Sy42ZBHga.png)

## experiment

FHHQ dataset [29] downsample to 512x512 resolution
**FID**
![](https://hackmd.io/_uploads/rJnDzBSep.png)
![](https://hackmd.io/_uploads/SySQQSBlp.png)
![](https://hackmd.io/_uploads/S1ghmHreT.png)

**control precision**
![](https://hackmd.io/_uploads/BJZXIBHeT.png)
![](https://hackmd.io/_uploads/Hku-DHHlT.png)

**Qualitative evaluation**
![](https://hackmd.io/_uploads/B1ctAHBgT.png)
![](https://hackmd.io/_uploads/ryVqCSBxT.png)
