---
# System prepended metadata

title: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

---

# Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

---

[TOC]

---

## Abstract
- We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. 
- GOAL
    1. ![](https://i.imgur.com/4S1XOIL.png)
        - G(X) is indistinguishable from the distribution Y
    2. ![](https://i.imgur.com/GXEXhAd.png)
        - an inverse mapping
    3. ![](https://i.imgur.com/3fUzl4V.png)
        - a new cycle consistency loss

## Introduction
- Goal
    - capturing special characteristics of one image collection and figuring out how these characteristics could be translated into the other image collection, all in the absence of any paired training examples.
    - converting an image from one representation to another
- Contribution
    - obtaining paired training data can be difficult and expensive
    - only a couple of datasets exist for tasks
    - Obtaining input-output pairs for graphics
    - more difficult since the desired output is highly complex
    - the desired output is not even well-defined
-  How
    -  training both the mapping G and F simultaneously, and adding a cycle consistency loss that encourages F (G(x)) ≈ x and G(F (y)) ≈ y

## Related work
- Generative Adversarial Networks (GANs)
    - The key to GANs’ success is the idea of an adversarial loss that forces the generated images to be, in principle, indistinguishable from real photos.
- Image-to-Image Translation
    - Our approach builds on the “pix2pix” framework
        - uses a conditional generative adversarial network to learn a mapping from input to output images 
- Unpaired Image-to-Image Translation
    - on any task-specific, predefined similarity function between the input and output, nor do we assume that the input and output have to lie in the same low-dimensional embedding space
    - general-purpose solution for many vision and graphics tasks
- Cycle Consistency
    - enforcing simple forward-backward consistency has been a standard trick
- Neural Style Transfer
    - trying to capture correspondences between higher-level appearance structures > single sample transfer 

## Formulation
- 整體概念(2 GAN/autoencoder)
    ![](https://i.imgur.com/0wpQWBG.png)
- domains
    - X
    - Y
- data distribution：
    - x ∼ pdata(x)
    - y ∼ pdata(y)
- two mappings(generator)
    - G: X → Y
    - F: Y → X
- two adversarial discriminators
    - D$_X$
        - distinguish
            - {x}
            - {F(y)}
    - D$_Y$
      - distinguish
            - {y}
            - {G(x)}
- contains two types of terms
    - adversarial losses
        - 公式
            - ![name=Andy zero](https://i.imgur.com/Z6fAp9i.png)
        - 用法
            - ![](https://i.imgur.com/uwBcSHp.png)
            - ![](https://i.imgur.com/g1ea0rx.png)
    - cycle consistency losses
        - 概念
            - ![](https://i.imgur.com/BN6qvDW.png)
            - ![](https://i.imgur.com/5xIEDkw.png)
        - 公式
            - ![](https://i.imgur.com/YpvXwB6.png)
        - 用法
            - min![](https://i.imgur.com/Kd48KLm.png)
:::danger
- Full Objective
    - ![](https://i.imgur.com/ztPTrVw.png)
        - λ:
            - controls the relative importance of the two objectives
- Train
    - ![](https://i.imgur.com/cr0MLua.png)
:::

## Implementation
- Network Architecture
    - generative networks(from Johnson)
        - three convolutions
            - two fractionally-strided with stride 1/2
            - one maps features to RGB 
        - several residual blocks
            - 128 × 128 images(6 blocks)
            - 256 × 256 and higher-resolution images(9 blocks)
        - instance normalization
    - discriminator networks(from PatchGANs)
        - 70 × 70 PatchGANs
            - a patch-level discriminator architecture has fewer parameters than a full-image discriminator and can work on arbitrarily-sized images in a fully convolutional fashion
- Training details
    - use least-squares loss
        - ![](https://i.imgur.com/ybbGpVs.png)
            - ![](https://i.imgur.com/vgNZkO7.png)
            - ![](https://i.imgur.com/ECI5Fjq.png)![](https://i.imgur.com/Iiz6hus.png)
    - reduce model oscillation(Shrivastava et al.’s strategy)
        - update the discriminators using a history of generated images rather than the ones produced by the latest generators
        - mage buffer that stores the 50 previously created images
    - set λ = 10
    - use Adam solver
    - 1 batch size
    - 0.0002 learning rate
        - 0~100 epochs = 0.0002
        - 101~200 epochs linearly decay to 0

## Results
1. our approach V.S. recent methods
    - use same evaluation datasets and metrics as “pix2pix“
2. our full method V.S. several variants
    - for study the importance of both the adversarial loss and the cycle consistency loss
3. demonstrate the generality
