# Notes on "FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation" [[Paper]](https://arxiv.org/pdf/2011.09230.pdf)

###### tags: `notes` `unsupervised` `domain-adaptation`

Notes Author: [Rohit Lal](https://rohitlal.net/)

---

## Brief Outline
- propose a UDA method that effectively handles large domain discrepancies.
- introduce a fixed ratio-based mixup to augment multiple intermediate domains between the source and target domain.

## Introduction

- an augmented domain close to the source domain has more reliable label information, but it has a lower correlation with the target domain.
- apply self penalization, which penalizes its own model to improve performance through self-training.
- to prevent divergence of the augmented models generated in different domains, we propose a consistency regularization using an augmented domain with the same ratio of source and target samples.

## Methodology
FixBi algorithm

### Fixed Ratio based Mixup
- Given a pair of input samples and their corresponding one-hot labels in the source and target domain: $(x_i^s, y_i^s)$ and $(x_i^t, \hat{y}_i^t)$, mixup settings are defined as follows:
Note: $\hat{y}_i^t$ obtained from pseudo labels got from DANN
$\lambda \in \{\lambda_{sd}, \lambda_{td}\}$ such that $\lambda_{sd} + \lambda_{td}\ = 1$
- Taking advantage of the fixed ratio-based mixup, we construct two network models that act as bridges between the source and target domain
- The source-dominant model has strong supervision for the source domain but relatively weak supervision for the target domain. By contrast, the target-dominant model has strong target supervision but weak source supervision.

### Confidence based Learning
- one model teaches the other model using the positive pseudo-labels or teach itself using the negative pseudo-labels.

#### Bidirectional Matching with positive pseudo-labels
- when one network assigns the class probability of input above a certain threshold $\tau$ , we assume that this predicted label as a pseudo-label.
- Then we train the peer network to make its predictions match these positive pseudo-labels via a standard cross-entropy loss.

#### Self-penalization with negative pseudo-labels.
- the negative pseudo-label indicates the most confident label (top-1 label) predicted by the network with a confidence lower than the threshold $\tau$
- Since the negative pseudo-label is unlikely to be a correct label, we need to increase the probability values of all other classes except for this negative pseudo-label. Therefore, we optimize the output probability corresponding to the negative pseudo-label to be close to zero.
- $\tau$ changes adaptively by the sample mean and standard deviation of a mini-batch, not a fixed one.

#### Consistency Regularization
- assume that the well-trained models should be regularized to have consistent results in the same space.
-