---
# System prepended metadata

title: Research report

---

[toc]
# Research report

### 1. Bootstrap Your Own Latent

##### Network

<p>
<img src='https://raw.githubusercontent.com/khoatranrb/Img4Md/master/071220_1.png'>
</p>

<p>
<img src='https://raw.githubusercontent.com/khoatranrb/Img4Md/master/071220_2.png'>
</p>

* Mục tiêu: Học Image embedding $\mathbf{y}_\theta$ bằng cách tối thiểu hàm loss:

$$
\mathcal{L}_{\theta,\xi}^{BYOL}=\mathcal{L}_{\theta,\xi}+\mathcal{L}_{\theta,\xi}^{'}
$$

trong đó:

$$
\mathcal{L}_{\theta,\xi}=||\overline{q_\theta}(z_\theta)-\overline{z'}_\xi||_2^2
$$

với $\overline{z'}_\xi=\frac{z'_\xi}{||z'_\xi||_2}$, tương tự với $\overline{q_\theta}(z_\theta)$

và $\mathcal{L}_{\theta,\xi}^{'}$ là loss function khi dùng $v$ là đầu vào của target network và $v'$ là đầu vào của online network.

##### Update weights
$$
\begin{align}
\theta&\leftarrow\rm{optimizer}(\theta,\triangledown_\theta\mathcal{L}_{\theta,\xi}^{BYOL})\\
\xi&\leftarrow\tau\xi+(1-\tau)\theta
\end{align}
$$

#####  Intuitions on BYOL’s behavior

### SimSam

##### Network

<p>
<img src='https://raw.githubusercontent.com/khoatranrb/Img4Md/master/071220_3.png'>
</p>

##### Algorithm

<p>
<img src='https://raw.githubusercontent.com/khoatranrb/Img4Md/master/071220_4.png'>
</p>

### Related paper

1. [x] [A Simple Framework for Contrastive Learning of Visual Representations](https://arxiv.org/pdf/2002.05709.pdf)
2. [x] [Unsupervised Learning of Visual Featuresby Contrasting Cluster Assignments](https://arxiv.org/pdf/2006.09882.pdf)

<p>
<img src='https://camo.githubusercontent.com/422d479e294553eceac31e1defa3ae5084d45cbaa035e5f1f8e9e5eec4bbc87e/68747470733a2f2f646c2e666261697075626c696366696c65732e636f6d2f64656570636c75737465722f616e696d617465642e676966'>
</p>

3. [x] [Unsupervised Embedding Learning via Invariant and SpreadingInstance Feature](https://arxiv.org/pdf/1904.03436.pdf)
4. [Unsupervised Feature Learning via Non-Parametric Instance Discrimination](https://arxiv.org/pdf/1805.01978.pdf)
5. [x] [Representation Learning withContrastive Predictive Coding](https://arxiv.org/pdf/1807.03748.pdf)
6. [Self-Supervised Learning of Pretext-Invariant Representations](https://openaccess.thecvf.com/content_CVPR_2020/papers/Misra_Self-Supervised_Learning_of_Pretext-Invariant_Representations_CVPR_2020_paper.pdf)
7. [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://arxiv.org/pdf/1905.09272.pdf)
8. [LEARNING  DEEP  REPRESENTATIONS  BY  MUTUAL  IN-FORMATION ESTIMATION AND  MAXIMIZATION](https://arxiv.org/pdf/1808.06670.pdf)
9. [Momentum Contrast for Unsupervised Visual Representation Learning](https://arxiv.org/pdf/1911.05722.pdf)
10. [Improved Baselines with Momentum Contrastive Learning](https://arxiv.org/pdf/2003.04297.pdf)