# 7. HUNG-YI LEE 2022 ML - Self-supervised learning
###### tags: `Machine Learning`
Recent Advance of Self-supervised learning for NLP
## 巨大的模型
* BERT series
* hw3 model的一千倍
* GPT series

## self-supervised learning
* hw1~5 - 監督式學習
* cycle GAN - unsupervised learning
其中self-supervised learning為unsupervised learning的子集

無label情況下,自己產生Label自己學習

* BERT = Transformer Encoder
* Masking Input
* https://drive.google.com/file/d/1yeVssXHlwFif_3Fyo2jRyf13kJEmYhTI/view?usp=drivesdk