# 2022 Machine Learning 李宏毅 [課程筆記]
台大電機 - 李宏毅老師 機器學習課程筆記
**[課程網頁](https://speech.ee.ntu.edu.tw/~hylee/ml/2022-spring.php)**
## 課程筆記
- [機器學習基本概念簡介](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rylSvo-Oq)
- [Lecture 1 : Introduction of Deep Learning](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByWJTCQAF)
- [機器學習任務攻略 (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/BkO-5mEuq)
- [Classification (還沒)](https://hackmd.io/ffcq0nWGR2KLZyJRHuLY-g)
- [Logistic Regression (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rJvzpPtKc)
- [Lecture 2 : What to do if my network fails to train](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Sk7laeMzq)
- [Lecture 3 : Image as input](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HJ-BWbQMc)
- [自注意力機制 (Self-attention)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkDX5vNu9)
- [Lecture 5 : Sequence to sequence (數學還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bk8kuPEdq)
- [卷積神經網路 (Convolutional Neural Networks, CNN)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryQlKrxqq)
- [類神經網路訓練不起來怎麼辦 (一)：局部最小值 (local minima) 與鞍點 (saddle point) (數學還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1ZP6bAdc)
- [類神經網路訓練不起來怎麼辦 (二)：批次 (batch) 與動量 (momentum)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByDNCWRd9)
- [類神經網路訓練不起來怎麼辦 (三)：自動調整學習速率 (Learning Rate)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HJFNRZA_9)
- [類神經網路訓練不起來怎麼辦 (四)：損失函數 (Loss) 也可能有影響](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bk9NCZAOq)
- [類神經網路訓練不起來怎麼辦 (五)： 批次標準化 (Batch Normalization) 簡介](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1CMsdKtc)
- [如何有效的使用自督導式模型 - Data-Efficient & Parameter-Efficient Tuning (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/BkwxR_Ftq)
- [Recurrent Neural Network (RNN)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HkTAAOYKc)
- [Transformer](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJFL4ee59)
- [生成式對抗網路 (Generative Adversarial Network, GAN) (一) – 基本概念介紹](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1hRSxe59)
- [生成式對抗網路 (Generative Adversarial Network, GAN) (二) – 理論介紹與 WGAN](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rJn0Hel9c)
- [生成式對抗網路 (Generative Adversarial Network, GAN) (三) – 生成器效能評估與條件式生成](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1n0Hgeq5)
- [生成式對抗網路 (Generative Adversarial Network, GAN) (四) – Cycle GAN](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1TRBxlqq)
- [Pointer Network (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJSASBxcq)
- [Spatial Transformer Layer (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByBfTsW5c)
- [GNN (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HynRwiVc5)
- [自督導式學習 (Self-supervised Learning) (上)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryr0_oN5c)
- [自督導式學習 (Self-supervised Learning) (下)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bkzc4aH99)
- [自編碼器 (Auto-encoder)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1Oh9sNq5)
- [Anomaly Detection](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5)
- [機器學習模型的可解釋性 (Explainable ML)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryjgai4q9)
- [來自人類的惡意攻擊 (Adversarial Attack)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/H1RgCsVc5)
- [概述領域自適應 (Domain Adaptation)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SyThC06q9)
- [概述增強式學習 (Reinforcement Learning, RL) (一) – 增強式學習跟機器學習一樣都是三個步驟 (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SyCYykRqq)
- [概述增強式學習 (Reinforcement Learning, RL) (二) – Policy Gradient 與修課心情 (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/S1y5yJA55)
- [概述增強式學習 (Reinforcement Learning, RL) (三) - Actor-Critic (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkJ9yJ0qq)
- [概述增強式學習 (Reinforcement Learning, RL) (四) - 回饋非常罕見的時候怎麼辦？機器的望梅止渴 (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SkecJ1C99)
- [概述增強式學習 (Reinforcement Learning, RL) (五) - 如何從示範中學習？逆向增強式學習 (Inverse RL) (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkWqkkA59)
- [神經網路壓縮 (Network Compression) (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ry2BZyCqq)
- [機器終身學習 (Life Long Learning, LL) (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkk7f1Rqq)
- [元學習 Meta Learning (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJeQGk0q5)
## 完成清單
| Date | Topic | Preparation | Class Material | Extra Material |
| ---- | ----------------------------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2/18 | Lecture 1 : Introduction of Deep Learning | <input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rylSvo-Oq) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rylSvo-Oq) | <input type="checkbox" checked="checked"> [Week1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByWJTCQAF) <br> <input type="checkbox" checked="checked"> [Pytorch1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByWJTCQAF) <br> <input type="checkbox" checked="checked"> [Pytorch2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByWJTCQAF) <br> <input type="checkbox" checked="checked"> [Colab](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByWJTCQAF) | <input type="checkbox"> Introduction of Deep Learning <br> <input type="checkbox"> Backpropagation <br> <input type="checkbox"> Predicting Pokemon CP <br> <input type="checkbox"> [Pokemon classification](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByAq2PKtc) <br> <input type="checkbox"> [Logistic Regression](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rJvzpPtKc) |
| 2/25 | Lecture 2 : What to do if my network fails to train | <input type="checkbox"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/BkO-5mEuq) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1ZP6bAdc) <br> <input type="checkbox" checked="checked"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByDNCWRd9) <br> <input type="checkbox" checked="checked"> [影片4](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HJFNRZA_9) <br> <input type="checkbox" checked="checked"> [影片5](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bk9NCZAOq) <br> | <input type="checkbox" checked="checked"> [Week2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Sk7laeMzq) | <input type="checkbox"> Gradient Descent (Demo by AOE) <br><input type="checkbox"> Beyond Adam (part 1) <br><input type="checkbox"> Beyond Adam (part 2) <br> |
| 3/04 | Lecture 3 : Image as input | <input type="checkbox" checked="checked"> [影片](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryQlKrxqq) | <input type="checkbox" checked="checked"> [Week 3 - Validation](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HJ-BWbQMc) <br> <input type="checkbox" checked="checked"> [Week 3 - Why Deep](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HJ-BWbQMc) | <input type="checkbox"> [Spatial Transformer Layer](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ByBfTsW5c) |
|3/11|Lecture 4 : Sequence as input|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkDX5vNu9) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkDX5vNu9)|科技部 X Talent 課堂演講|<input type="checkbox" checked="checked"> [RNN (part 1)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HkTAAOYKc) <br><input type="checkbox" checked="checked"> [RNN (part 2)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HkTAAOYKc) <br><input type="checkbox"> [GNN (part 1)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HynRwiVc5) <br><input type="checkbox"> [GNN (part 2)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/HynRwiVc5) <br>|
|3/18|Lecture 5 : Sequence to sequence|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1CMsdKtc) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJFL4ee59) <br> <input type="checkbox" checked="checked"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJFL4ee59) |<input type="checkbox" checked="checked"> [Week5](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bk8kuPEdq) |<input type="checkbox"> NAT model <br><input type="checkbox"> [Pointer network](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJSASBxcq) <br>|
|3/25|Lecture 6 : Generation|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1hRSxe59) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rJn0Hel9c) <br> <input type="checkbox" checked="checked"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1n0Hgeq5) <br> <input type="checkbox" checked="checked"> [影片4](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/r1TRBxlqq) |Privacy for ML (吳沛遠老師授課)|<input type="checkbox"> Theory of GAN (part 1) <br><input type="checkbox"> Theory of GAN (part 2) <br><input type="checkbox"> Theory of GAN (part 3) <br><input type="checkbox"> VAE <br><input type="checkbox"> FLOW-based Model |
|4/01|Recent Advance of Self-supervised learning for NLP|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryr0_oN5c) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryr0_oN5c) <br> <input type="checkbox" checked="checked"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bkzc4aH99) <br> <input type="checkbox" checked="checked"> [影片4](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/Bkzc4aH99)|<input type="checkbox"> [Week7](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/BkwxR_Ftq) |<input type="checkbox"> BERT (part 1) <br><input type="checkbox"> BERT (part 2) <br> <input type="checkbox"> GPT-3|
|4/08|期中考週不上課||||
|4/15|Lecture 7 : Self-supervised learning for Speech and Image||<input type="checkbox"> Speech ||
|4/22|Lecture 8 : Auto-encoder/ Anomaly Detection|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1Oh9sNq5) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1Oh9sNq5) <br> <input type="checkbox" checked="checked"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br> <input type="checkbox" checked="checked"> [影片4](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br><input type="checkbox" checked="checked"> [影片5](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br> <input type="checkbox" checked="checked"> [影片6](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br> <input type="checkbox" checked="checked"> [影片7](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br> <input type="checkbox" checked="checked"> [影片8](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5) <br> <input type="checkbox" checked="checked"> [影片9](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1XtosNc5)||<input type="checkbox"> PCA <br><input type="checkbox"> t-SNE |
|4/29|Lecture 9 : Explainable AI|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryjgai4q9) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ryjgai4q9)|<input type="checkbox"> Week11 ||
|5/06|Lecture 10 : Attack|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/H1RgCsVc5) <br> <input type="checkbox" checked="checked"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/H1RgCsVc5) |<input type="checkbox"> Week12||
|5/13|Lecture 11 : Adaptation|<input type="checkbox" checked="checked"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SyThC06q9) |<input type="checkbox"> Week13||
|5/20|Lecture 12 : Reinforcement Learning|<input type="checkbox"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SyCYykRqq) <br> <input type="checkbox"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/S1y5yJA55) <br> <input type="checkbox"> [影片3](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkJ9yJ0qq) <br> <input type="checkbox"> [影片4](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SkecJ1C99) <br> <input type="checkbox"> [影片5](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkWqkkA59)|<input type="checkbox"> Speech||
|5/27|Lecture 13 : Network Compression|<input type="checkbox"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ry2BZyCqq) <br> <input type="checkbox"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/ry2BZyCqq)||<input type="checkbox"> PPO <br><input type="checkbox"> Q-learning (part 1) <br><input type="checkbox"> Q-learning (part 2) |
|6/03|Lecture 14 : Life-long Learning|<input type="checkbox"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkk7f1Rqq) <br> <input type="checkbox"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rkk7f1Rqq) |期末考週不上課||
|6/10|Lecture 15 : Meta Learning|<input type="checkbox"> [影片1](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJeQGk0q5) <br> <input type="checkbox"> [影片2](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/SJeQGk0q5)|<input type="checkbox"> Meta|超多|
## 作業筆記
- [HW1 COVID-19 Cases Prediction](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/rJlyt4XG9)
- [HW2 Multiclass Classification](https://hackmd.io/RpkF-cnESc--LF3nbNr9fw?view)
- [HW3 Image Classification](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/BJew9-4fq)
- [HW4 Speaker Identification (還沒)](https://hackmd.io/@_DpzRc_ARk-2NkdnL1lU2g/B1xdGAN195)
## 作業清單
| HW | Date | Topic | Complete |
| ---- | ---- | -------------- | --- |
| HW1 | 2/18 | Regression | <input type="checkbox"> |
| HW2 | 2/25 | Classification | <input type="checkbox"> |
| HW3 | 3/04 | CNN | <input type="checkbox"> |
| HW4 | 3/11 | Self-attention | <input type="checkbox"> |
| HW5 | 3/18 | Transformer | <input type="checkbox"> |
| HW6 | 4/01 | GAN | <input type="checkbox"> |
| HW7 | 4/15 | BERT | <input type="checkbox"> |
| HW8 | 4/22 | Autoencoder | <input type="checkbox"> |
| HW9 | 4/29 | Explainable AI | <input type="checkbox"> |
| HW10 | 5/06 | Attack | <input type="checkbox"> |
| HW11 | 5/13 | Adaptation | <input type="checkbox"> |
| HW12 | 5/20 | RL | <input type="checkbox"> |
| HW13 | 5/27 | Compression | <input type="checkbox"> |
## 參考筆記
- [2018 DRL 課程筆記](https://hackmd.io/@shaoeChen/Bywb8YLKS/https%3A%2F%2Fhackmd.io%2F%40shaoeChen%2FHkH2hSKuS)
- [2020 ML 課程筆記](https://hackmd.io/@shaoeChen/B1CoXxvmm/https%3A%2F%2Fhackmd.io%2Fs%2FHyKhr5sRz)
- [Github 2020 HW](https://github.com/IPINGCHOU/NTU_MachineLearning)
- [Github 2021 HW](https://github.com/ga642381/ML2021-Spring)
- [Github 2022](https://github.com/Fafa-DL/Lhy_Machine_Learning)
###### tags: `2022 李宏毅_機器學習`
Sign in with Wallet
Connect another wallet