# BDSE 深度學習
## 課程內容下載
- [資料集](https://www.dropbox.com/s/vn1efjse01qvhrn/BDSE_v2023.03.tar?dl=1)
- [dataloader.py](https://www.dropbox.com/s/lisd1525tfgvrbs/dataloader.py?dl=1)
- [densenet.py](https://www.dropbox.com/s/08mkytqyovchl3s/densenet.py?dl=1)
- [Notebook](https://web.goodnotes.com/s/hJcb8vGjbLbYiyYkICDgsN)
## 課程環境設定
本課程將使用GPU運行深度學習環境。
為了避免上課花費過多時間在環境安裝與設定，請同學預先安裝 `WSL2`, `CUDA`, 和 `cuDNN`。
### 安裝WSL2
1. 開啟 Windows PowerShell，於Powershell終端機內執行
```bash=
wsl --install
```
安裝過程畫面如下:
安裝完畢後請重新開機。
2. 重新開機後，需要設定Linux帳號與密碼:
設定你自己的帳號密碼以完成WSL安裝。
3. 驗證`WSL2`的安裝
開啟 PowerShell，於Powershell終端機內確認WSL版本
```bash=
PS C:\Users\student> wsl -l -v
NAME STATE VERSION
* Ubuntu Stopped 2
```
你應該要看到 WSL 版本 為 `VERSION 2`。若你看到版本為 `VERSION 1`，可嘗試參考[微軟的說明](https://learn.microsoft.com/zh-tw/windows/wsl/install)，將WSL更新為`VERSION 2`。
### 安裝CUDA
進入WSL終端機(從Windows開始功能表搜尋`wsl`然後執行)。進入WSL終端機後，執行以下指令:
```bash=
CUDA_VER="11.8.0-1"
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt update && \
sudo apt -y install cuda=${CUDA_VER}
```
## 安裝cuDNN
安裝完CUDA後，透過以下指令安裝cuDNN:
```bash=
OS="ubuntu2204"
CUDNN_VER="8.8.1.3-1+cuda11.8"
sudo apt update && \
sudo apt install zlib1g
wget https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/cuda-${OS}.pin
sudo mv cuda-${OS}.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/ /"
sudo apt-get update && \
sudo apt-get install libcudnn8=${CUDNN_VER}
```
---
When for
將 `C:\Program Files\NVIDIA Corporation\Nsight Systems 2021.5.2\host-windows-x64\zlib.dll`
更名並複製到
`C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\zlibwapi.dll`
```bash=
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-get update
apt-get -y install cuda=11.8.0-1
OS=ubuntu2204
wget https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/cuda-${OS}.pin
sudo mv cuda-${OS}.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/${OS}/x86_64/ /"
sudo apt-get update
sudo apt-get install libcudnn8=8.8.1.3-1+cuda11.8
# sudo apt-get install libcudnn8-dev=8.8.1.3-1+cuda11.8
```
solution
1. host windows -> install Python, CUDA + poetry directly on Windows
2. WSL2 -> CUDA container + poetry
3. WSL2 -> CUDA + cuDNN + poetry
which one? -> depend on how easy WSL2 can be installed and configured.
also it's probably better for the students as ML, DL working env shall be linux
# 環境設定
1. 開啟 Windows PowerShell，於Powershell終端機內部執行
```bash=
wsl --install
```
you'll need to reboot after wsl installation.
```bash=
PS C:\Users\student> wsl -l -v
NAME STATE VERSION
* Ubuntu Stopped 2
```
go to: https://www.docker.com/products/docker-desktop/
docker run -it -d --name cuda --gpus all -v /mnt/c/Users/Student:/workspace -p 2022:22 -p 8889:8888 0fc
docker pull nvidia/cuda:11.8.0-devel-ubuntu22.04
apt update && apt install python3.10 curl vim
curl -sSL https://install.python-poetry.org | python3 -
---
When for
將 `C:\Program Files\NVIDIA Corporation\Nsight Systems 2021.5.2\host-windows-x64\zlib.dll`
更名並複製到
`C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\zlibwapi.dll`
## Install WSL2
1. execute `vim ~/.bashrc`
2. add `export HOME=/mnt/c/Users/student/Desktop` to `.bashrc`.
3. install Docker
```bash=
sudo apt update && \
sudo apt install -y docker.io
```
4. fetch the TensorFlow docker image:
```bash=
docker pull tensorflow/tensorflow:2.10.1-gpu-jupyter
```
:::info
TensorFlow Docker images can be found at:
* [[Dockerhub]](https://hub.docker.com/r/tensorflow/tensorflow/tags)
* [[NGC]](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tensorflow/tags)
:::
## Docker
1. Install WSL2
2. Install Docker Desktop
3. Pull the TensorFlow image
```bash=
docker pull tensorflow/tensorflow:2.10.1-gpu-jupyter
```
## VSCode
* Install extensions: pylance, python.
* Enable `yapf` as formatter
* Enable powerful pylance linting ability
* Allow formatting on save
* Open `settings.json`; add:
```json "editor.codeActionsOnSave": {
"source.organizeImports": true
},
```
## GPU相關套件 - 安裝方式1
Goal: install TensorFlow (TF) or PyTorch (PYT)
* TF support CUDA version= Unknown
* PYT support CUDA version= v11.6
Therefore:
* TF version? or PYT v1.13.1 <- cuDNN v8.4.x <- CUDA v11.6 <- Driver v522.30 (support CUDA version <=11.8)
---
0. 安裝[NVIDIA Driver](https://www.nvidia.com/Download/index.aspx)
(此步驟可能可略過，因為教室的機器應該都有安裝好NVIDIA驅動程式)
2. 設定NVSMI (看NVIDIA DRIVER有沒有裝好)。
1. 將```C:\Program Files\NVIDIA Corporation\NVSMI```加入System ```PATH```。
2. 加入後, 開啟終端機 (如GIT BASH或Windows命令提示字元), 然後輸入```nvidia-smi.exe```, 按Enter, 即可顯示出GPU使用率等資訊。
3. 安裝[CUDA v11.6.1](https://developer.nvidia.com/cuda-11.3.0-download-archive)。[[下載連結]](https://developer.download.nvidia.com/compute/cuda/11.6.1/local_installers/cuda_11.6.1_511.65_windows.exe)
4. 安裝[cuDNN v 8.4.0 (需與CUDA v11.6.1相容)](https://developer.nvidia.com/cudnn)。[[下載連結]](https://www.dropbox.com/s/vg4yp1cf5hvf2p3/cudnn-windows-x86_64-8.4.0.27_cuda11.6-archive.zip?dl=1)
將解壓縮後的cuDNN資料夾內的檔案, 逐一複製到```C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6```。
4. 執行以下指令，確定nvcc有被安裝 (用以間接檢查看CUDA有沒有裝起來)
```nvcc -V```
5. 安裝PyTorch (要選擇支持CUDA v11.3的Python Wheel)
https://pytorch.org/get-started/locally/
安裝好後, 執行python, 然後載入PyTorch並且嘗試看能否印出PyTorch版本:
```python
import torch
print(torch.__version__) # 1.10.0
```
6. 安裝TensorFlow:
1. 執行 ```pip install tensorflow-gpu``` 即可完成安裝。
2. 檢查TensorFlow是否可把張量丟到GPU
```python
In [1]: import tensorflow as tf
In [2]: tf.__version__
Out[2]: '2.7.0'
In [3]: import numpy as np
In [4]: tmp = np.random.normal(0, 1, (3, 3))
In [5]: tmp
Out[5]:
array([[-0.58497332, 0.91560168, 0.35745953],
[ 0.36502097, 0.81025645, 1.17180727],
[ 0.50818861, -1.05783736, -1.82408266]])
In [6]: tmp = tmp.astype(np.float32)
In [7]: tmp.dtype
Out[7]: dtype('float32')
In [8]: tmp = tf.constant(tmp)
2021-12-28 20:23:09.054421: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-12-28 20:23:09.522473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2774 MB memory: -> device: 0, name: NVIDIA GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1
In [9]: tmp.device
Out[9]: '/job:localhost/replica:0/task:0/device:GPU:0'
```
## GPU相關套件 - 安裝方式2 (WSL2 + Docker)
1. [install WSL2](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10)
2. [using Docker](https://docs.docker.com/desktop/windows/wsl/)
3. [fetch TensorFlow on DockerHub](https://hub.docker.com/r/tensorflow/tensorflow/)
## 檢查套件安裝
```python=
import os
import cv2
import numpy as np
import tensorflow as tf
print(f"TensorFlow version= {tf.__version__}")
print(f"NumPy version= {np.__version__}")
print(f"cv2 version= {cv2.__version__}")
```
My output:
```
Python v3.10.7
TensorFlow version= 2.10.1
NumPy version= 1.23.5
cv2 version= 4.6.0
```
## 安裝其他非GPU相關套件
請安裝Anaconda後，加裝如下套件:
* 基本套件
```bash
pip install pandas matplotlib seaborn numpy scipy
```
* 機器學習套件
```bash
pip install sklearn
```
* 其它套件
```bash
pip install pillow graphviz
pip install mlxtend
pip install numexpr
# conda install -y shapely
# pip install imgaug opencv-python==4.4.0.46
pip install shapely
pip install opencv-python imgaug
pip install jupyterlab
```
# References
* Find state-of-art model implementations (if papers are with codes :arrow_right: :100:): https://paperswithcode.com
* Understand CNN basics (:100: you must read this, understand every lines of words): https://cs231n.github.io
* Learn Deep Learning basics (Wow! Learning by doing! :100:): https://zh.d2l.ai
* Machine Learning basic theories (:100: for those who wants to understand every general bits in Machine Learning): https://cs229.stanford.edu/notes2022fall/main_notes.pdf
* Understand Backpropagation: https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf
---
wengchihung@gmail.com
Sign in with Wallet
Connect another wallet