# Pytorch入門
## Pytorch環境搭建 使用SSH連至server
`$ ssh kaowt@140.114.95.170 -p 23`
用你的帳號連線至server
`$ git clone https://github.com/CCLoLab/pytorch_tutorial`
找個好地方把範例程式碼clone下來～
`$ python3 -m venv env`
用virtual environments搭建環境,避免你安裝的環境跟其他人打架.env為自訂路徑.venv的文件看[這裡](https://docs.python.org/3/library/venv.html), 沒有venv要用pip先安裝.
`$ source env/bin/activate`
掛載虛擬環境
要開始來安裝pytorch了！
先檢查你的GPU型號和驅動版本(沒有驅動要先安裝! [安裝nvidia驅動](https://linuxconfig.org/how-to-install-the-nvidia-drivers-on-ubuntu-20-04-focal-fossa-linux))
`$ nvidia-smi`
要注意顯卡型號, driver版本及CUDA版本的相容性, 這是容易遇到的坑...
去[NVIDIA官網](https://docs.nvidia.com/deploy/cuda-compatibility/index.html)查看相容性.
去pytorch官網選擇對應的CUDA版本進行安裝, 這裡我們用pip安裝.
https://pytorch.org/
`$ pip install torch torchvision`
## MNIST數字辨識
(來源:[ml4](https://ml4a.github.io/ml4a/looking_inside_neural_nets/))
```python=
# -*- coding: utf-8 -*-
import torch
import torch.nn as nn
import torch.optim as optim
import torch.utils.data as dset
from torchvision import datasets, transforms
```
匯入pytorch套件.
```python=+
# GPU
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
print('GPU State:', device)
```
檢查GPU可用.
```python=+
# Transform
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,)),]
)
```
pytorch的transform模組將圖片轉換.
`transforms.ToTensor()`將圖片轉為torch的Tensor.
`transforms.Normalize()`將[0,255]的灰階正規化為[0,1]的範圍
```python=+
# Data
trainSet = datasets.MNIST(root='MNIST', download=True, train=True, transform=transform)
testSet = datasets.MNIST(root='MNIST', download=True, train=False, transform=transform)
trainLoader = dset.DataLoader(trainSet, batch_size=64, shuffle=True)
testLoader = dset.DataLoader(testSet, batch_size=64, shuffle=False)
```
直接從pytorch匯入訓練用的資料,豪方便～
```python=+
# Model
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.main = nn.Sequential(
nn.Linear(in_features=784, out_features=128),
nn.ReLU(),
nn.Linear(in_features=128, out_features=64),
nn.ReLU(),
nn.Linear(in_features=64, out_features=10),
nn.LogSoftmax(dim=1)
)
def forward(self, input):
return self.main(input)
net = Net().to(device)
print(net)
```
創建模型.這裡使用全連接層.
```python=+
# Parameters
epochs = 3
lr = 0.002
criterion = nn.NLLLoss()
optimizer = optim.SGD(net.parameters(), lr=0.002, momentum=0.9)
```
epochs: 訓練的迭代次數,完整通過一次訓練集的樣本.
lr: learning rate, 反向傳播的學習率.
criterion: 我們使用的loss funtion.
optimizer: 優化器, 這裡使用pytorch內建的SGD(stochastic gradient descent, 隨機梯度下降)
```python=+
# Train
for epoch in range(epochs):
running_loss = 0.0
for times, data in enumerate(trainLoader):
inputs, labels = data[0].to(device), data[1].to(device)
inputs = inputs.view(inputs.shape[0], -1)
# Zero the parameter gradients
optimizer.zero_grad()
# Foward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# Print statistics
running_loss += loss.item()
if times % 100 == 99 or times+1 == len(trainLoader):
print('[%d/%d, %d/%d] loss: %.3f' % (epoch+1, epochs, times+1, len(trainLoader), running_loss/2000))
print('Training Finished.')
```
訓練過程
```python=+
# Test
correct = 0
total = 0
with torch.no_grad():
for data in testLoader:
inputs, labels = data
inputs, labels = inputs.to(device), labels.to(device)
inputs = inputs.view(inputs.shape[0], -1)
outputs = net(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test images: %d %%' % (100*correct / total))
class_correct = [0 for i in range(10)]
class_total = [0 for i in range(10)]
with torch.no_grad():
for data in testLoader:
inputs, labels = data[0].to(device), data[1].to(device)
inputs = inputs.view(inputs.shape[0], -1)
outputs = net(inputs)
_, predicted = torch.max(outputs, 1)
c = (predicted == labels).squeeze()
for i in range(10):
label = labels[i]
class_correct[label] += c[i].item()
class_total[label] += 1
print(class_correct)
print(class_total)
for i in range(10):
print('Accuracy of %d: %3f' % (i, (class_correct[i]/class_total[i])))
```
測試
## CIFAR-10
[官方教程](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py)
What is [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html)?
```python=
# -*- coding: utf-8 -*-
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
# GPU
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
print('GPU state:', device)
# Cifar-10 data
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
# Data
trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)
trainLoader = torch.utils.data.DataLoader(trainset, batch_size=8, shuffle=True, num_workers=2)
testLoader = torch.utils.data.DataLoader(testset, batch_size=8, shuffle=False, num_workers=2)
# Data classes
classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
# Model structure
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(3, 6, 5)
self.conv2 = nn.Conv2d(6, 16, 5)
self.pool = nn.MaxPool2d(2, 2)
self.fc1 = nn.Linear(16*5*5, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16*5*5) #-1 表示自動計算有多少neuron
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
net = Net().to(device)
print(net)
# Parameters
criterion = nn.CrossEntropyLoss()
lr = 0.001
epochs = 10
optimizer = optim.SGD(net.parameters(), lr=lr, momentum=0.9)
# Train
for epoch in range(epochs):
running_loss = 0.0
for times, data in enumerate(trainLoader, 0):
inputs, labels = data
inputs, labels = inputs.to(device), labels.to(device)
# Zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# print statistics
running_loss += loss.item()
if times % 100 == 99 or times+1 == len(trainLoader):
print('[%d/%d, %d/%d] loss: %.3f' % (epoch+1, epochs, times+1, len(trainLoader), running_loss/2000))
print('Finished Training')
# Test
correct = 0
total = 0
with torch.no_grad():
for data in testLoader:
inputs, labels = data
inputs, labels = inputs.to(device), labels.to(device)
outputs = net(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test inputs: %d %%' % (100 * correct / total))
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
for data in testLoader:
inputs, labels = data
inputs, labels = inputs.to(device), labels.to(device)
outputs = net(inputs)
_, predicted = torch.max(outputs, 1)
c = (predicted == labels).squeeze()
for i in range(8):
label = labels[i]
class_correct[label] += c[i].item()
class_total[label] += 1
for i in range(10):
print('Accuracy of %5s : %2d %%' % (classes[i], 100 * class_correct[i] / class_total[i]))
```
## 小練習
1. 我要成為調參俠！
- 更改網路架構和參數, 嘗試讓準確度提高.
3. 你有看過這隻貓嗎? 長的超級可愛，如果你沒看過，現在讓你看看~
- 用訓練好的模型辨認這可愛的貓貓和美妙的數字.
- 用`torch.save(model.state_dict(), PATH)`和`model.load_state_dict(torch.load(PATH))`保存和寫入模型. [官方文件](https://pytorch.org/tutorials/beginner/saving_loading_models.html)
- 用PIL套件處理影像[Importing Image Data into NumPy Arrays](https://www.pluralsight.com/guides/importing-image-data-into-numpy-arrays)
## Reference
[[PyTorch 教學] Getting Start: 訓練分類器 —— CIFAR-10](https://clay-atlas.com/blog/2019/10/20/pytorch-chinese-tutorial-classifier-cifar-10/)
[[PyTorch 教學] Getting Start: 訓練分類器 —— MNIST](https://clay-atlas.com/blog/2019/10/19/pytorch-%E6%95%99%E5%AD%B8-getting-start-%E8%A8%93%E7%B7%B4%E5%88%86%E9%A1%9E%E5%99%A8-mnist/)
[[PyTorch 教學] Getting Start: Neural Networks 神經網路的基本介紹](https://clay-atlas.com/blog/2019/10/16/pytorch-%E6%95%99%E5%AD%B8-getting-start-neural-networks-%E7%A5%9E%E7%B6%93%E7%B6%B2%E8%B7%AF%E7%9A%84%E5%9F%BA%E6%9C%AC%E4%BB%8B%E7%B4%B9/)
## 推薦課程
[CS231n](http://cs231n.stanford.edu/2019/)
[李宏毅 機器學習](http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML19.html)
loss function & cost function
https://medium.com/@gatorsquare/ml-gradient-descent-%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95-c664b5874e5c
### 下載圖片:
curl https://i.imgur.com/KVHgwu1.jpg%20=50%x |grep -oP 'data-src=(.*?).jpg'|cut -d '"' -f2 |xargs -i curl {} -O
# Mnist 練習 byK
https://colab.research.google.com/drive/1QHD32BoRA3y6Nq-FckxxLC3dXIZLVwrt?authuser=2#scrollTo=TEv8S8xoPWbL
# 鐵達尼號（來看看自己的生存率） byK
https://colab.research.google.com/drive/1LGQszyDMbZsc5iw_M48OHvBWJZ7bkGZP?usp=sharing
Google Drive
Gist
Clipboard
Sign in with Wallet
