###### tags: `PyTorch`
# PyTorch - CNN 卷積神經網絡 - MNIST手寫數字辨識
在練習MNIST 使用Linear NN 訓練之後，將 model 改為 CNN 做進一步練習。
CNN 基礎了解，可以參考我 [Keras 練習的文章](https://www.pyexercise.com/2019/01/mnist-cnn.html)。
這邊練習的步驟基本上都差不多，只需要修改 model 的部分還有 `input_shape`
1. Import Libraries
2. 資料預處理
3. <font color=#FF0000>建立模型</font>
4. <font color=#FF0000>訓練模型</font>
只有在 3 & 4 的部分做修改!! 如果有閱讀過[前一篇文章](https://www.pyexercise.com/2019/04/pytorch-hello-world-mnist.html)可以略過。
## **1. Import Libraries**
```python=
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
```
## **2. 資料預處理**
這邊使用 `keras.datasets` 直接載入 MNIST data，方便練習
```python=+
from keras.datasets import mnist
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
```
normalization
```python=+
X_train = X_train.astype('float32') / 255
X_test = X_test.astype('float32') / 255
```
利用 sklearn，將 X_train 切出 0.2 比例的 Validation_data
```python=+
features_train, features_test, targets_train, targets_test = train_test_split(X_train, Y_train, test_size = 0.2, random_state = 42)
```
接著將切好的data 通通轉成 torch 的 tensor 形式，
後面丟進 TensorDataset 需要符合 torch 的 tensor 形式。
```python=+
featuresTrain = torch.from_numpy(features_train)
targetsTrain = torch.from_numpy(targets_train).type(torch.LongTensor) # data type is long
featuresTest = torch.from_numpy(features_test)
targetsTest = torch.from_numpy(targets_test).type(torch.LongTensor) # data type is long
```
**`torch.utils.data.TensorDataset(data_tensor, target_tensor)`**，可[參閱 PyTorch 文檔](https://pytorch.org/docs/stable/data.html)，用意就是將input 數據與目標 output 打包。
```python=+
# Pytorch train and test TensorDataset
train = torch.utils.data.TensorDataset(featuresTrain,targetsTrain)
test = torch.utils.data.TensorDataset(featuresTest,targetsTest)
```
超參數設定:
```python=+
# Hyper Parameters
# batch_size, epoch and iteration
LR = 0.01
batch_size = 100
n_iters = 10000
num_epochs = n_iters / (len(features_train) / batch_size)
num_epochs = int(num_epochs)
```
**`torch.utils.data.TensorDatasetDataLoader(dataset, batch_size=1, shuffle=False,...)`**，可[參閱 PyTorch 文檔](https://pytorch.org/docs/stable/data.html)，為數據加載器。組合數據集和採樣器，並在數據集上提供單進程或多進程迭代器。如需要打亂數據則將 `shuffle=True`。
```python=+
# Pytorch DataLoader
train_loader = torch.utils.data.DataLoader(train, batch_size = batch_size, shuffle = True)
test_loader = torch.utils.data.DataLoader(test, batch_size = batch_size, shuffle = True)
```
## **3. 建立模型**
建立 CNN model ，需要注意的是 `input shape` 在經過每一層後的變化，在設計的時候要稍微計算一下，最後在接上fully conec.層時要符合其 `input shape` 。
```python=+
# Create CNN Model
class CNN_Model(nn.Module):
def __init__(self):
super(CNN_Model, self).__init__()
# Convolution 1 , input_shape=(1,28,28)
self.cnn1 = nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5, stride=1, padding=0) #output_shape=(16,24,24)
self.relu1 = nn.ReLU() # activation
# Max pool 1
self.maxpool1 = nn.MaxPool2d(kernel_size=2) #output_shape=(16,12,12)
# Convolution 2
self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, stride=1, padding=0) #output_shape=(32,8,8)
self.relu2 = nn.ReLU() # activation
# Max pool 2
self.maxpool2 = nn.MaxPool2d(kernel_size=2) #output_shape=(32,4,4)
# Fully connected 1 ,#input_shape=(32*4*4)
self.fc1 = nn.Linear(32 * 4 * 4, 10)
def forward(self, x):
# Convolution 1
out = self.cnn1(x)
out = self.relu1(out)
# Max pool 1
out = self.maxpool1(out)
# Convolution 2
out = self.cnn2(out)
out = self.relu2(out)
# Max pool 2
out = self.maxpool2(out)
out = out.view(out.size(0), -1)
# Linear function (readout)
out = self.fc1(out)
return out
```
將模型 print 出，選擇優化器 Adam，loss function : CrossEntropyLoss() (在多分類任務中使用)
```python=+
model = CNN_Model()
print(model)
optimizer = torch.optim.Adam(model.parameters(), lr=LR) # optimize all cnn parameters
loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted
input_shape = (-1,1,28,28)
```
## 4. 訓練模型
這邊我直接將上一篇的結果寫成函式`def fit_model():` 直接調用即可。
在調用前，需先定義 model, loss_func, optimizer, input_shape, num_epochs, train_loader, test_loader 這些參數。
代碼如下:
```python=+
def fit_model(model, loss_func, optimizer, input_shape, num_epochs, train_loader, test_loader):
# Traning the Model
#history-like list for store loss & acc value
training_loss = []
training_accuracy = []
validation_loss = []
validation_accuracy = []
for epoch in range(num_epochs):
#training model & store loss & acc / epoch
correct_train = 0
total_train = 0
for i, (images, labels) in enumerate(train_loader):
# 1.Define variables
train = Variable(images.view(input_shape))
labels = Variable(labels)
# 2.Clear gradients
optimizer.zero_grad()
# 3.Forward propagation
outputs = model(train)
# 4.Calculate softmax and cross entropy loss
train_loss = loss_func(outputs, labels)
# 5.Calculate gradients
train_loss.backward()
# 6.Update parameters
optimizer.step()
# 7.Get predictions from the maximum value
predicted = torch.max(outputs.data, 1)[1]
# 8.Total number of labels
total_train += len(labels)
# 9.Total correct predictions
correct_train += (predicted == labels).float().sum()
#10.store val_acc / epoch
train_accuracy = 100 * correct_train / float(total_train)
training_accuracy.append(train_accuracy)
# 11.store loss / epoch
training_loss.append(train_loss.data)
#evaluate model & store loss & acc / epoch
correct_test = 0
total_test = 0
for images, labels in test_loader:
# 1.Define variables
test = Variable(images.view(input_shape))
# 2.Forward propagation
outputs = model(test)
# 3.Calculate softmax and cross entropy loss
val_loss = loss_func(outputs, labels)
# 4.Get predictions from the maximum value
predicted = torch.max(outputs.data, 1)[1]
# 5.Total number of labels
total_test += len(labels)
# 6.Total correct predictions
correct_test += (predicted == labels).float().sum()
#6.store val_acc / epoch
val_accuracy = 100 * correct_test / float(total_test)
validation_accuracy.append(val_accuracy)
# 11.store val_loss / epoch
validation_loss.append(val_loss.data)
print('Train Epoch: {}/{} Traing_Loss: {} Traing_acc: {:.6f}% Val_Loss: {} Val_accuracy: {:.6f}%'.format(epoch+1, num_epochs, train_loss.data, train_accuracy, val_loss.data, val_accuracy))
return training_loss, training_accuracy, validation_loss, validation_accuracy
```
接著，我們就可以一行代碼訓練 model 了:
```python=+
training_loss, training_accuracy, validation_loss, validation_accuracy = fit_model(model, loss_func, optimizer, input_shape, num_epochs, train_loader, test_loader)
```
最後將訓練結果畫出:
```python=+
# visualization
plt.plot(range(num_epochs), training_loss, 'b-', label='Training_loss')
plt.plot(range(num_epochs), validation_loss, 'g-', label='validation_loss')
plt.title('Training & Validation loss')
plt.xlabel('Number of epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
plt.plot(range(num_epochs), training_accuracy, 'b-', label='Training_accuracy')
plt.plot(range(num_epochs), validation_accuracy, 'g-', label='Validation_accuracy')
plt.title('Training & Validation accuracy')
plt.xlabel('Number of epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()
```
OK，訓練結果比使用 Linear NN 還要好1%，別小看這1%，在kaggle 競賽當中，差了0.0001 就可以是1~200名的差距了呢!!
Sign in with Wallet
Connect another wallet