###### tags: `Collegue Life` `Coding-X` `AI & Machine Learning`

# Practice - MNIST with CNN

## Introduction

This is a class practice for CNN practicing,
**learning using Keras to build a CNN model for MNIST**,
with source code provided in PPT for class.

Here's the model's imformation:

Codes below are rewrited by me.
And I alse added more detailed comments and some of my thoughts, wish could help you if you're interested in it. Enjoy!

## Code
[Github link](https://github.com/chwchao/2019-Coding-X/tree/master/MachineLearning/CNN_MNIST)
1. **Open training file**

with open('train.csv', 'r')as file:
    csv_lines = file.readlines()

<br>

2. **Access and store the data**

According to the format of MNIST data,
1st column is label, the answer of the picture in the other hand.
Columns left are the pixel imformation of the picture (Gray scale - 0~255)

So we store pictures and labels into different lists

**Notice:**
Because we're using convolition, we have to reshape the one-dimension data into the origin shape we expected, which is a picture having 28 x 28 pixcels.

import numpy as np
pic = []
label = []
for i in range(1, len(csv_lines)):
    row = csv_lines[i].replace('\n', '').split(',')
    pic.append(np.array(list(map(int, row[1:]))).reshape(28, 28, 1))
    label.append(list(map(int, row[0])))

<br>

3. **One-hot encoding**

Because our labels are only map from 10 numbers (0~9), we can use "One-hot Encoding" to simplify the data.

from keras.utils import to_categorical
label = to_categorical(y, num_classes = 10)

<br>

4. **Transform into numpy.array, and also normalize**

Because we know that the data of the pictures are mapped from 0~255 (gray scale),
we divide them by 255 to normalize, which are re-mapped to 0~1.

pic = np.array(pic)/255.0
label = np.array(label)

print(pic.shape)
print(label.shape)


<br>

5. **Build the model**

We can see it as an empty model at first, and we need to add layers one by one.
>First layer:
>A convolution layer, which is the only one we need to set the input shape.

>Hidden layers:
>Described in the imformation of the model.

>Output layer:
>The last one layer, its number of neurals must be same as the options we want the model come out. Take this situation for example, we will only get 0~9 total ten numbers, so the output layer must have 10 neurals.
>Also, we use "Softmax" activation function to make outputs converge.

from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPool2D

# Declare a sequential model
model = Sequential()

# Add a convolution layer (it's the first layer, input shape is needed)
model.add(Conv2D(
    filters=32,
    kernel_size=(3,3),
    input_shape=(28, 28, 1),
    activation='relu',
    padding='same'
))

# Dropout
model.add(Dropout(0.2))

# Pooling - 2*2
model.add(MaxPool2D(pool_size=(2, 2)))

# Add a convolution layer 
model.add(Conv2D(
    filters=32,
    kernel_size=(3, 3),
    activation='relu',
    padding='same'
))

# Dropout
model.add(Dropout(0.2))

# Pooling - 2*2
model.add(MaxPool2D(pool_size=(2, 2)))

# Add a flatten layer (transform data back to one-dimension for DNN)
model.add(Flatten())

# Dropout
model.add(Dropout(0.25))

# MLP
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(10, activation='softmax'))

# Print out the summary of the model
print(model.summary())

*relu -- Rectified Linear Unit*
*softmax -- Softmax*


<br>

6. **Compile the model**

Set imformations of the model.

model.compile(
    loss='categorical_crossentropy',
    optimizer='adam',
    metrics=['accuracy']
)

<br>

7. **Start training**

>validation -- how much data is splited out used to validate the model's loss. accuracy...
>batch_size -- how much data used in a gradient
>epochs -- how many times all data is used
>
>verbose -- 0 : No log , 1 : Show progress bar , 2 : Log of each epochs

history = model.fit(
    pic,
    label,
    validation_split=0.2,
    epochs=10,
    batch_size=128,
    verbose=1
)


<br>

8. **Evaluation**

Show the accuracy when training and validating on a plot.
>acc -- accuracy when training
>val_acc -- accuracy when validating

import matplotlib.pyplot as plt
def show_train_history(train_history):
    plt.plot(train_history.history['acc'])
    plt.plot(train_history.history['val_acc'])
    plt.xticks([i for i in range(0, len(train_history.history['acc']))])
    plt.title('Train History')
    plt.ylabel('acc')
    plt.xlabel('epoch')
    plt.legend(['train', 'validation'], loc='lower right')
    plt.show()

show_train_history(history)

<br>

9. **Confusion Matrix**

import itertools
from sklearn.metrics import confusion_matrix, classification_report

def plot_confusion_matrix(
    cm, 
    classes, 
    title='Confusion matrix', 
    cmap=plt.cm.Blues
):
    fig = plt.figure()
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, cm[i, j],horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")
    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')
    # fig.savefig(title + '.eps', format='eps', dpi=600, quality=95)
    # fig.savefig(title + '.png', dpi=600, quality=95)
    plt.show()
    plt.close()

# Mnist label
classes = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']

# Predict the values from the training 
Y_pred = model.predict([pic])

# Convert predictions classes to one hot vectors
y_pred = np.argmax(Y_pred, axis = 1)   

# Convert training observations to one hot vectors
Y_true = np.argmax(label, axis = 1)    

# compute the confusion matrix
confusion_mtx = confusion_matrix(Y_true, y_pred)    

# plot the confusion matrix
plot_confusion_matrix(
    confusion_mtx, 
    classes=classes,  
    title='Confusion_matrix_train'
) 


<br>

10. **Submit the result to submit.csv**

# The format of the output, also the title of the form
submit='ImageId,Lable\n'

# Open the testing file
with open('test.csv', 'r')as file:
    csv_lines = file.readlines()

# Count for row
image_id = 1

for i in range(1, len(csv_lines)):

    # delete '\n', and split by ',' into lists
    row = csv_lines[i].replace('\n', '').split(',')

    # do predict (format must be same as fit())
    img = np.array(list(map(int, row))).reshape(28, 28, 1)
    result = model.predict_classes(np.array([img])/255.0)[0]
    submit += str(image_id) + ',' + str(result) + '\n'
    image_id += 1

# Write file
open('submit.csv', 'w').write(submit)