# NCHC國網機器學習工讀生 研究報告

[TOC]

## 壹. 訓練程序與結果 Training Results

### (1) 使用自己筆電training
最基礎入門的是使用cifar-10提供的image classifier訓練並完成分10類的task，
使用cifar-10練習的code網路上很容易查詢，
但因為不是使用**自己的資料集**，本篇報告將著重在**如何使用自己的資料集訓練出想要的影像分類模型**

本次任務使用的資料集為：ImageNet提供的影像。

:::warning
第一階段目標: 訓練一個**分三類**的影像模型
* 三類為: **Husky, Hen, Penguin**
* 電腦中資料的路徑為:
`D:/NCHC/ImageNet/`下有三個資料夾
分別為:`/Husky`，`/Hen`，`/Penguin`
:::

參照`keras-聖經`書中教材：首先撰寫一個最簡單的CNN 影像分類模型
程式碼:

import numpy as np
import matplotlib.pyplot as plt
import os  # handle data path.
import cv2

# DataPATH = "D:/NCHC/PetImages"
DataPATH = "D:/NCHC/ImageNet"
# CATEGORIES = ["Dog","Cat"]
CATEGORIES = ["Husky","Hen", "Penguin"]

for category in CATEGORIES:
    path = os.path.join(DataPATH, category)
    for img in os.listdir(path):
        img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
        # plt.imshow(img_array, cmap="gray")
        # plt.show()
        break
    break

# print(img_array.shape)

# Resize (all uniform size.)
IMG_SIZE = 50
new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
plt.imshow(new_array, cmap='gray')
plt.show()


training_data = []
# Need to map things(answer: index) to a physical meaning.(a class)

def create_training_data():
   for category in CATEGORIES:
    path = os.path.join(DataPATH, category)
    class_num = CATEGORIES.index(category) # 0: Husky ; 1: Hen ; 2: Penguin
    for img in os.listdir(path):
        try:
            img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
            new_array = cv2.resize(img_array, (IMG_SIZE,IMG_SIZE))
            training_data.append([new_array, class_num])
        except Exception as e: # if some images are broken.
            pass

create_training_data()

print(len(training_data))

# Make sure the training data are balance. (例如 )
import random
random.shuffle(training_data)
for sample in training_data[:10]:
    print(sample[1])

X = []
y = []

for features,label in training_data:
    X.append(features)
    y.append(label)

# print(X[0].reshape(-1, IMG_SIZE, IMG_SIZE, 1))

X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1)

import pickle
import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.
from tensorflow.keras.callbacks import TensorBoard
from keras.utils import np_utils
import time

# y_label_train_OneHot = np_utils.to_categorical(y_label_train)
# y_label_test_OneHot = np_utils.to_categorical(y_label_test)


X = X/255.0
y = np_utils.to_categorical(y)
#  y要換成one Hot

########### Constructing Model ##############
model = Sequential()
model.add(Conv2D(filters=32,kernel_size=(3,3),
                 input_shape= X.shape[1:], 
                 activation='relu', 
                 padding='same'))
model.add(Dropout(rate=0.25))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=64, kernel_size=(3, 3), 
                 activation='relu', padding='same'))
model.add(Dropout(0.25))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dropout(rate=0.25))
model.add(Dense(1024, activation='relu'))
model.add(Dropout(rate=0.25))
model.add(Dense(3, activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'],
              )

####　Start Training ###
model.fit(X, y,
          batch_size=32,
          epochs=10,
          validation_split=0.3)
#  callbacks=[tensorboard]

# print("Model structure : ")
print(model.summary())

在沒有做影像擴增等調整前，可想而知會遇到**過擬合**的問題，如下:
Training Accuracy : 0.95
Validation Accuracy: 0.775

### (2) 優化模型，提高驗證準確率

接著我們將原始資料的相片進行擴增等預處理，將訓練資料量變多，以降低過擬合的問題。
如此一來驗證正確率便提高到約有0.9 (90%)

* 主要改變：使用`datagen`和`model.fit_generator`兩大API
* Accuracy:

* Loss:

* 程式碼如下：

# import the needed libraries
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, MaxPool2D, Flatten, Activation
from tensorflow.keras import backend as K
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# config
img_width, img_height = 28,28 #width & height of input image
input_depth = 1 #1: gray image
train_data_dir = 'D:/NCHC/ImageNet' #data training path
testing_data_dir = 'dataset/mnist/testing' #data testing path
epochs = 2 #number of training epoch
batch_size = 5 #training batch size

# define image generator for Keras,
# here, we map pixel intensity to 0-1
train_datagen = ImageDataGenerator(rescale=1/255)
test_datagen = ImageDataGenerator(rescale=1/255)

# read image batch by batch
train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    color_mode='grayscale',#inpput iameg: gray
    target_size=(img_width,img_height),#input image size
    batch_size=batch_size,#batch size
    class_mode='categorical')#categorical: one-hot encoding format class label
testing_generator = test_datagen.flow_from_directory(
    testing_data_dir,
    color_mode='grayscale',
    target_size=(img_width,img_height),
    batch_size=batch_size,
    class_mode='categorical')
# DataPATH = "D:/NCHC/PetImages"
DataPATH = "D:/NCHC/ImageNet"
# CATEGORIES = ["Dog","Cat"]
CATEGORIES = ["Husky","Hen", "Penguin"]

for category in CATEGORIES:
    path = os.path.join(DataPATH, category)
    for img in os.listdir(path):
        img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
        # plt.imshow(img_array, cmap="gray")
        # plt.show()
        break
    break


# print(img_array.shape)

# Resize (all uniform size.)
IMG_SIZE = 50
new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
plt.imshow(new_array, cmap='gray')
plt.show()


training_data = []
# Need to map things(answer: index) to a physical meaning.(a class)

def create_training_data():
   for category in CATEGORIES:
    path = os.path.join(DataPATH, category)
    class_num = CATEGORIES.index(category) # 0: Husky ; 1: Hen ; 2: Penguin
    for img in os.listdir(path):
        try:
            img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
            # new_array = cv2.resize()
            training_data.append([new_array, class_num])
        except Exception as e: # if some images are broken.
            pass

create_training_data()

print(len(training_data))

# Make sure the training data are balance. (例如 )
import random

random.shuffle(training_data)
for sample in training_data[:10]:
    print(sample[1])

X = []
y = []

#  y要換成one Hot

for features,label in training_data:
    X.append(features)
    y.append(label)

# print(X[0].reshape(-1, IMG_SIZE, IMG_SIZE, 1))

X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1)

import pickle



########### Constructing Model ##############
import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D


from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.
from tensorflow.keras.callbacks import TensorBoard
from keras.utils import np_utils

import time

NAME = "Cats-vs-dogs-64x2-CNN"

# y_label_train_OneHot = np_utils.to_categorical(y_label_train)
# y_label_test_OneHot = np_utils.to_categorical(y_label_test)

y = np_utils.to_categorical(y)
X = X/255.0

model = Sequential()
model.add(Conv2D(filters=32,kernel_size=(3,3),
                 input_shape= X.shape[1:], 
                 activation='relu', 
                 padding='same'))
model.add(Dropout(rate=0.25))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=64, kernel_size=(3, 3), 
                 activation='relu', padding='same'))
model.add(Dropout(0.25))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dropout(rate=0.25))
model.add(Dense(1024, activation='relu'))
model.add(Dropout(rate=0.25))
model.add(Dense(3, activation='softmax'))

tensorboard = TensorBoard(log_dir="logs/{}".format(NAME))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'],
              )

model.fit(X, y,
          batch_size=32,
          epochs=10,
          validation_split=0.3)
#  callbacks=[tensorboard]


# print("Model structure : ")
print(model.summary())

### (3) 基於VGG16的影像分類模型

構想如上圖：拿人家練好的厲害的Model
將最後一個輸出層**客製化**成符合我們需求的模型

第一個方法個人電腦較負擔的起：**特徵萃取法**

**想法：先把之前的features萃取出來**
存入numpy 陣列，接著將input data經由這些feature後
最後輸入convolutional base
此方法速度快成本也低。
唯一缺點：**不允許資料擴增法**

我在電腦的`D:/NCHC/ImageNet/test/test_imgs`
放了七張沒有**非訓練照片**
一張一張餵給Model做分類
以下為分類結果:

:::info
* 輸出格式:
1. 這張圖片實際檔名 (Your input images: )
2. 模型分類後的結果 (Output prediction: )
:::
可以看到：正確率是100% ! (7/7)

當然因為這個task(分七類)還是相對簡單
更複雜的task可能花更多心思來設計


在Jupyter Notebook 上測試的結果

Found 8711 images belonging to 7 classes.
Found 10 images belonging to 1 classes.
1/1 [==============================] - 1s 869ms/step

Classes order: {'Coffee': 0, 'Geysir': 1, 'Hen': 2, 'Husky': 3, 'Leopard': 4, 'Papaya': 5, 'Penguin': 6}

Your input images: test_imgs\coffee0.jpg

Output Prediction : Coffee

Your input images: test_imgs\coffee1.jpg

Output Prediction : Coffee

Your input images: test_imgs\geysir0.jpg

Output Prediction : Geysir

Your input images: test_imgs\hen0.jpg

Output Prediction : Hen

Your input images: test_imgs\husky0.jpg

Output Prediction : Husky

Your input images: test_imgs\husky1.jpg

Output Prediction : Husky

Your input images: test_imgs\leopard.jpg

Output Prediction : Leopard

Your input images: test_imgs\leopard1.jpg

Output Prediction : Leopard

Your input images: test_imgs\papaya0.jpg

Output Prediction : Papaya

Your input images: test_imgs\penguin0.jpg

Output Prediction : Penguin

* Accuracy & Loss figure



* 程式碼如下:

import numpy as np
import matplotlib.pyplot as plt
import os  # handle data path.
import cv2
import pickle

import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D
# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.
from tensorflow.keras.callbacks import TensorBoard
from keras.utils import np_utils
import time
from keras.applications import VGG16

conv_base = VGG16( weights = 'imagenet',
                   include_top = False,
                   input_shape = (150,150, 3))
conv_base.summary()

#　Method 1 : use conv_base & own Dense Layer (output layer)

base_dir = 'D:/NCHC/ImageNet'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')
test_dir = os.path.join(base_dir, 'test')

datagen = ImageDataGenerator(rescale = 1./255)
batch_size = 1

def extract_features(directory, sample_count):
    features = np.zeros(shape=(sample_count, 4, 4, 512))
    labels = np.zeros(shape=(sample_count,7) )
    print("labels shape",labels.shape)
    generator = datagen.flow_from_directory( directory,
                                             target_size=(150,150),
                                             batch_size = batch_size,
                                             class_mode = 'categorical')
    i = 0
    for inputs_batch, labels_batch in generator:
        
        features_batch = conv_base.predict(inputs_batch)
#         print("features batch shape",features_batch.shape)
#         print("label batch ",labels_batch.shape)
        features[i * batch_size :(i + 1) * batch_size] = features_batch
#         print("features shape",features.shape)
        labels[ i * batch_size : (i + 1) * batch_size] = labels_batch
        
        i += 1
        print(i, end = '')
        if i * batch_size >= sample_count:
            break
    return features, labels

train_features, train_labels = extract_features(train_dir, 8711)
validation_features, validation_labels = extract_features(validation_dir, 128)
test_features, test_labels = extract_features(test_dir, 3)
train_features = np.reshape(train_features, (8711, 4 * 4 * 512))
validation_features = np.reshape(validation_features, (128, 4 * 4 * 512))
test_features = np.reshape(test_features, (3, 4 * 4 * 512))


##### Constructing our custom 'Dense Layer', and train! ####

from keras import models
from keras import layers
from keras import optimizers

model = models.Sequential()
model.add(layers.Dense(256, activation='relu', input_dim=4*4*512))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(7, activation='sigmoid'))

model.compile( optimizer = optimizers.RMSprop(lr = 2e-5),
               loss = 'categorical_crossentropy',
               metrics = ['acc'])
history = model.fit(train_features, train_labels,
                    epochs = 30,
                    batch_size = 20,
                    validation_data = (validation_features, validation_labels))

#############　 Plot accuracy & loss  #############

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()


model.save_weights("CNN04_params.h5")
from keras.models import load_model 
model.save("CNN04_model.h5")

#############　Load existed model and Predict (test) the accuracy of your model. #############

IMG_SIZE = 150
train_datagen = ImageDataGenerator(
    rescale = 1./255,
    rotation_range = 40,
    width_shift_range = 0.2,
    height_shift_range=0.2,
    shear_range = 0.2,
    zoom_range = 0.2,
    horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_dir = "D:/NCHC/ImageNet/train"
validation_dir ="D:/NCHC/ImageNet/validation"
train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size = (IMG_SIZE,IMG_SIZE),
    batch_size = 32,
    class_mode = 'categorical'
)


test_dir = 'D:/NCHC/ImageNet/test'
test_datagen = datagen

test_generator = test_datagen.flow_from_directory(
    test_dir, 
    target_size=(IMG_SIZE, IMG_SIZE),
    batch_size=10,
    class_mode= 'categorical', 
    shuffle=False)
test_generator.reset()

custom_dense = load_model("CNN04_model.h5")

model_pred = Sequential()
model_pred.add(conv_base)
model_pred.add(layers.Flatten())
model_pred.add(custom_dense)


pred = model_pred.predict_generator(test_generator, verbose=1)

predicted_class_indices = np.argmax(pred, axis=1)
labels = (train_generator.class_indices)
label = dict((v,k) for k,v in labels.items())

# 建立代码标签与真实标签的关系
predictions = [label[i] for i in predicted_class_indices]

print("\n\nClasses order: ", train_generator.class_indices)

#建立预测结果和文件名之间的关系
filenames = test_generator.filenames
for idx in range(len(filenames )):
#     print('\nOutput probability distributions: ', pred[idx])
    print('\nYour input images: %s\n' % filenames[idx])
    print('Output Prediction :  %s' % (predictions[idx]))
#     print('Actual  :  %s' % filenames[idx])
    print('')
    
    
print('\n\n')

### (4) 延續(3)但使用直接擴增模型方式訓練(需較多計算資源)

此為使用**微調**部分
引入整個VGG模型，加入最後一層Dense Layer
再整個拿去訓練。
此訓練結果請見下方：使用Azure訓練

## 貳. 使用Azure 幫助訓練你的神經網路

### 1. Steps

#### (1) Log in your Azure account:


登入後，點選左上角選擇欄位->訂用帳戶
發現它寫：您沒有任何訂用帳戶
因此要先取得訂用帳戶

本範例是採用 **Azure 開放授權**的方式,取得金鑰，進而取得計算點數

:::danger
這邊依照各單位教學，如何取得**授權**，涉及機密，在此不說明
:::

#### (2) 取得授權後

可看到右上角跳出通知：餘額點數尚有多少
接著可以回到首頁

#### (3) 新建機器學習服務

首頁:

點選 `新建-> AI & 機器學習服務`
會出現以下頁面:

點選標題下的 `+新建`

訂用帳戶：選擇你的帳戶
資源群組：若無，可新建一個（取名字）
工作區名稱：此機器學習workspace名字
其他欄位任意填寫即可

完成資料大概如下:

之後按下檢閱+建立：
檢驗成功後會出現以下畫面
按下建立：
成功後會將你導入你的工作區主頁面
以此範例，我的工作區取名為`VM1`

#### (4) 完成建立 Workspace，遵照導引教學
初次使用，azure會有完整的導覽，基本上按照他的導引，就能學會如何使用機器學習workspace~

### 2. How to import your machine learning code.
如果你已經有jupyter notebook,可以從
`機器學習工作區/筆記本`這個分頁下，
點選第三個欄位 **上傳**
可以上傳檔案或資料夾，非常方便
**你的Dataset**也可以透過此方法上傳，缺點是資料很大的時候（上萬張照片）上傳時間就會比較久

上傳成功後，上方一樣會出線綠色通知條

### 3. How to execute your notebook

將機器學習程式碼與datasets都上傳到工作區後，我們要**執行**此程式碼，
你會發現它寫：

找不到任何計算
這表示：你要[**新增計算單元**](https://docs.microsoft.com/zh-tw/azure/machine-learning/concept-compute-instance)
點選最右邊的三個小點：點選`新增計算`


會出現新增**計算執行個體**

這個步驟攸關你之後run notebook時的執行速度快慢
若你需要處理較多圖形運算，選擇GPU當然比CPU快許多
不過要注意選擇GPU後，你的程式碼可能要稍做修改，否則會有資源不夠等問題（我就郁到了...，短時間內沒那麼容易修好）

新建完計算執行個體後，可以在`計算`頁面發現你剛剛新增的計算單元：

回到筆記本
選擇在 Jupyter Lab中編輯:

進入 Jupyter Lab 後：可以點選上面的**執行**

開始執行後的畫面如下：

正在訓練的樣子：

本次訓練結果：
使用 VGG16-base 加上自己的output layer 疊成之神經網路:
validation accuracy : 97.5 %
(只訓練5 epochs)




## 重要！！如何停止工作區扣款

必須**刪除計算執行個體**才能停止計費

[取得授權](https://www.microsoft.com/Licensing/servicecenter/LicensingInfo/RelationshipSummary/ChildDetailsKeys.aspx?edlicnfo=MIIDCwYJKoZIhvcNAQcDoIIC%2FDCCAvgCAQAxggFIMIIBRAIBADAsMBUxEzARBgNVBAMTCk1TSVQgQ0EgWjICE20Ajr7hvppSbNEz004AAQCOvuEwDQYJKoZIhvcNAQEBBQAEggEAX6pIhSNGvWtyjccTQvCQ1PocXXZiV2hLiHsHodnavj1uVD6jHggjjsc4V9%2FPy4upi%2BZ4wJ0LW1vrqLjo7op%2FDcronEq5FXP%2B1aMSXSCmw2%2FjZGxdIAr9UIBXmC4sBBJiBPTptToCgm%2FzEmeVi%2Fv0EuucHdQbsWWgXdF3lym2HBGLh%2BW%2FSVJZ5ZVDdITdCjuUzwTeUB83Wbze7DxgbcddNC%2FEgMi%2BLsIIrboUQRQ381g8pOrQ6Ey96%2B2UMYBZLO1W5vrm4YH6rZGduLkraCxaZE%2BfweytAdTF8TufMZLxz83YhbArpusPhIEKh9Ezt9xvfqhjyl0I%2FEOqEJig6F8CcjCCAaUGCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIKWQ591HIkgKAggGAuTWxDF6OUvmQ7qwdkoIVQ1dpwFefmqAHAdpEUbrlpAM8u8RXnzREi4hgBUXFIzbZVMlxzX0bDED4267zmM7EqmRzxcZ0Pz6wYTvXSH8Kf7gE5Fn5CEHxgx6LmpV6ysYMOWTr7Bjr4jw26%2BSjiN4%2BF46qsZllku9o6uHk7hwkxFcxi8jSV1fJsVMfoiaZaXsIKg0a80eJqHcyNzJPCrFp%2Fd5snLG6OyrHH2mMNEX%2BWvRo6zgohjPbM9J0tW9oZtFKhGDry8aJExRvxsl9rM6MjSwR%2FED1PrN97XefrY6ygmtPMkGBr9ELKlXZpwGqj5JWdtzKOxlPv%2FOKnzrtQR9YZ82g3B3lKncNCTWUnsJ4jCQOfcf12U5fUGwjarRg6EWan5QOXKRPU1yfauVsJfPHt9ry8TgedBRfML8WK23HaLQit%2FU41%2Fy41x5xR2xkN9h6hF6%2BNj4a%2F3oD7g7Kf4JP8QqoIORH%2FoNdLJ1FSvXlCg43yG%2Fe5Ph%2FmwxV0Nn%2BgLC7)

[Notebook說明](https://azure.microsoft.com/zh-tw/pricing/details/machine-learning/)