---
# System prepended metadata

title: Machine Learning 2
tags: [大學, ML]

---

{%hackmd theme-dark %}
# Machine Learning 2
###### tags: `ML` `大學`

## Digit Recognizer
* https://www.kaggle.com/c/digit-recognizer/
* 28x28 1-channel input
* 10 categories output
### Model
* Fully connected
    * Linear ReLU Stack x4
* Cross-entropy Loss
* Stochastic Gradient Descend
* Model Diagram
![](https://i.imgur.com/RyOYW7t.png)
* Accuracy/Epoch
![](https://i.imgur.com/WaUm632.png)
* Test Loss/Epoch
![](https://i.imgur.com/2y5iF0C.png)
* Training Loss/Update
![](https://i.imgur.com/eyKPHQj.png)
* 0.9979 Verify Accuracy
* 0.9663 Test Accuracy
* Ran with batch size 64

### Remarks
* Simple network
* Visualized using TensorBoard
* Tried 3x Linear ReLU stack
    * Not shown in graphs
    * Reached accuracy of 72%
* Same network, same parameters, significant different outcomes

## Chinese MNIST
* https://www.kaggle.com/gpreda/chinese-mnist
* 64x64 1-channel input
* 15 categories output
### Model
* ConvNet
    * 3x Conv ReLU MaxPool
* Fully connected
    * 3x Linear ReLU
* Cross-entropy Loss
* Adam
* Model Diagram
![](https://i.imgur.com/L7LHTKL.png)
* Accuracy/Epoch
![](https://i.imgur.com/ELwyB9J.png)
* Test Loss/Epoch
![](https://i.imgur.com/j9WoCrv.png)
* Training Loss/Update
![](https://i.imgur.com/n8zNhrj.png)
* 0.9713 Test Accuracy 
    * 12k Train 3k Test
* Ran with batch size 32

### Remarks
* Visualized using TensorBoard
* Tried SGD
    * Not shown in graphs
    * Accuracy consistently "stuck" at ~40%
* Takes forever to run on CPU
* 3 Epoch Train Time
    * CPU: ?mins
    * GTX1060: ~5mins
    * RTX2080Ti: <1mins