# DAY 1 EXERCISES

## Exercise 2: MLPs

| Dataset | Network architecture | Test accuracy | Submitter |
| ------- | -------------------- | -------- | --------- |
| MNIST | Default MLP | 95.61% | Markus |
| MNIST | Default MLP with 40 units | 96.92% | Markus |
| Fashion MNIST | Default MLP | 85.64% | Markus |
| MNIST | My awesome model | 0.01% | Mats |
| MNIST | My custom model | 98.20% | Prince |
| MNIST | Experimental MLP with 2 hidden layers of 200,50 units each and drop out rate=0.1. | 98.01% | Sreeram|
| MNIST | 3 hidden layers,300,200,100 units and sigmoid function for the last layer,RMSprop as optimizer. | 98.15% | Neha|
| MNIST | two_layer_mlp_model | 97.13% | Margherita
|MNIST| my_two_layer_model|97,19%| Marija
|MNIST| 2 hidden layers;1st - 100 units 0.2 dropout rate ; 2nd 100 units 0.4 dropout rate|97.71%|Narcis
| MNIST | 64 batch size, 20 iterations| 97.4% | James
| MNIST | 3 hidden layers(400,100,50), 0.2 dropout| 97.80% | Prakshal
| MNIST | 2 hidden layers(50,50), 0.2 dropout, batch size -32| 97.25% | Trang
| MNIST | 2 hidden layers(50,40), one dropout 0.2, batch size -32| 97.17% | Tomas
| MNIST | 2 hidden layers(50,40), nodropout, batch size -32| 97.42% | Purbaj

If interested, there is a Pytorch version of the notebook at `optional/pytorch-mnist-mlp.ipynb`

My 40-unit MLP:

x = layers.Flatten()(inputs)
x = layers.Dense(units=40, activation="relu")(x)
outputs = layers.Dense(units=10, activation='softmax')(x)

---
## Exercise 3: CNNs

| Dataset | Network architecture | Test accuracy | Submitter |
| ------- | ------------------------------------------------------------------------------------------------------ | ------------- | ---------- |
| MNIST | Default CNN | 98.49% | Markus |
| MNIST | Default CNN, 20 epochs | 98.64% | Markus |
| MNIST | My modeltest | 98.84% | JoseManuel |
| MNIST | Adamax optimizer | 99.24% | JoseManuel |
| MNIST | my_SGD_model | 98.82% | Marija |
| MNIST | CNN with 2 conv 2d with 32,32 units & dense layer units=128, drop rate=0.5. Optimizer=adam | 99.02% | Neha |
| MNIST | CNN with 2 conv 2d with 32,32 units & dense layer units=64, drop rate=0.1, epochs=10. Optimizer=Adamax | 99.19% | Sreeram |
| MNIST | adam, epoch=5,batch size=128 | 98.95% | Prince |
| MNIST | adamax, epoch=5,batch size=128 | 99.19% | Prince |
| MNIST | adadelta,epoch=5,batch size=128 | 99.19% | Prince |
| MNIST | nadam, epoch=5,batch size=128 | 99.17% | Prince |
| MNIST | rmsprop,epoch=5,batch size=128 | 99.03% | Prince |
| MNIST | ftrl,epoch=5,batch size=128 | 11.35% | Prince |
| MNIST | adagrad,epoch=5,batch size=12 | 11.35% | Prince |
| MNIST | My_Model 10 epocs | 99.12% | Purbaj |
| MNIST | CNN with 2 conv 2d with 32,21 units & dense layer units = 128, drop rate = 0.5, optimizer=Adam | 98.57% | Trang |
| MNIST | Default CNN | 99.1% | James |
| MNIST | Adadelta | 99.02% | Javed |
| MNIST | RMSprop | 99.19% | Prakshal |
| MNIST | my_cnn_model | 98.37% | Margherita |
| MNIST | Adamax Opt and Batch size=64 | 99.26% | JoseManuel |
| MNIST | Default CNN, 10 epochs | 99.14% | Bruno | |
| MNIST | Default CNN, 5 epochs Adam Op | 99.24% | Ramkumar |

If you feel that Notebooks is too slow for your CNN experiments, you can also try Colab. With CNNs, a GPU makes a big difference: https://colab.research.google.com/github/csc-training/intro-to-dl/blob/master/day1/03-tf2-mnist-cnn.ipynb

Summary of different models and their accuracies on the MNIST dataset: https://paperswithcode.com/sota/image-classification-on-mnist

---
## Exercise 4: RNNs

| Dataset | Network architecture | Test accuracy | Submitter |
| ------- | ------------------------------------- | ------------- | --------- |
| IMDB | Default 1-layer LSTM | 82.96% | Mats |
| IMDB | Default 1-layer CNN (optional folder) | 88.94% | Mats |
| IMDB | Two layers | 82.13% | Jose Manuel |
|IMDB| 2Layer | 83.09%|Purbaj|
|IMDB| LSTM 2 layers with 32 units each, epoc=5, activation=relu, dropout=0.1 | 83.40% |Sreeram|
|IMDB| LSTM 2 layers with 32 units each, epoc=5, activation=sigmoid, dropout =0.2 | 83.68% |Neha Ram|
|IMDB| LSTM x 2 ADAMAX | 84.53%||
IMDB| LSTM x 2 ADAMAX | 84.40%|Ramkumar |
| IMDB | custom_model |82.15|Marija|
|IMDB| LSTM 2 layers with 32 units each, epoc=5, activation=relu, dropout=0.1 x 2 | 83.58|Trang