DAY 1 EXERCISES

Exercise 2: MLPs

Dataset	Network architecture	Test accuracy	Submitter
MNIST	Default MLP	95.61%	Markus
MNIST	Default MLP with 40 units	96.92%	Markus
Fashion MNIST	Default MLP	85.64%	Markus
MNIST	My awesome model	0.01%	Mats
MNIST	My custom model	98.20%	Prince
MNIST	Experimental MLP with 2 hidden layers of 200,50 units each and drop out rate=0.1.	98.01%	Sreeram
MNIST	3 hidden layers,300,200,100 units and sigmoid function for the last layer,RMSprop as optimizer.	98.15%	Neha
MNIST	two_layer_mlp_model	97.13%	Margherita
MNIST	my_two_layer_model	97,19%	Marija
MNIST	2 hidden layers;1st - 100 units 0.2 dropout rate ; 2nd 100 units 0.4 dropout rate	97.71%	Narcis
MNIST	64 batch size, 20 iterations	97.4%	James
MNIST	3 hidden layers(400,100,50), 0.2 dropout	97.80%	Prakshal
MNIST	2 hidden layers(50,50), 0.2 dropout, batch size -32	97.25%	Trang
MNIST	2 hidden layers(50,40), one dropout 0.2, batch size -32	97.17%	Tomas
MNIST	2 hidden layers(50,40), nodropout, batch size -32	97.42%	Purbaj

If interested, there is a Pytorch version of the notebook at optional/pytorch-mnist-mlp.ipynb

My 40-unit MLP:

x = layers.Flatten()(inputs)
x = layers.Dense(units=40, activation="relu")(x)
outputs = layers.Dense(units=10, activation='softmax')(x)

Dataset	Network architecture	Test accuracy	Submitter
MNIST	Default CNN	98.49%	Markus
MNIST	Default CNN, 20 epochs	98.64%	Markus
MNIST	My modeltest	98.84%	JoseManuel
MNIST	Adamax optimizer	99.24%	JoseManuel
MNIST	my_SGD_model	98.82%	Marija
MNIST	CNN with 2 conv 2d with 32,32 units & dense layer units=128, drop rate=0.5. Optimizer=adam	99.02%	Neha
MNIST	CNN with 2 conv 2d with 32,32 units & dense layer units=64, drop rate=0.1, epochs=10. Optimizer=Adamax	99.19%	Sreeram
MNIST	adam, epoch=5,batch size=128	98.95%	Prince
MNIST	adamax, epoch=5,batch size=128	99.19%	Prince
MNIST	adadelta,epoch=5,batch size=128	99.19%	Prince
MNIST	nadam, epoch=5,batch size=128	99.17%	Prince
MNIST	rmsprop,epoch=5,batch size=128	99.03%	Prince
MNIST	ftrl,epoch=5,batch size=128	11.35%	Prince
MNIST	adagrad,epoch=5,batch size=12	11.35%	Prince
MNIST	My_Model 10 epocs	99.12%	Purbaj
MNIST	CNN with 2 conv 2d with 32,21 units & dense layer units = 128, drop rate = 0.5, optimizer=Adam	98.57%	Trang
MNIST	Default CNN	99.1%	James
MNIST	Adadelta	99.02%	Javed
MNIST	RMSprop	99.19%	Prakshal
MNIST	my_cnn_model	98.37%	Margherita
MNIST	Adamax Opt and Batch size=64	99.26%	JoseManuel
MNIST	Default CNN, 10 epochs	99.14%	Bruno
MNIST	Default CNN, 5 epochs Adam Op	99.24%	Ramkumar

If you feel that Notebooks is too slow for your CNN experiments, you can also try Colab. With CNNs, a GPU makes a big difference: https://colab.research.google.com/github/csc-training/intro-to-dl/blob/master/day1/03-tf2-mnist-cnn.ipynb

Summary of different models and their accuracies on the MNIST dataset: https://paperswithcode.com/sota/image-classification-on-mnist

Dataset	Network architecture	Test accuracy	Submitter
IMDB	Default 1-layer LSTM	82.96%	Mats
IMDB	Default 1-layer CNN (optional folder)	88.94%	Mats
IMDB	Two layers	82.13%	Jose Manuel
IMDB	2Layer	83.09%	Purbaj
IMDB	LSTM 2 layers with 32 units each, epoc=5, activation=relu, dropout=0.1	83.40%	Sreeram
IMDB	LSTM 2 layers with 32 units each, epoc=5, activation=sigmoid, dropout =0.2	83.68%	Neha Ram
IMDB	LSTM x 2 ADAMAX	84.53%
IMDB	LSTM x 2 ADAMAX	84.40%	Ramkumar
IMDB	custom_model	82.15	Marija
IMDB	LSTM 2 layers with 32 units each, epoc=5, activation=relu, dropout=0.1 x 2	83.58	Trang