# IDK Challenge
---

**Made by:** *Danilo Paes da Silva* \
**Email:** *[e-mail]* \
**Phone:** *[phone]*

[*Repository link*](https://bit.ly/3wkctUk)

---

## The Challenge

The IDK Challenge consists of image recognition of a dataset of flowers. The original challenge can be found at [this repository](https://bit.ly/3KZaqKI).

## Introduction

The code was initialy made in Linux Ubuntu version 18.04.5 LTS with VsCode and python venv in Python 3.6 but in order to use some recent functions of TensorFlow an earlier version of Python such as Python 3.9 was needed and because of it Docker seemed to be a good solution.

I set up a docker enviroment using [tensorflow image](https://bit.ly/3M3aUAO) provided by the library and some other added libraries (Matplotlib and numpy) but my CPU is not compatible with earlier versions of TensorFlow as shown below:
> Terminal Print

And the proposed solutions on the internet were:

> [Reference of proposed solutions](https://bit.ly/3vXxJAf)

That means that I would need to use TensorFlow 1.5 (from 2018) instead of earlier verions of TensorFlow wich greatly changed in version 2.0 (syntax, structure, etc.)

Therefore I chose to delevop in Google Colab as Jupyter Notebook and later export the deployable version to the git repository. I kept the repository updated by using Google Colab's feature of saving notebooks as commits to a github repository.

Later I will explain how to build and run the module.

## Model of choice

CNN (Covolution Neural Network) was the chosen model because of its benifits over other neural networks. CNN was inspired in how animals see the world, processing small samples of images and emphasising their crucial information to recognize what is been seen. This process is simulated by a series of hidden layers of filters that analyse strides of info and extract relevant features that are later re-introduced into the model as feedback to improve it. It's currently the most used model of image recognition because it preserves positional information, defines the features on its own and has many optimizations such as pooling and dropout that reduces required processing without losing relevant information.

## Implementation and execution

As I said I'm using Google Colabs enviroment with TensorFlow, Keras and Keras Tuner for hyperparameter optimization. I used 250x250x3 images of 32 batches with Keras using an CNN with 4 Conv2D layers with Max2D pooling, a dropout layer and a flatten layer before the final Dense layer, all with filters and neuron quantites optimized with Keras Tuner RandomSearch. We latter plot some info and graphics with matplotlib to analyze top 5 models.

The easiest way to run the code is in Google Colabs by opening the `main.ipynb` file. Make sure you set up your colab to use GPU since it greatly increases processing speed, go to `Runtime Enviroment -> Change runtime type -> Select GPU`. To import the data we first save our `archieve.zip` in a Google Drive directory described in the variable `DATA_PATH_IN_DRIVE` and it unzips and saves it in colabs local acess on `DATA_DIRECTORY` variable. It automatically checks if the dataset folder is already in the colab and as long as you upload `archieve.zip` to your drive and update its path in `DATA_PATH_IN_DRIVE` you should have no trouble (Obs: you only need to change after "My Drive" in `DATA_PATH_IN_DRIVE`, the rest is colabs directories).

Now to run it, just do `CTRL+F9` or `Runtime Enviroment -> Execute all` to run it all.

As I said I've configured a Dockerfile so I could deploy it in docker but I ended up needing a cloud enviroment. Nonetheless to use it you would need to run `docker build -t name-of-image` and then `docker run name-of-image` or follow the tutorial to optimize it to VsCode [linked bellow](https://www.youtube.com/watch?v=cJbvcH0JNGA). The only thing that is missing is the requirements.txt that could be generated with `pip freeze > requirements.txt` in a python venv or manually write all the needed dependecies to install which I haven't fineshed.

## Conclusion

The model seens to be working fine but is having trouble to find good results, I think there is some problem in data processing since the version of this [commit](https://github.com/DPaess/IDK-Challenge/commit/ac18b2c26fc2fac2779e58ecc768a4ac059b9e10) perfoms better even when use the same parameters in the recent version. The method `tf.keras.utils.image_dataset_from_directory ` returns a hard-to-split dataset which made it hard to extract x_train,y_train,x_val,y_val that were needed for Keras Tuner, this part definitelly needs some tweaking.

## References

- Theorical
- https://bit.ly/3sqzIuZ
- https://bit.ly/3Ftzmcl
- https://bit.ly/3ysEmwo
- https://bit.ly/396nMHI

- Pratical
- https://bit.ly/3skQNWT
- https://bit.ly/3KSNAEE
- https://bit.ly/3FDchUv
- https://bit.ly/3yn7R2w
- Hyperparemeter Tuning:
- https://bit.ly/3l2Ixaa
- https://bit.ly/38gKCg3

- Python enviroment in VsCode
- https://bit.ly/3Nkp0OL
- https://bit.ly/3snTK98
- https://bit.ly/3MXbS1u
- https://bit.ly/3wcXoUJ

- Setup Docker
- https://bit.ly/37uYdjd
- https://bit.ly/3N0po4H
- https://dockr.ly/3M87QDO

#### This document was made in [HackMD](https://bit.ly/3M2ZqgT)

###### tags: `IDK Digital`