###### tags: `Tensorflow colab`

Tensorflow with colab local connect
===

[TOC]

# Introduction

This document help you to prepare local machine for run google colab. This instruction is specific for Machine Learning with Scikit-Learn and Deep Learning with tensorflow.

# 1. Prepare your local machine

## 1.1 Install miniconda
Install platform specific miniconda from [here](https://docs.conda.io/en/latest/miniconda.html#system-requirements)

## 1.2 Create working folder
Create a working folder at anywhere and create a sub-folder `env`

## 1.3 Create conda environment
1. Start miniconda in terminal
2. Go to folder that was created before
4. Run `conda create -p [current directory either ./env or .\env(windows)]`and follow the rest of insutruction. **-p** is for path

## 1.4 Activate conda environment
1. Start miniconda in terminal (if not)
2. Run `conda env list` to list all available environment
3. Select one you would like to work with
4. Run `conda activate [either environment name or path]`

## 1.5 Install necessary modules
1. Follow **1.4** instruction to activate conda environment
2. Run `conda install python=3.7 numpy pandas matplotlib jupyter pip` and follow the reset of instruction
3. Run `conda install -c conda-forge scikit-learn` to install [scikit-learn](https://scikit-learn.org/stable/install.html)
4. Run `conda install -c rmg wget` to install wget
5. Run `conda install -c conda-forge kaggle` to install kaggle (Only if you use kaggle API)

## 1.6 Install tensorflow
1. Follow **1.4** instruction to activate conda environment
2. Run `pip install tensorflow [tensorflow-hub (optional)]` to install [tensorflow](https://www.tensorflow.org/install)
3. Follow the rest of instruction

## 1.7 Install cuda toolkit (Only GPU support CUDA)
1. Make sure your gpu is supported see [here](https://www.tensorflow.org/install/gpu#hardware_requirements)
2. Install all necessary software [here](https://www.tensorflow.org/install/gpu#software_requirements). Both [cuda toolkit](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn) are necessary.
3. Restart computer

# 2. Prepare local runtime for colab

## 2.1 Start local runtime
1. Follow **1.4** instruction to activate conda environment
2. Go to working directory (if not)
3. Run `pip install jupyter_http_over_ws`
4. Run `jupyter serverextension enable --py jupyter_http_over_ws`
5. Run `jupyter notebook --NotebookApp.allow_origin='https://colab.research.google.com' --port=8888 [--NotebookApp.port_retries=0 or --no-browser]`
6. Wait for it to finish
7. Copy token which look similar to this **http://localhost:8888/?token=d622eedd2fece15cb7887773424de1d0c7981f0d8d8e0431**

# 3. Connect local runtime from colab

## 3.1 Connect to local runtime
1. Open your colab notebook
2. Click **Connect to local runtime** from up right dropdown menu with an arrow
3. Past token and click **Connect**

# 4. Easy way to prepare your local machine (Optional)

## Requirement
1. Make sure you installed miniconda
2. Download conda env files (I prepared) from [here](https://github.com/NPMachineLearning/Conda-Env-ML) Click **Code** and download file as zip or you can fork repo and use git.

## 4.1 Windows
1. Unzip files into a directory
2. Open miniconda command line and go to the directory you unzip files and then go to subdirectory **win**
- .yml: YAML file is conda env file
- .bat: The batch file is for starting backend for Colab to connect at local machine and your can open it in any editor to change the batch file commands.
3. Run `conda env create -f=.\tf-env-win.yml -p=.\env`
-f: path to env YAML file
-p: path to env folder, create automatically if folder does not exists and all libraries will be installed in this folder
4. Now you can either follow 2. and 3. or launch batch file **(you need to modify first command)**
## 4.2 MacOSX
1. Unzip files into a directory
2. Start terminal and go to the directory you unzip files and then go to subdirectory **mac**
- .yml: YAML file is conda env file
- .bat: The batch file is for starting backend for Colab to connect at local machine and your can open it in any editor to change the batch file commands.
3. Run `conda env create -f=./tf-env-mac.yml -p=./env`
-f: path to env YAML file
-p: path to env folder, create automatically if folder does not exists and all libraries will be installed in this folder
4. Now you can either follow **2.** and **3.** or launch shell script file

# 5. Missing library runing colab notebook
If any libraries is missing while runing colab notebook then you just need to run `conda install [library name]` and for searching library in conda go [here](https://anaconda.org/) and type in package name to search or try `pip install [library name]`

# Note
You can install libraries through colab notebook for example `!pip install wget` but it will not guaratee to work.

Sometime libraries installed through **pip** will work but sometime they don't. For example to use **tensorflow with GPU enable** you must install with **pip**

If a library don't work with **pip** installation then try search same library in conda and install.