###### tags: `Tutorial`

# QNAP Container-Station AI Framework Test Step (Tensorflow, Pytorch, Caffe, MXNet, CNTK, Chainer, NVIDIA Digits)

## Tensorflow

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `tensorflow/tensorflow:1.11.0-gpu`

Farmware : **QTS 4.4.X**
* Full tag : `tensorflow/tensorflow:1.11.0-gpu`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"tensorflow"**
4. Change to **"Docker Hub"** tab
5. Click **"Create"** button on the **"tensorflow/tensorflow"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Click the **"URL"** to open jupyter (container overview page, url => http://192.168.0.207:32769/ => 8888)
10. Enter the token.
**PS** : It is shown in console, for example, `http://(e769b96a4c44 or 127.0.0.1):8888/?token=9b655d3334c44a00282bb5e2d5a03f0960965962bd48b4ab`, the token is `9b655d3334c44a00282bb5e2d5a03f0960965962bd48b4ab`
11. Open **"3_mnist_from_scratch.ipynb"**, after tensorflow 1.13, open **tensorflow-tutorials** -> **basic_classification**
12. Click **"Kernel"** -> **"Restart & Rull All"**
13. The cursor will move to end of document. And a **"*"** will appeared on left of each code.
14. All code completed (appear a number) if no error.
15. Stop notebook and remove container to save resources.
16. Normally completed the whole training job in a few minutes (depend on the GPU card)

**PASS** :

## Pytorch

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `pytorch/pytorch:0.4.1-cuda9-cudnn7-devel`

Farmware : **QTS 4.4.X**
* Full tag : `pytorch/pytorch:1.0.1-cuda10.0-cudnn7-devel`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"pytorch"**
4. Change to **"Docker Hub"** tab
5. Click **"Create"** button on the **"pytorch/pytorch"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Open console
10. Run Command :

    cd ~
    apt update
    apt install git vim -y
    git clone https://github.com/pytorch/examples.git
    cd examples/mnist
    pip install torchvision
    python main.py

16. Pass if no error.
17. Stop notebook and remove container to save resources.

**PASS (注意，要檢查 GPU 是否啟用，pytorch 在沒有抓到 GPU 時會自動改用 CPU 訓練。速度會慢非常多)** :

## Caffe

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `bvlc/caffe:gpu`

Farmware : **QTS 4.4.X**
* Full tag : `bvlc/caffe:gpu`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"caffe"**
4. Change to **"Docker Hub"** tab
5. Click **"Create"** button on the **"bvlc/caffe"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Open console
10. Run command :

    cd /opt/caffe
    sh ./data/mnist/get_mnist.sh
    sh ./examples/mnist/create_mnist.sh
    sh ./examples/mnist/train_lenet.sh

14. Pass if no error.
15. Stop notebook and remove container to save resources.

**PASS** :

## MXNet

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `mxnet/python:1.3.1_gpu_cu90_mkl_py3`

Farmware : **QTS 4.4.X**
* Full tag : `mxnet/python:1.3.1_gpu_cu92_mkl_py3`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"mxnet"**
4. Change to "Docker Hub" tab
5. Click **"Create"** button on the **"mxnet/python"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Open console
10. Run command :

    cd ~
    apt update
    apt install git vim -y
    git clone https://github.com/apache/incubator-mxnet.git
    cd incubator-mxnet/example/image-classification

14. If mxnet version == v1.3.* or v1.2.* RUN `git checkout b3be92f`
15. Run `python train_mnist.py --gpus 0` or `python3 train_mnist.py --gpus 0` if using py3 image
16. Pass if no error.
17. Stop notebook and remove container to save resources.

**PASS** :

## CNTK

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `microsoft/cntk:2.6-gpu-python3.5-cuda9.0-cudnn7.0`

Farmware : **QTS 4.4.X**
* Full tag : `microsoft/cntk:2.6-gpu-python3.5-cuda9.0-cudnn7.0`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"cntk"**
4. Change to **"Docker Hub"** tab
5. Click "Create" button on the **"microsoft/cntk"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Open console
10. Run command :

    cd /cntk/Examples/Image/DataSets/MNIST
    python install_mnist.py
    cd /cntk/Examples/Image/GettingStarted
    cntk configFile=02_OneConv.cntk

14. Pass if no error.
15. Stop notebook and remove container to save resources.

* If console show this :

cntk: error while loading shared libraries: libnccl.so.2: cannot open shared object file: No such file or directory

Please fallow this to install NCCL : https://docs.nvidia.com/deeplearning/sdk/nccl-install-guide/index.html#down

**PASS** :

## Chainer

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `chainer/chainer:v5.0.0b4-python3`

Farmware : **QTS 4.4.X**
* Full tag : `chainer/chainer:v6.0.0b1`

### Test step

1. Open container station
2. Select **"Create"**
3. Enter **"chainer"**
4. Change to **"Docker Hub"** tab
5. Click **"Create"** button on the **"chainer/chainer"** column
6. Select desired version (tag)
7. Wait till container download and created
8. Open the created docker
9. Open console
10. Run command :

    cd ~
    apt update
    apt install git vim -y
    git clone https://github.com/chainer/chainer.git
    cd chainer/examples/mnist

14. Check chainer version
* if chainer version == v5 RUN `git checkout 72d7238`
* if chainer version == v6.0.0b1 RUN `git checkout f7724b6`
15. Run `python train_mnist.py --gpu=0` or `python3 train_mnist.py --gpu=0` if using py3 image
16. Pass if no error. (See Optimization Done. at last line)
17. Stop notebook and remove container to save resources.

**PASS** :

## NVIDIA Digits

### Suggest image tag

Farmware : **QTS 4.3.5**
* Full tag : `nvidia/digits:6.0`

Farmware : **QTS 4.4.X**
* Full tag : `nvidia/digits:6.0`

### Test step

#### NVIDIA DIGITS5:

GPU=nvidia0 gpu-docker run -d -h digits5_demo --name digits5_demo -p 15000:5000 nvidia/digits:6.0
docker exec -it digits5_demo bash

#### NVIDIA DIGITS6:

GPU=nvidia0 gpu-docker run -d -h digits6_demo --name digits6_demo -p 15000:5000 nvidia/digits:6.0
docker exec -it digits6_demo bash

After docker exec, run below command to download sample data:

mkdir /data
python -m digits.download_data mnist /data/mnist
ln /dev/null /dev/raw1394
exit

1. Open NVIDIA DIGITIS url: http://nasip:15000
2. Datasets -> New Dataset -> Images -> Classification

**Image Type** : Grayscale
**Size** : 28 x 28
**Training Images** : /data/mnist/train
**Dataset name** : mnist

3. Click "**Create**" button
4. Wait until finish (show Done in status)
5. Models -> New Model -> Images -> Classification

**Select dataset** : mnist
**Training epochs** : 30 (*lesser if GPU memory not enough*)
Standard networks -> Caffe -> LeNet
**Model name** : mnist

6. Click "**Create**" button
7. Wait until finish (show Done)

**PASS** :