# Use POT to Quantize VGG19 Public Model

###### tags: `POT`

## Use OpenVINO dockerhub image

docker run -it -v ~/Downloads:/mnt --rm openvino/ubuntu20_data_dev:latest

## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,

#### 1. Download VGG19
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/downloader.py -o /home/openvino/openvino_models --name vgg19

#### 2. Convert VGG19 to IR
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/converter.py -d /home/openvino/openvino_models -o /home/openvino/openvino_models --name vgg19

#### 3. Run Accuracy Checker on VGG19
accuracy_check -c vgg19-int8.yml -m openvino_models/public/vgg19/FP16/

#### 4. Run POT on VGG19
rm imagenet1000.pickle

pot -c vgg19-int8.json

#### 5. Copy vgg19 FP16-INT8 IR

mkdir /home/openvino/openvino_models/public/vgg19/FP16-INT8/
cp -ar results/vgg19_DefaultQuantization/2021-04-19_15-04-55/optimized/* /home/openvino/openvino_models/public/vgg19/FP16-INT8/

## Reference

drwxrwxr-x 2 openvino openvino 12288 Mar 30 13:08 ILSVRC2012_img_val
-rw-r--r-- 1 openvino openvino 65725 Mar 30 14:01 imagenet1001.pickle
-rw-r--r-- 1 openvino openvino 57035 Mar 30 13:59 imagenet1001.pickle.acc
-rwxr-xr-x 1 openvino openvino   602 Mar 30 13:55 resnet-50-tf-int8.json
-rw-r--r-- 1 openvino openvino   762 Mar 30 13:54 resnet-50-tf-int8.yml
drwxr-xr-x 3 openvino openvino  4096 Mar 30 13:59 results
lrwxrwxrwx 1 openvino openvino    20 Mar 30 13:26 openvino_models
openvino_models/public/vgg19/FP16/vgg19.mapping
openvino_models/public/vgg19/FP16/vgg19.xml
openvino_models/public/vgg19/FP16/vgg19.bin
openvino_models/public/vgg19/FP32/vgg19.mapping
openvino_models/public/vgg19/FP32/vgg19.xml
openvino_models/public/vgg19/FP32/vgg19.bin
openvino_models/public/vgg19/FP16-INT8/vgg19.mapping
openvino_models/public/vgg19/FP16-INT8/vgg19.xml
openvino_models/public/vgg19/FP16-INT8/vgg19.bin

Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host and /mnt in the container.

### vgg19-int8.yml

models:
  - name: vgg19
    launchers:
      - framework: dlsdk
        device: CPU
        adapter: classification

    datasets:
      - name: imagenet_1000_classes
        annotation_conversion:
          converter: imagenet
          annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
        annotation: imagenet1000.pickle
        data_source: /home/openvino/ILSVRC2012_img_val
        metrics:
          - name: accuracy@top1
            type: accuracy
            top_k: 1
          - name: accuracy@top5
            type: accuracy
            top_k: 5
        preprocessing:
          - type: resize
            size: 256
            aspect_ratio_scale: greater
          - type: crop
            size: 224

### vgg19-int8.json

{
    "model": {
        "model_name": "vgg19",
        "model": "/home/openvino/openvino_models/public/vgg19/FP16/vgg19.xml",
        "weights": "/home/openvino/openvino_models/public/vgg19/FP16/vgg19.bin"
    },
    "engine": {
        "config": "/home/openvino/vgg19-int8.yml"
    },
    "compression": {
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "params": {
                    "preset": "performance",
                    "stat_subset_size": 100
                }
            }
        ]
    }
}

### accuracy_checker log

accuracy_check -c vgg19-int8.yml -m openvino_models/public/vgg19/FP16/
Processing info:
model: vgg19
launcher: dlsdk
device: CPU
dataset: imagenet_1000_classes
OpenCV version: 4.5.2-openvino
Annotation conversion for imagenet_1000_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
Annotation conversion for imagenet_1000_classes dataset has been finished
Converted annotation for imagenet_1000_classes dataset will be saved to imagenet1000.pickle
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
    CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Found model openvino_models/public/vgg19/FP16/vgg19.xml
Found weights openvino_models/public/vgg19/FP16/vgg19.bin
Input info:
        Layer name: data
        precision: FP32
        shape [1, 3, 224, 224]

Output info
        Layer name: prob
        precision: FP32
        shape: [1, 1000]

200 objects processed in 15.434 seconds
accuracy@top1: 72.50%
accuracy@top5: 91.50%

### pot log

pot -c vgg19-int8.json -e
15:04:54 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/graph.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Mapping

15:04:54 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Mapping, Set, Iterable

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:112: DeprecationWarning: invalid escape sequence \
  """

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:182: DeprecationWarning: invalid escape sequence \
  """

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:283: DeprecationWarning: invalid escape sequence \
  """

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/front/tf/graph_utils.py:159: DeprecationWarning: invalid escape sequence \*
  """

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/compress_quantized_weights.py:30: DeprecationWarning: invalid escape sequence \
  """

15:04:54 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/optimization/algorithm.py:41: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+
  warnings.warn(

15:04:55 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload

INFO:app.run:Output log dir: ./results/vgg19_DefaultQuantization/2021-04-19_15-04-55
INFO:app.run:Creating pipeline:
 Algorithm: DefaultQuantization
 Parameters:
        preset                     : performance
        stat_subset_size           : 100
        target_device              : ANY
        model_type                 : None
        dump_intermediate_model    : False
        exec_log_dir               : ./results/vgg19_DefaultQuantization/2021-04-19_15-04-55
 ===========================================================================
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
    CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Annotation conversion for imagenet_1000_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
Annotation conversion for imagenet_1000_classes dataset has been finished
Converted annotation for imagenet_1000_classes dataset will be saved to imagenet1000.pickle
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
15:05:10 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/back/ie_ir_ver_2/emitter.py:243: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  if len(element.attrib) == 0 and len(element.getchildren()) == 0:

INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 200
200 objects processed in 11.720 seconds
INFO:compression.engines.ac_engine:Inference finished
INFO:app.run:accuracy@top1              : 0.725
INFO:app.run:accuracy@top5              : 0.91