# Use POT to Quantize inception v4 Public Model

###### tags: `POT`

## Use OpenVINO dockerhub image

docker run -it -v ~/Downloads:/mnt --rm openvino/ubuntu20_data_dev:latest

## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,

#### 1. Download googlenet-v4-tf
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/downloader.py --name googlenet-v4-tf -o /home/openvino/openvino_models

#### 2. Convert googlenet-v4-tf to IR
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/converter.py --name googlenet-v4-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models

#### 3. Run Accuracy Checker on resnet-50-tf
accuracy_check -c googlenet-v4-tf-int8.yml -m openvino_models/public/googlenet-v4-tf/FP16/

#### 4. Run POT on googlenet-v4-tf
rm imagenet1001.pickle

pot -c googlenet-v4-tf-int8.json

#### 5. Copy resnet-50-tf FP16-INT8 IR

mkdir /home/openvino/openvino_models/public/googlenet-v4-tf/FP16-INT8/
cp -ar results/googlenet-v4-tf_DefaultQuantization/2021-04-19_09-01-33/optimized/* /home/openvino/openvino_models/public/googlenet-v4-tf/FP16-INT8/

## Reference

drwxr-xr-x 2 openvino openvino 12288 Apr 19 09:00 ILSVRC2012_img_val
-rw-r--r-- 1 openvino openvino   620 Apr 19 09:01 googlenet-v4-tf-int8.json
-rw-r--r-- 1 openvino openvino   739 Apr 19 08:59 googlenet-v4-tf-int8.yml
-rw-r--r-- 1 openvino openvino 65725 Apr 19 09:01 imagenet1001.pickle
drwxr-xr-x 3 openvino openvino  4096 Apr 19 09:01 results
drwxr-xr-x 3 openvino openvino  4096 Apr 19 08:55 openvino_models
openvino_models/public/googlenet-v4-tf/FP16/googlenet-v4-tf.bin
openvino_models/public/googlenet-v4-tf/FP16/googlenet-v4-tf.xml
openvino_models/public/googlenet-v4-tf/FP16/googlenet-v4-tf.mapping
openvino_models/public/googlenet-v4-tf/FP32/googlenet-v4-tf.bin
openvino_models/public/googlenet-v4-tf/FP32/googlenet-v4-tf.xml
openvino_models/public/googlenet-v4-tf/FP32/googlenet-v4-tf.mapping
openvino_models/public/googlenet-v4-tf/FP16-INT8/googlenet-v4-tf.bin
openvino_models/public/googlenet-v4-tf/FP16-INT8/googlenet-v4-tf.xml
openvino_models/public/googlenet-v4-tf/FP16-INT8/googlenet-v4-tf.mapping

Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host and /mnt in the container.

### googlenet-v4-tf-int8.yml

models:
  - name: googlenet-v4-tf
    launchers:
      - framework: dlsdk
        device: CPU
        adapter: classification

    datasets:
      - name: imagenet_1001_classes
        annotation_conversion:
          converter: imagenet
          annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
          has_background: True
        annotation: imagenet1001.pickle
        data_source: /home/openvino/ILSVRC2012_img_val
        metrics:
          - name: accuracy@top1
            type: accuracy
            top_k: 1
          - name: accuracy@top5
            type: accuracy
            top_k: 5
        preprocessing:
          - type: crop
            central_fraction: 0.875
          - type: resize
            size: 299

### googlenet-v4-tf-int8.json

{
    "model": {
        "model_name": "googlenet-v4-tf",
        "model": "/home/openvino/openvino_models/public/googlenet-v4-tf/FP16/googlenet-v4-tf.xml",
        "weights": "/home/openvino/openvino_models/public/googlenet-v4-tf/FP16/googlenet-v4-tf.bin"
    },
    "engine": {
        "config": "/home/openvino/googlenet-v4-tf-int8.yml"
    },
    "compression": {
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "params": {
                    "preset": "performance",
                    "stat_subset_size": 100
                }
            }
        ]
    }
}

### accuracy_checker log

accuracy_check -c googlenet-v4-tf-int8.yml -m openvino_models/public/googlenet-v4-tf/FP32
Processing info:
model: googlenet-v4-tf
launcher: dlsdk
device: CPU
dataset: imagenet_1001_classes
OpenCV version: 4.5.2-openvino
Annotation conversion for imagenet_1001_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
has_background: True
Annotation conversion for imagenet_1001_classes dataset has been finished
Converted annotation for imagenet_1001_classes dataset will be saved to imagenet1001.pickle
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
    CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Found model openvino_models/public/googlenet-v4-tf/FP32/googlenet-v4-tf.xml
Found weights openvino_models/public/googlenet-v4-tf/FP32/googlenet-v4-tf.bin
Input info:
        Layer name: input
        precision: FP32
        shape [1, 3, 299, 299]

Output info
        Layer name: InceptionV4/Logits/Predictions
        precision: FP32
        shape: [1, 1001]

200 objects processed in 9.028 seconds
accuracy@top1: 76.50%
accuracy@top5: 96.00%

### pot log

pot -c googlenet-v4-tf-int8.json -e
09:01:32 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/graph.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Mapping

09:01:32 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
  from collections import Mapping, Set, Iterable

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:112: DeprecationWarning: invalid escape sequence \
  """

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:182: DeprecationWarning: invalid escape sequence \
  """

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:283: DeprecationWarning: invalid escape sequence \
  """

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/front/tf/graph_utils.py:159: DeprecationWarning: invalid escape sequence \*
  """

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/compress_quantized_weights.py:30: DeprecationWarning: invalid escape sequence \
  """

09:01:32 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/optimization/algorithm.py:41: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+
  warnings.warn(

09:01:32 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload

INFO:app.run:Output log dir: ./results/googlenet-v4-tf_DefaultQuantization/2021-04-19_09-01-33
INFO:app.run:Creating pipeline:
 Algorithm: DefaultQuantization
 Parameters:
        preset                     : performance
        stat_subset_size           : 100
        target_device              : ANY
        model_type                 : None
        dump_intermediate_model    : False
        exec_log_dir               : ./results/googlenet-v4-tf_DefaultQuantization/2021-04-19_09-01-33
 ===========================================================================
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
    CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Annotation conversion for imagenet_1001_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
has_background: True
Annotation conversion for imagenet_1001_classes dataset has been finished
Converted annotation for imagenet_1001_classes dataset will be saved to imagenet1001.pickle
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
09:02:03 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/back/ie_ir_ver_2/emitter.py:243: DeprecationWarning: This method will be removed in future versions.  Use 'list(elem)' or iteration over elem instead.
  if len(element.attrib) == 0 and len(element.getchildren()) == 0:

INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 200
200 objects processed in 7.869 seconds
INFO:compression.engines.ac_engine:Inference finished
INFO:app.run:accuracy@top1              : 0.755
INFO:app.run:accuracy@top5              : 0.94