---
# System prepended metadata

title: Use POT to Quantize yolo-v3-tiny-tf Public Model
tags: [POT]

---

# Use POT to Quantize yolo-v3-tiny-tf Public Model on ov 2023.1

###### tags: `POT`

## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt -u root --rm openvino/ubuntu22_dev:latest
```


## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,

#### 0. Download COCO 2017 trainval dataset and annotation

cd /home/openvino
apt update
apt install unzip

mkdir coco_dataset
cd coco_dataset
curl http://images.cocodataset.org/zips/val2017.zip -o val2017.zip
unzip val2017.zip

curl http://images.cocodataset.org/annotations/annotations_trainval2017.zip -o trainval_2017.zip
unzip trainval_2017.zip

##### coco_dataset content

```
coco_dataset/
|--annotations/
  |-- captions_train2017.json
  |-- captions_val2017.json
  |-- instances_train2017.json
  |-- instances_val2017.json
  |-- person_keypoints_train2017.json
  `-- person_keypoints_val2017.json
|-- val2017/
    |-- 000000042102.jpg
    |-- 000000060102.jpg
    |-- 000000245102.jpg
    ...
    `-- 000000364102.jpg
```

#### 1. Download yolo-v3-tiny-tf
omz_downloader --name yolo-v3-tiny-tf -o /home/openvino/openvino_models

#### 2. Convert yolo-v3-tiny-tf to IR
omz_converter --name yolo-v3-tiny-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models

#### 3. Run Accuracy Checker on yolo-v3-tiny-tf

cd /home/openvino
git clone https://github.com/openvinotoolkit/open_model_zoo.git

accuracy_check -c yolo-v3-tf-tiny-int8.yml -m /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16 -d /home/openvino/open_model_zoo/data/dataset_definitions.yml -ss 300

#### 4. Run POT on yolo-v3-tiny-tf

pot -c yolo-v3-tf-tiny-int8.json -e

#### 5. Copy yolo-v3-tiny-tf FP16-INT8 IR

mkdir /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16-INT8/
cp -ar results/yolo-v3-tiny-tf_DefaultQuantization/2021-04-13_04-01-17/optimized/* /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16-INT8/

## Referece

```
drwxr-xr-x 4 openvino openvino 4096 Apr 13 03:27 coco_dataset
drwxr-xr-x 3 root     root     4096 Apr 13 03:36 openvino_models

drwxr-xr-x 3 root     root     4096 Apr 13 04:01 results
./results/yolo-v3-tf_DefaultQuantization/2021-04-13_04-01-17/log.txt
./results/yolo-v3-tf_DefaultQuantization/2021-04-13_04-01-17/optimized
./results/yolo-v3-tf_DefaultQuantization/2021-04-13_04-01-17/optimized/yolo-v3-tf.mapping
./results/yolo-v3-tf_DefaultQuantization/2021-04-13_04-01-17/optimized/yolo-v3-tf.bin
./results/yolo-v3-tf_DefaultQuantization/2021-04-13_04-01-17/optimized/yolo-v3-tf.xml

-rw-r--r-- 1 root     root      591 Apr 13 03:54 yolo-v3-tf-int8.json
-rw-r--r-- 1 root     root     1386 Apr 13 03:53 yolo-v3-tf-int8.yml


openvino_models/
`-- public
    `-- yolo-v3-tf
        |-- FP16
        |   |-- yolo-v3-tf.bin
        |   |-- yolo-v3-tf.mapping
        |   `-- yolo-v3-tf.xml
        |-- FP16-INT8
        |   |-- yolo-v3-tf.bin
        |   |-- yolo-v3-tf.mapping
        |   `-- yolo-v3-tf.xml
        |-- FP32
        |   |-- yolo-v3-tf.bin
        |   |-- yolo-v3-tf.mapping
        |   `-- yolo-v3-tf.xml
        |-- yolo-v3.json
        `-- yolo-v3.pb
```

Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host  and /mnt in the container.

### yolo-v3-tiny-tf-int8.yml
```
models:
  - name:  yolo_v3_tiny_tf
    launchers:
      - framework: dlsdk
        device: CPU
        adapter:
          type: yolo_v3
          anchors: tiny_yolo_v3
          num: 3
          coords: 4
          classes: 80
          anchor_masks: [[3, 4, 5], [1, 2, 3]]
          outputs:
            - conv2d_9/Conv2D/YoloRegion
            - conv2d_12/Conv2D/YoloRegion
    datasets:
      - name: ms_coco_detection_80_class_without_background
        annotation_conversion:
          converter: mscoco_detection
          annotation_file:  /home/openvino/coco_dataset/annotations/instances_val2017.json
        data_source: /home/openvino/coco_dataset/val2017
        preprocessing:
          - type: resize
            size: 416

        postprocessing:
          - type: resize_prediction_boxes
          - type: filter
            apply_to: prediction
            min_confidence: 0.001
            remove_filtered: True
          - type: nms
            overlap: 0.5
          - type: clip_boxes
            apply_to: prediction

        metrics:
          - type: map
            integral: 11point
            ignore_difficult: true
            presenter: print_scalar
          - type: coco_precision
            max_detections: 100
            threshold: 0.5
```

### yolo-v3-tf-tiny-int8.json
```
{
    "model": {
        "model_name": "yolo-v3-tiny-tf",
        "model": "/home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16/yolo-v3-tiny-tf.xml",
        "weights": "/home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16/yolo-v3-tiny-tf.bin"
    },
    "engine": {
        "config": "/home/openvino/yolo-v3-tf-tiny-int8.yml"
    },
    "compression": {
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "params": {
                    "preset": "performance",
                    "stat_subset_size": 300
                }
            }
        ]
    }
}
```

### accuracy_checker log
```
root@f16837777495:/home/openvino# accuracy_check -c yolo-v3-tf-tiny-int8.yml -m /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16 -d /home/openvino/open_model_zoo/data/dataset_definitions.yml -ss 300
/usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/preprocessor/launcher_preprocessing/ie_preprocessor.py:21: FutureWarning: OpenVINO Inference Engine Python API is deprecated and will be removed in 2024.0 release.For instructions on transitioning to the new API, please refer to https://docs.openvino.ai/latest/openvino_2_0_transition_guide.html
  from openvino.inference_engine import ResizeAlgorithm, PreProcessInfo, ColorFormat, MeanVariant  # pylint: disable=import-outside-toplevel,package-absolute-imports

/usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/launcher/dlsdk_launcher.py:60: FutureWarning: OpenVINO nGraph Python API is deprecated and will be removed in 2024.0 release.For instructions on transitioning to the new API, please refer to https://docs.openvino.ai/latest/openvino_2_0_transition_guide.html
  import ngraph as ng

Processing info:
model: yolo_v3_tiny_tf
launcher: openvino
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.8.0-dev
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.311s
200 / 5000 processed in 0.314s
300 / 5000 processed in 0.308s
400 / 5000 processed in 0.309s
500 / 5000 processed in 0.308s
600 / 5000 processed in 0.306s
700 / 5000 processed in 0.303s
800 / 5000 processed in 0.302s
900 / 5000 processed in 0.304s
1000 / 5000 processed in 0.305s
1100 / 5000 processed in 0.304s
1200 / 5000 processed in 0.313s
1300 / 5000 processed in 0.303s
1400 / 5000 processed in 0.303s
1500 / 5000 processed in 0.307s
1600 / 5000 processed in 0.306s
1700 / 5000 processed in 0.306s
1800 / 5000 processed in 0.304s
1900 / 5000 processed in 0.304s
2000 / 5000 processed in 0.316s
2100 / 5000 processed in 0.309s
2200 / 5000 processed in 0.305s
2300 / 5000 processed in 0.306s
2400 / 5000 processed in 0.303s
2500 / 5000 processed in 0.306s
2600 / 5000 processed in 0.306s
2700 / 5000 processed in 0.306s
2800 / 5000 processed in 0.314s
2900 / 5000 processed in 0.304s
3000 / 5000 processed in 0.336s
3100 / 5000 processed in 0.328s
3200 / 5000 processed in 0.314s
3300 / 5000 processed in 0.323s
3400 / 5000 processed in 0.358s
3500 / 5000 processed in 0.362s
3600 / 5000 processed in 0.325s
3700 / 5000 processed in 0.336s
3800 / 5000 processed in 0.315s
3900 / 5000 processed in 0.298s
4000 / 5000 processed in 0.294s
4100 / 5000 processed in 0.289s
4200 / 5000 processed in 0.305s
4300 / 5000 processed in 0.328s
4400 / 5000 processed in 0.301s
4500 / 5000 processed in 0.295s
4600 / 5000 processed in 0.325s
4700 / 5000 processed in 0.302s
4800 / 5000 processed in 0.301s
4900 / 5000 processed in 0.305s
5000 / 5000 processed in 0.309s
5000 objects processed in 15.547 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
ms_coco_detection_80_class_without_background dataset metadata will be saved to mscoco_det_80.json
Converted annotation for ms_coco_detection_80_class_without_background dataset will be saved to mscoco_det_80.pickle
2023-09-27 08:01:03.592691: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-09-27 08:01:03.593982: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-27 08:01:03.613169: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-27 08:01:03.613328: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-09-27 08:01:03.958504: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
IE version: 2023.1.0-12185-9e6b00e51cd-releases/2023/1
Loaded CPU plugin version:
    CPU - openvino_intel_cpu_plugin: 2023.1.2023.1.0-12185-9e6b00e51cd-releases/2023/1
Found model /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16/yolo-v3-tiny-tf.xml
Found weights /home/openvino/openvino_models/public/yolo-v3-tiny-tf/FP16/yolo-v3-tiny-tf.bin
Input info:
	Node name: image_input
	Tensor names: image_input:0, image_input
	precision: f32
	shape: (1, 416, 416, 3)

Output info
	Node name: conv2d_12/Conv2D/YoloRegion/sink_port_0
	Tensor names: conv2d_12/Conv2D/YoloRegion, conv2d_12/Conv2D/YoloRegion:0
	precision: f32
	shape: (1, 255, 26, 26)

	Node name: conv2d_9/Conv2D/YoloRegion/sink_port_0
	Tensor names: conv2d_9/Conv2D/YoloRegion:0, conv2d_9/Conv2D/YoloRegion
	precision: f32
	shape: (1, 255, 13, 13)

08:01:04 accuracy_checker WARNING: /usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/adapters/yolo.py:414: UserWarning: Number of output layers (2) not equal to detection grid size (3). Must be equal with each other, if output tensor resize is required
  warnings.warn('Number of output layers ({}) not equal to detection grid size ({}). '

300 objects processed in 17.356 seconds                                                                                                                                                                    
map: 42.17%
coco_precision: 47.41%

```

### pot log

```
root@f16837777495:/home/openvino# pot -c yolo-v3-tf-tiny-int8.json -e
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
[ WARNING ] /usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/preprocessor/launcher_preprocessing/ie_preprocessor.py:21: FutureWarning: OpenVINO Inference Engine Python API is deprecated and will be removed in 2024.0 release.For instructions on transitioning to the new API, please refer to https://docs.openvino.ai/latest/openvino_2_0_transition_guide.html
  from openvino.inference_engine import ResizeAlgorithm, PreProcessInfo, ColorFormat, MeanVariant  # pylint: disable=import-outside-toplevel,package-absolute-imports

[ WARNING ] /usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/launcher/dlsdk_launcher.py:60: FutureWarning: OpenVINO nGraph Python API is deprecated and will be removed in 2024.0 release.For instructions on transitioning to the new API, please refer to https://docs.openvino.ai/latest/openvino_2_0_transition_guide.html
  import ngraph as ng

Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
Nevergrad package could not be imported. If you are planning to use any hyperparameter optimization algo, consider installing it using pip. This implies advanced usage of the tool. Note that nevergrad is compatible only with Python 3.7+
Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
INFO:openvino.tools.pot.app.run:Output log dir: ./results/yolo-v3-tiny-tf_DefaultQuantization/2023-09-27_08-02-46
INFO:openvino.tools.pot.app.run:Creating pipeline:
 Algorithm: DefaultQuantization
 Parameters:
	preset                     : performance
	stat_subset_size           : 300
	target_device              : ANY
	model_type                 : None
	dump_intermediate_model    : False
	inplace_statistics         : True
	exec_log_dir               : ./results/yolo-v3-tiny-tf_DefaultQuantization/2023-09-27_08-02-46
 ===========================================================================
IE version: 2023.1.0-12185-9e6b00e51cd-releases/2023/1
Loaded CPU plugin version:
    CPU - openvino_intel_cpu_plugin: 2023.1.2023.1.0-12185-9e6b00e51cd-releases/2023/1
08:02:47 accuracy_checker WARNING: /usr/local/lib/python3.10/dist-packages/openvino/tools/accuracy_checker/adapters/yolo.py:414: UserWarning: Number of output layers (2) not equal to detection grid size (3). Must be equal with each other, if output tensor resize is required
  warnings.warn('Number of output layers ({}) not equal to detection grid size ({}). '

Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.300s
200 / 5000 processed in 0.294s
300 / 5000 processed in 0.298s
400 / 5000 processed in 0.292s
500 / 5000 processed in 0.291s
600 / 5000 processed in 0.289s
700 / 5000 processed in 0.301s
800 / 5000 processed in 0.289s
900 / 5000 processed in 0.288s
1000 / 5000 processed in 0.291s
1100 / 5000 processed in 0.290s
1200 / 5000 processed in 0.290s
1300 / 5000 processed in 0.293s
1400 / 5000 processed in 0.293s
1500 / 5000 processed in 0.291s
1600 / 5000 processed in 0.289s
1700 / 5000 processed in 0.290s
1800 / 5000 processed in 0.290s
1900 / 5000 processed in 0.296s
2000 / 5000 processed in 0.292s
2100 / 5000 processed in 0.309s
2200 / 5000 processed in 0.290s
2300 / 5000 processed in 0.288s
2400 / 5000 processed in 0.294s
2500 / 5000 processed in 0.287s
2600 / 5000 processed in 0.287s
2700 / 5000 processed in 0.291s
2800 / 5000 processed in 0.289s
2900 / 5000 processed in 0.293s
3000 / 5000 processed in 0.291s
3100 / 5000 processed in 0.293s
3200 / 5000 processed in 0.292s
3300 / 5000 processed in 0.299s
3400 / 5000 processed in 0.295s
3500 / 5000 processed in 0.293s
3600 / 5000 processed in 0.325s
3700 / 5000 processed in 0.294s
3800 / 5000 processed in 0.293s
3900 / 5000 processed in 0.291s
4000 / 5000 processed in 0.291s
4100 / 5000 processed in 0.293s
4200 / 5000 processed in 0.290s
4300 / 5000 processed in 0.288s
4400 / 5000 processed in 0.295s
4500 / 5000 processed in 0.291s
4600 / 5000 processed in 0.292s
4700 / 5000 processed in 0.289s
4800 / 5000 processed in 0.297s
4900 / 5000 processed in 0.290s
5000 / 5000 processed in 0.303s
5000 objects processed in 14.658 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
2023-09-27 08:03:02.374152: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-09-27 08:03:02.375406: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-27 08:03:02.394533: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-27 08:03:02.394706: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-09-27 08:03:02.796169: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO:openvino.tools.pot.pipeline.pipeline:Inference Engine version:                2023.1.0-12185-9e6b00e51cd-releases/2023/1
INFO:openvino.tools.pot.pipeline.pipeline:Model Optimizer version:                 2023.1.0-12185-9e6b00e51cd-releases/2023/1
INFO:openvino.tools.pot.pipeline.pipeline:Post-Training Optimization Tool version: 2023.1.0-12185-9e6b00e51cd-releases/2023/1
INFO:openvino.tools.pot.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:openvino.tools.pot.statistics.collector:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
08:03:41 accuracy_checker WARNING: /usr/local/lib/python3.10/dist-packages/openvino/tools/pot/algorithms/quantization/utils.py:318: FutureWarning: Unlike other reduction functions (e.g. `skew`, `kurtosis`), the default behavior of `mode` typically preserves the axis it acts along. In SciPy 1.11.0, this behavior will change: the default value of `keepdims` will become False, the `axis` over which the statistic is taken will be eliminated, and the value None will no longer be accepted. Set `keepdims` to True or False to avoid this warning.
  input_shape = mode(activations_statistics[input_node_name]['shape'])[0][0]

INFO:openvino.tools.pot.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:openvino.tools.pot.pipeline.pipeline:Evaluation of generated model
INFO:openvino.tools.pot.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 5000
1000 / 5000 processed in 47.237s
2000 / 5000 processed in 48.310s
3000 / 5000 processed in 47.503s
4000 / 5000 processed in 47.693s
5000 / 5000 processed in 47.919s
5000 objects processed in 238.662 seconds
INFO:openvino.tools.pot.engines.ac_engine:Inference finished
INFO:openvino.tools.pot.app.run:map                        : 0.35655414620366077
INFO:openvino.tools.pot.app.run:coco_precision             : 0.39341015285894115
```