---
# System prepended metadata

title: 'EASE : Use POT to Quantize yolo-v4-tf Public Model'
tags: [EASE, POT]

---

# EASE : Use POT to Quantize yolo-v4-tf Public Model

###### tags: `POT` `EASE`

## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt -u root --rm openvino/ubuntu20_dev:latest
```


## Run Accuracy Checker and POT
In ubuntu20_dev docker image,

#### 0. Download COCO 2017 trainval dataset and annotation

```=bash
cd /home/openvino
apt update
apt install unzip

mkdir coco_dataset
cd coco_dataset
curl http://images.cocodataset.org/zips/val2017.zip -o val2017.zip
unzip val2017.zip

curl http://images.cocodataset.org/annotations/annotations_trainval2017.zip -o trainval_2017.zip
unzip trainval_2017.zip
```

##### coco_dataset content

```
coco_dataset/
|--annotations/
  |-- captions_train2017.json
  |-- captions_val2017.json
  |-- instances_train2017.json
  |-- instances_val2017.json
  |-- person_keypoints_train2017.json
  `-- person_keypoints_val2017.json
|-- val2017/
    |-- 000000042102.jpg
    |-- 000000060102.jpg
    |-- 000000245102.jpg
    ...
    `-- 000000364102.jpg
```

#### 1. Download yolo-v4-tf

```=bash
omz_downloader.py --name yolo-v4-tf -o /home/openvino/openvino_models
```

#### 2. Convert yolo-v4-tf to IR

```=bash
omz_converter.py --name yolo-v4-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models
```

#### 3. Run Accuracy Checker on yolo-v4-tf

```=bash
accuracy_check -c yolo-v3-4f-int8.yml -m /home/openvino/openvino_models/public/yolo-v4-tf/FP32
```

#### 4. Run POT on yolo-v3-tf

```=bash
pot -c yolo-v4-tf-int8.json -e
```

#### 5. Copy yolo-v4-tf FP16-INT8 IR

```=bash
mkdir /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/
cp -ar results/yolo-v4-tf_DefaultQuantization/2*/optimized/* /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/

#### Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host  and /mnt in the container.
```

## Reference

### yolo-v4-tf-int8.yml
```
models:
  - name: yolo_v4_tf
    launchers:
      - framework: openvino
        device: CPU
        adapter:
          type: yolo_v3
          anchors: 12,16,19,36,40,28,36,75,76,55,72,146,142,110,192,243,459,401
          num: 3
          coords: 4
          classes: 80
          threshold: 0.001
          anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
          raw_output: True
          output_format: HWB
          outputs:
            - Func/StatefulPartitionedCall/output/_544
            - Func/StatefulPartitionedCall/output/_542
            - Func/StatefulPartitionedCall/output/_543
    datasets:
      - name: ms_coco_detection_80_class_without_background
        annotation_conversion:
          converter: mscoco_detection
          annotation_file:  /home/openvino/coco_dataset/annotations/instances_val2017.json
        data_source: /home/openvino/coco_dataset/val2017
        preprocessing:
          - type: resize
            size: 608
        postprocessing:
          - type: resize_prediction_boxes
          - type: filter
            apply_to: prediction
            min_confidence: 0.001
            remove_filtered: true
          - type: diou_nms
            overlap: 0.5
          - type: clip_boxes
            apply_to: prediction
        metrics:
          - type: map
            integral: 11point
            ignore_difficult: true
            presenter: print_scalar
            reference: 0.7123
          - name: AP@0.5
            type: coco_precision
            max_detections: 100
            threshold: 0.5
            reference: 0.774
          - name: AP@0.5:0.05:95
            type: coco_precision
            max_detections: 100
            threshold: '0.5:0.05:0.95'
            reference: 0.5026
```

### yolo-v4-tf-int8.json
```
{
    "model": {
        "model_name": "yolo-v4-tf",
        "model": "/mnt/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.xml",
        "weights": "/mnt/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.bin"
    },
    "engine": {
        "config": "/mnt/yolo-v4-tf-int8.yml"
    },
    "compression": {
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "params": {
                    "preset": "performance",
                    "stat_subset_size": 300
                }
            }
        ]
    }
}
```

### accuracy_checker log
```
accuracy_check -c /mnt/yolo-v4-tf-int8.yml -m /mnt/openvino_models/public/yolo-v4-tf/FP32 -ss 300
Processing info:
model: yolo_v4_tf
launcher: openvino
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.5
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.485s
200 / 5000 processed in 0.482s
300 / 5000 processed in 0.484s
400 / 5000 processed in 0.483s
500 / 5000 processed in 0.483s
600 / 5000 processed in 0.485s
700 / 5000 processed in 0.488s
800 / 5000 processed in 0.490s
900 / 5000 processed in 0.488s
1000 / 5000 processed in 0.489s
1100 / 5000 processed in 0.495s
1200 / 5000 processed in 0.493s
1300 / 5000 processed in 0.494s
1400 / 5000 processed in 0.495s
1500 / 5000 processed in 0.494s
1600 / 5000 processed in 0.498s
1700 / 5000 processed in 0.501s
1800 / 5000 processed in 0.500s
1900 / 5000 processed in 0.502s
2000 / 5000 processed in 0.506s
2100 / 5000 processed in 0.499s
2200 / 5000 processed in 0.502s
2300 / 5000 processed in 0.501s
2400 / 5000 processed in 0.501s
2500 / 5000 processed in 0.498s
2600 / 5000 processed in 0.500s
2700 / 5000 processed in 0.500s
2800 / 5000 processed in 0.500s
2900 / 5000 processed in 0.499s
3000 / 5000 processed in 0.499s
3100 / 5000 processed in 0.496s
3200 / 5000 processed in 0.499s
3300 / 5000 processed in 0.495s
3400 / 5000 processed in 0.495s
3500 / 5000 processed in 0.494s
3600 / 5000 processed in 0.495s
3700 / 5000 processed in 0.491s
3800 / 5000 processed in 0.490s
3900 / 5000 processed in 0.490s
4000 / 5000 processed in 0.489s
4100 / 5000 processed in 0.486s
4200 / 5000 processed in 0.484s
4300 / 5000 processed in 0.483s
4400 / 5000 processed in 0.534s
4500 / 5000 processed in 0.514s
4600 / 5000 processed in 0.511s
4700 / 5000 processed in 0.491s
4800 / 5000 processed in 0.499s
4900 / 5000 processed in 0.524s
5000 / 5000 processed in 0.510s
5000 objects processed in 24.805 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
IE version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
Loaded CPU plugin version:
    CPU - openvino_intel_cpu_plugin: 2022.1.2022.1.0-7019-cdb9bec7210-releases/2022/1
Found model /mnt/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.xml
Found weights /mnt/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.bin
Input info:
	Node name: image_input
	Tensor names: image_input:0, image_input
	precision: f32
	shape: (1, 608, 608, 3)

Output info
	Node name: Func/StatefulPartitionedCall/output/_542:0
	Tensor names: StatefulPartitionedCall/model/conv2d_101/BiasAdd:0, StatefulPartitionedCall/Identity:0, Func/StatefulPartitionedCall/output/_542:0, Identity:0
	precision: f32
	shape: (1, 38, 38, 255)

	Node name: Func/StatefulPartitionedCall/output/_543:0
	Tensor names: Identity_1:0, StatefulPartitionedCall/model/conv2d_109/BiasAdd:0, Func/StatefulPartitionedCall/output/_543:0, StatefulPartitionedCall/Identity_1:0
	precision: f32
	shape: (1, 19, 19, 255)

	Node name: Func/StatefulPartitionedCall/output/_544:0
	Tensor names: Identity_2:0, StatefulPartitionedCall/Identity_2:0, Func/StatefulPartitionedCall/output/_544:0, StatefulPartitionedCall/model/conv2d_93/BiasAdd:0
	precision: f32
	shape: (1, 76, 76, 255)

  0%|          | 1/300 [00:00<02:53]12:30:45 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/openvino/tools/accuracy_checker/adapters/yolo.py:426: RuntimeWarning: overflow encountered in exp
  prob_correct=lambda x: 1.0 / (1.0 + np.exp(-x)))

                                      300 objects processed in 167.019 seconds
map: 71.78% [FAILED:  abs error = 0.5475 | relative error = 0.007686]
AP@0.5: 78.38% [FAILED:  abs error = 0.9796 | relative error = 0.01266]
AP@0.5:0.05:95: 53.79% [FAILED:  abs error = 3.53 | relative error = 0.07023]
```

### POT log
```
pot -c /mnt/yolo-v4-tf-int8.json -e --output-dir INFO:openvino.tools.pot.app.run:Output log dir: /mnt/openvino_models/public/yolo-v4-tf/FP16-INT8/yolo-v4-tf_DefaultQuantization/2022-05-02_13-42-23
INFO:openvino.tools.pot.app.run:Creating pipeline:
 Algorithm: DefaultQuantization
 Parameters:
	preset                     : performance
	stat_subset_size           : 300
	target_device              : ANY
	model_type                 : None
	dump_intermediate_model    : False
	inplace_statistics         : True
	exec_log_dir               : /mnt/openvino_models/public/yolo-v4-tf/FP16-INT8/yolo-v4-tf_DefaultQuantization/2022-05-02_13-42-23
 ===========================================================================
IE version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
Loaded CPU plugin version:
    CPU - openvino_intel_cpu_plugin: 2022.1.2022.1.0-7019-cdb9bec7210-releases/2022/1
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.543s
200 / 5000 processed in 0.537s
300 / 5000 processed in 0.544s
400 / 5000 processed in 0.554s
500 / 5000 processed in 0.545s
600 / 5000 processed in 0.543s
700 / 5000 processed in 0.556s
800 / 5000 processed in 0.561s
900 / 5000 processed in 0.616s
1000 / 5000 processed in 0.623s
1100 / 5000 processed in 0.633s
1200 / 5000 processed in 0.621s
1300 / 5000 processed in 0.615s
1400 / 5000 processed in 0.617s
1500 / 5000 processed in 0.624s
1600 / 5000 processed in 0.616s
1700 / 5000 processed in 0.620s
1800 / 5000 processed in 0.618s
1900 / 5000 processed in 0.619s
2000 / 5000 processed in 0.587s
2100 / 5000 processed in 0.572s
2200 / 5000 processed in 0.563s
2300 / 5000 processed in 0.560s
2400 / 5000 processed in 0.565s
2500 / 5000 processed in 0.562s
2600 / 5000 processed in 0.559s
2700 / 5000 processed in 0.561s
2800 / 5000 processed in 0.587s
2900 / 5000 processed in 0.602s
3000 / 5000 processed in 0.558s
3100 / 5000 processed in 0.558s
3200 / 5000 processed in 0.557s
3300 / 5000 processed in 0.556s
3400 / 5000 processed in 0.554s
3500 / 5000 processed in 0.553s
3600 / 5000 processed in 0.552s
3700 / 5000 processed in 0.550s
3800 / 5000 processed in 0.550s
3900 / 5000 processed in 0.552s
4000 / 5000 processed in 0.547s
4100 / 5000 processed in 0.546s
4200 / 5000 processed in 0.544s
4300 / 5000 processed in 0.545s
4400 / 5000 processed in 0.542s
4500 / 5000 processed in 0.542s
4600 / 5000 processed in 0.539s
4700 / 5000 processed in 0.543s
4800 / 5000 processed in 0.540s
4900 / 5000 processed in 0.539s
5000 / 5000 processed in 0.535s
5000 objects processed in 28.424 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
INFO:openvino.tools.pot.pipeline.pipeline:Inference Engine version:                2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.pipeline.pipeline:Model Optimizer version:                 2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.pipeline.pipeline:Post-Training Optimization Tool version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:openvino.tools.pot.statistics.collector:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:openvino.tools.pot.pipeline.pipeline:Evaluation of generated model
INFO:openvino.tools.pot.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 5000
1000 / 5000 processed in 173.116s
2000 / 5000 processed in 176.532s
3000 / 5000 processed in 173.620s
4000 / 5000 processed in 175.386s
5000 / 5000 processed in 175.397s
5000 objects processed in 874.052 seconds
INFO:openvino.tools.pot.engines.ac_engine:Inference finished
INFO:openvino.tools.pot.app.run:map                        : 0.7084540024167995
INFO:openvino.tools.pot.app.run:AP@0.5                     : 0.7726537047800925
INFO:openvino.tools.pot.app.run:AP@0.5:0.05:95             : 0.4798230247936459
```