# EASE : Use POT to Quantize yolo-v4-tf Public Model
###### tags: `POT` `EASE`
## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt -u root --rm openvino/ubuntu20_dev:latest
```
## Run Accuracy Checker and POT
In ubuntu20_dev docker image,
#### 0. Download COCO 2017 trainval dataset and annotation
```=bash
cd /home/openvino
apt update
apt install unzip
mkdir coco_dataset
cd coco_dataset
curl http://images.cocodataset.org/zips/val2017.zip -o val2017.zip
unzip val2017.zip
curl http://images.cocodataset.org/annotations/annotations_trainval2017.zip -o trainval_2017.zip
unzip trainval_2017.zip
```
##### coco_dataset content
```
coco_dataset/
|--annotations/
|-- captions_train2017.json
|-- captions_val2017.json
|-- instances_train2017.json
|-- instances_val2017.json
|-- person_keypoints_train2017.json
`-- person_keypoints_val2017.json
|-- val2017/
|-- 000000042102.jpg
|-- 000000060102.jpg
|-- 000000245102.jpg
...
`-- 000000364102.jpg
```
#### 1. Download yolo-v4-tf
```=bash
omz_downloader.py --name yolo-v4-tf -o /home/openvino/openvino_models
```
#### 2. Convert yolo-v4-tf to IR
```=bash
omz_converter.py --name yolo-v4-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models
```
#### 3. Run Accuracy Checker on yolo-v4-tf
```=bash
accuracy_check -c yolo-v3-4f-int8.yml -m /home/openvino/openvino_models/public/yolo-v4-tf/FP32
```
#### 4. Run POT on yolo-v3-tf
```=bash
pot -c yolo-v4-tf-int8.json -e
```
#### 5. Copy yolo-v4-tf FP16-INT8 IR
```=bash
mkdir /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/
cp -ar results/yolo-v4-tf_DefaultQuantization/2*/optimized/* /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/
#### Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host and /mnt in the container.
```
## Reference
### yolo-v4-tf-int8.yml
```
models:
- name: yolo_v4_tf
launchers:
- framework: openvino
device: CPU
adapter:
type: yolo_v3
anchors: 12,16,19,36,40,28,36,75,76,55,72,146,142,110,192,243,459,401
num: 3
coords: 4
classes: 80
threshold: 0.001
anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
raw_output: True
output_format: HWB
outputs:
- Func/StatefulPartitionedCall/output/_544
- Func/StatefulPartitionedCall/output/_542
- Func/StatefulPartitionedCall/output/_543
datasets:
- name: ms_coco_detection_80_class_without_background
annotation_conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
data_source: /home/openvino/coco_dataset/val2017
preprocessing:
- type: resize
size: 608
postprocessing:
- type: resize_prediction_boxes
- type: filter
apply_to: prediction
min_confidence: 0.001
remove_filtered: true
- type: diou_nms
overlap: 0.5
- type: clip_boxes
apply_to: prediction
metrics:
- type: map
integral: 11point
ignore_difficult: true
presenter: print_scalar
reference: 0.7123
- name: AP@0.5
type: coco_precision
max_detections: 100
threshold: 0.5
reference: 0.774
- name: AP@0.5:0.05:95
type: coco_precision
max_detections: 100
threshold: '0.5:0.05:0.95'
reference: 0.5026
```
### yolo-v4-tf-int8.json
```
{
"model": {
"model_name": "yolo-v4-tf",
"model": "/mnt/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.xml",
"weights": "/mnt/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.bin"
},
"engine": {
"config": "/mnt/yolo-v4-tf-int8.yml"
},
"compression": {
"algorithms": [
{
"name": "DefaultQuantization",
"params": {
"preset": "performance",
"stat_subset_size": 300
}
}
]
}
}
```
### accuracy_checker log
```
accuracy_check -c /mnt/yolo-v4-tf-int8.yml -m /mnt/openvino_models/public/yolo-v4-tf/FP32 -ss 300
Processing info:
model: yolo_v4_tf
launcher: openvino
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.5
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.485s
200 / 5000 processed in 0.482s
300 / 5000 processed in 0.484s
400 / 5000 processed in 0.483s
500 / 5000 processed in 0.483s
600 / 5000 processed in 0.485s
700 / 5000 processed in 0.488s
800 / 5000 processed in 0.490s
900 / 5000 processed in 0.488s
1000 / 5000 processed in 0.489s
1100 / 5000 processed in 0.495s
1200 / 5000 processed in 0.493s
1300 / 5000 processed in 0.494s
1400 / 5000 processed in 0.495s
1500 / 5000 processed in 0.494s
1600 / 5000 processed in 0.498s
1700 / 5000 processed in 0.501s
1800 / 5000 processed in 0.500s
1900 / 5000 processed in 0.502s
2000 / 5000 processed in 0.506s
2100 / 5000 processed in 0.499s
2200 / 5000 processed in 0.502s
2300 / 5000 processed in 0.501s
2400 / 5000 processed in 0.501s
2500 / 5000 processed in 0.498s
2600 / 5000 processed in 0.500s
2700 / 5000 processed in 0.500s
2800 / 5000 processed in 0.500s
2900 / 5000 processed in 0.499s
3000 / 5000 processed in 0.499s
3100 / 5000 processed in 0.496s
3200 / 5000 processed in 0.499s
3300 / 5000 processed in 0.495s
3400 / 5000 processed in 0.495s
3500 / 5000 processed in 0.494s
3600 / 5000 processed in 0.495s
3700 / 5000 processed in 0.491s
3800 / 5000 processed in 0.490s
3900 / 5000 processed in 0.490s
4000 / 5000 processed in 0.489s
4100 / 5000 processed in 0.486s
4200 / 5000 processed in 0.484s
4300 / 5000 processed in 0.483s
4400 / 5000 processed in 0.534s
4500 / 5000 processed in 0.514s
4600 / 5000 processed in 0.511s
4700 / 5000 processed in 0.491s
4800 / 5000 processed in 0.499s
4900 / 5000 processed in 0.524s
5000 / 5000 processed in 0.510s
5000 objects processed in 24.805 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
IE version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
Loaded CPU plugin version:
CPU - openvino_intel_cpu_plugin: 2022.1.2022.1.0-7019-cdb9bec7210-releases/2022/1
Found model /mnt/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.xml
Found weights /mnt/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.bin
Input info:
Node name: image_input
Tensor names: image_input:0, image_input
precision: f32
shape: (1, 608, 608, 3)
Output info
Node name: Func/StatefulPartitionedCall/output/_542:0
Tensor names: StatefulPartitionedCall/model/conv2d_101/BiasAdd:0, StatefulPartitionedCall/Identity:0, Func/StatefulPartitionedCall/output/_542:0, Identity:0
precision: f32
shape: (1, 38, 38, 255)
Node name: Func/StatefulPartitionedCall/output/_543:0
Tensor names: Identity_1:0, StatefulPartitionedCall/model/conv2d_109/BiasAdd:0, Func/StatefulPartitionedCall/output/_543:0, StatefulPartitionedCall/Identity_1:0
precision: f32
shape: (1, 19, 19, 255)
Node name: Func/StatefulPartitionedCall/output/_544:0
Tensor names: Identity_2:0, StatefulPartitionedCall/Identity_2:0, Func/StatefulPartitionedCall/output/_544:0, StatefulPartitionedCall/model/conv2d_93/BiasAdd:0
precision: f32
shape: (1, 76, 76, 255)
0%| | 1/300 [00:00<02:53]12:30:45 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/openvino/tools/accuracy_checker/adapters/yolo.py:426: RuntimeWarning: overflow encountered in exp
prob_correct=lambda x: 1.0 / (1.0 + np.exp(-x)))
300 objects processed in 167.019 seconds
map: 71.78% [FAILED: abs error = 0.5475 | relative error = 0.007686]
AP@0.5: 78.38% [FAILED: abs error = 0.9796 | relative error = 0.01266]
AP@0.5:0.05:95: 53.79% [FAILED: abs error = 3.53 | relative error = 0.07023]
```
### POT log
```
pot -c /mnt/yolo-v4-tf-int8.json -e --output-dir INFO:openvino.tools.pot.app.run:Output log dir: /mnt/openvino_models/public/yolo-v4-tf/FP16-INT8/yolo-v4-tf_DefaultQuantization/2022-05-02_13-42-23
INFO:openvino.tools.pot.app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : performance
stat_subset_size : 300
target_device : ANY
model_type : None
dump_intermediate_model : False
inplace_statistics : True
exec_log_dir : /mnt/openvino_models/public/yolo-v4-tf/FP16-INT8/yolo-v4-tf_DefaultQuantization/2022-05-02_13-42-23
===========================================================================
IE version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
Loaded CPU plugin version:
CPU - openvino_intel_cpu_plugin: 2022.1.2022.1.0-7019-cdb9bec7210-releases/2022/1
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.543s
200 / 5000 processed in 0.537s
300 / 5000 processed in 0.544s
400 / 5000 processed in 0.554s
500 / 5000 processed in 0.545s
600 / 5000 processed in 0.543s
700 / 5000 processed in 0.556s
800 / 5000 processed in 0.561s
900 / 5000 processed in 0.616s
1000 / 5000 processed in 0.623s
1100 / 5000 processed in 0.633s
1200 / 5000 processed in 0.621s
1300 / 5000 processed in 0.615s
1400 / 5000 processed in 0.617s
1500 / 5000 processed in 0.624s
1600 / 5000 processed in 0.616s
1700 / 5000 processed in 0.620s
1800 / 5000 processed in 0.618s
1900 / 5000 processed in 0.619s
2000 / 5000 processed in 0.587s
2100 / 5000 processed in 0.572s
2200 / 5000 processed in 0.563s
2300 / 5000 processed in 0.560s
2400 / 5000 processed in 0.565s
2500 / 5000 processed in 0.562s
2600 / 5000 processed in 0.559s
2700 / 5000 processed in 0.561s
2800 / 5000 processed in 0.587s
2900 / 5000 processed in 0.602s
3000 / 5000 processed in 0.558s
3100 / 5000 processed in 0.558s
3200 / 5000 processed in 0.557s
3300 / 5000 processed in 0.556s
3400 / 5000 processed in 0.554s
3500 / 5000 processed in 0.553s
3600 / 5000 processed in 0.552s
3700 / 5000 processed in 0.550s
3800 / 5000 processed in 0.550s
3900 / 5000 processed in 0.552s
4000 / 5000 processed in 0.547s
4100 / 5000 processed in 0.546s
4200 / 5000 processed in 0.544s
4300 / 5000 processed in 0.545s
4400 / 5000 processed in 0.542s
4500 / 5000 processed in 0.542s
4600 / 5000 processed in 0.539s
4700 / 5000 processed in 0.543s
4800 / 5000 processed in 0.540s
4900 / 5000 processed in 0.539s
5000 / 5000 processed in 0.535s
5000 objects processed in 28.424 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
INFO:openvino.tools.pot.pipeline.pipeline:Inference Engine version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.pipeline.pipeline:Model Optimizer version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.pipeline.pipeline:Post-Training Optimization Tool version: 2022.1.0-7019-cdb9bec7210-releases/2022/1
INFO:openvino.tools.pot.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:openvino.tools.pot.statistics.collector:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Finished: DefaultQuantization
===========================================================================
INFO:openvino.tools.pot.pipeline.pipeline:Evaluation of generated model
INFO:openvino.tools.pot.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 5000
1000 / 5000 processed in 173.116s
2000 / 5000 processed in 176.532s
3000 / 5000 processed in 173.620s
4000 / 5000 processed in 175.386s
5000 / 5000 processed in 175.397s
5000 objects processed in 874.052 seconds
INFO:openvino.tools.pot.engines.ac_engine:Inference finished
INFO:openvino.tools.pot.app.run:map : 0.7084540024167995
INFO:openvino.tools.pot.app.run:AP@0.5 : 0.7726537047800925
INFO:openvino.tools.pot.app.run:AP@0.5:0.05:95 : 0.4798230247936459
```
Sign in with Wallet
Connect another wallet