# Use POT to Quantize yolo-v4-tf Public Model
###### tags: `POT`
## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt -u root --rm openvino/ubuntu20_data_dev:latest
```
## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,
#### 0. Download COCO 2017 trainval dataset and annotation
```=bash
cd /home/openvino
apt update
apt install unzip
mkdir coco_dataset
cd coco_dataset
curl http://images.cocodataset.org/zips/val2017.zip -o val2017.zip
unzip val2017.zip
curl http://images.cocodataset.org/annotations/annotations_trainval2017.zip -o trainval_2017.zip
unzip trainval_2017.zip
```
##### coco_dataset content
```
coco_dataset/
|--annotations/
|-- captions_train2017.json
|-- captions_val2017.json
|-- instances_train2017.json
|-- instances_val2017.json
|-- person_keypoints_train2017.json
`-- person_keypoints_val2017.json
|-- val2017/
|-- 000000042102.jpg
|-- 000000060102.jpg
|-- 000000245102.jpg
...
`-- 000000364102.jpg
```
#### 1. Download yolo-v4-tf
```=bash
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/downloader.py --name yolo-v4-tf -o /home/openvino/openvino_models
```
#### 2. Convert yolo-v4-tf to IR
```=bash
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/converter.py --name yolo-v4-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models
```
#### 3. Run Accuracy Checker on yolo-v4-tf
```=bash
accuracy_check -c yolo-v4-tf-int8.yml -m /home/openvino/openvino_models/public/yolo-v4-tf/FP32 -d /opt/intel/openvino_2021.3.394/deployment_tools/open_model_zoo/tools/accuracy_checker/dataset_definitions.yml -ss 300
```
#### 4. Run POT on yolo-v4-tf
```=bash
pot -c yolo-v4-tf-int8.json -e
```
#### 5. Copy yolo-v4-tf FP16-INT8 IR
```=bash
mkdir /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/
cp -ar results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/optimized/* /home/openvino/openvino_models/public/yolo-v4-tf/FP16-INT8/
```
#### Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host and /mnt in the container.
## Referece
```
drwxr-xr-x 4 openvino openvino 4096 Apr 13 03:27 coco_dataset
drwxr-xr-x 3 root root 4096 Apr 13 03:36 openvino_models
drwxr-xr-x 4 root root 4096 Apr 13 05:44 results
results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/log.txt
results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/optimized
results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/optimized/yolo-v4-tf.xml
results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/optimized/yolo-v4-tf.bin
results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20/optimized/yolo-v4-tf.mapping
-rw-r--r-- 1 root root 591 Apr 13 05:44 yolo-v4-tf-int8.json
-rw-r--r-- 1 root root 1571 Apr 13 05:37 yolo-v4-tf-int8.yml
openvino_models/
`-- public
`-- yolo-v4-tf
|-- FP16
| |-- yolo-v4-tf.bin
| |-- yolo-v4-tf.mapping
| `-- yolo-v4-tf.xml
|-- FP16-INT8
| |-- yolo-v4-tf.bin
| |-- yolo-v4-tf.mapping
| `-- yolo-v4-tf.xml
|-- FP32
| |-- yolo-v4-tf.bin
| |-- yolo-v4-tf.mapping
| `-- yolo-v4-tf.xml
|-- keras-YOLOv3-model-set
| |-- cfg
| | `-- yolov4.cfg
| |-- common
| | |-- __pycache__
| | | `-- utils.cpython-38.pyc
| | |-- utils.py
| | `-- utils.py.orig
| |-- tools
| | `-- model_converter
| | |-- convert.py
| | |-- keras_to_tensorflow.py
| | `-- keras_to_tensorflow.py.orig
| `-- yolo4
| `-- models
| |-- __pycache__
| | `-- layers.cpython-38.pyc
| |-- layers.py
| `-- layers.py.orig
|-- yolo-v4.h5
|-- yolo-v4.pb
`-- yolov4.weights
```
### yolo-v4-tf-int8.yml
```
models:
- name: yolo_v4_tf
launchers:
- framework: dlsdk
device: CPU
adapter:
type: yolo_v3
anchors: 12,16,19,36,40,28,36,75,76,55,72,146,142,110,192,243,459,401
num: 3
coords: 4
classes: 80
threshold: 0.001
anchor_masks: [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
raw_output: True
outputs:
- conv2d_93/BiasAdd/Add
- conv2d_101/BiasAdd/Add
- conv2d_109/BiasAdd/Add
datasets:
- name: ms_coco_detection_80_class_without_background
annotation_conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
data_source: /home/openvino/coco_dataset/val2017
preprocessing:
- type: resize
size: 608
postprocessing:
- type: resize_prediction_boxes
- type: filter
apply_to: prediction
min_confidence: 0.001
remove_filtered: true
- type: diou_nms
overlap: 0.5
- type: clip_boxes
apply_to: prediction
metrics:
- type: map
integral: 11point
ignore_difficult: true
presenter: print_scalar
- name: AP@0.5
type: coco_precision
max_detections: 100
threshold: 0.5
- name: AP@0.5:0.05:95
type: coco_precision
max_detections: 100
threshold: '0.5:0.05:0.95'
```
### yolo-v4-tf-int8.json
```
{
"model": {
"model_name": "yolo-v4-tf",
"model": "/home/openvino/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.xml",
"weights": "/home/openvino/openvino_models/public/yolo-v4-tf/FP16/yolo-v4-tf.bin"
},
"engine": {
"config": "/home/openvino/yolo-v4-tf-int8.yml"
},
"compression": {
"algorithms": [
{
"name": "DefaultQuantization",
"params": {
"preset": "performance",
"stat_subset_size": 300
}
}
]
}
}
```
### accuracy_checker log
```
accuracy_check -c yolo-v4-tf-int8.yml -m /home/openvino/openvino_models/public/yolo-v4-tf/FP32 -d /opt/intel/openvino_2021.3.394/deployment_tools/open_model_zoo/tools/accuracy_checker/dataset_definitions.yml -ss 300
Processing info:
model: yolo_v4_tf
launcher: dlsdk
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.2-openvino
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.508s
200 / 5000 processed in 0.465s
300 / 5000 processed in 0.453s
400 / 5000 processed in 0.445s
500 / 5000 processed in 0.440s
600 / 5000 processed in 0.437s
700 / 5000 processed in 0.435s
800 / 5000 processed in 0.433s
900 / 5000 processed in 0.432s
1000 / 5000 processed in 0.434s
1100 / 5000 processed in 0.433s
1200 / 5000 processed in 0.436s
1300 / 5000 processed in 0.435s
1400 / 5000 processed in 0.522s
1500 / 5000 processed in 0.509s
1600 / 5000 processed in 0.484s
1700 / 5000 processed in 0.469s
1800 / 5000 processed in 0.463s
1900 / 5000 processed in 0.457s
2000 / 5000 processed in 0.454s
2100 / 5000 processed in 0.453s
2200 / 5000 processed in 0.449s
2300 / 5000 processed in 0.450s
2400 / 5000 processed in 0.449s
2500 / 5000 processed in 0.448s
2600 / 5000 processed in 0.444s
2700 / 5000 processed in 0.445s
2800 / 5000 processed in 0.444s
2900 / 5000 processed in 0.441s
3000 / 5000 processed in 0.442s
3100 / 5000 processed in 0.439s
3200 / 5000 processed in 0.439s
3300 / 5000 processed in 0.436s
3400 / 5000 processed in 0.435s
3500 / 5000 processed in 0.435s
3600 / 5000 processed in 0.433s
3700 / 5000 processed in 0.431s
3800 / 5000 processed in 0.428s
3900 / 5000 processed in 0.428s
4000 / 5000 processed in 0.424s
4100 / 5000 processed in 0.423s
4200 / 5000 processed in 0.422s
4300 / 5000 processed in 0.419s
4400 / 5000 processed in 0.420s
4500 / 5000 processed in 0.416s
4600 / 5000 processed in 0.417s
4700 / 5000 processed in 0.413s
4800 / 5000 processed in 0.417s
4900 / 5000 processed in 0.418s
5000 / 5000 processed in 0.414s
5000 objects processed in 22.117 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
ms_coco_detection_80_class_without_background dataset metadata will be saved to mscoco_det_80.json
Converted annotation for ms_coco_detection_80_class_without_background dataset will be saved to mscoco_det_80.pickle
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Found model /home/openvino/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.xml
Found weights /home/openvino/openvino_models/public/yolo-v4-tf/FP32/yolo-v4-tf.bin
Input info:
Layer name: image_input
precision: FP32
shape [1, 3, 608, 608]
Output info
Layer name: conv2d_101/BiasAdd/Add
precision: FP32
shape: [1, 255, 38, 38]
Layer name: conv2d_109/BiasAdd/Add
precision: FP32
shape: [1, 255, 19, 19]
Layer name: conv2d_93/BiasAdd/Add
precision: FP32
shape: [1, 255, 76, 76]
0%|▍ | 1/300 [00:00<04:54]05:37:51 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/adapters/yolo.py:373: RuntimeWarning: overflow encountered in exp
prob_correct=lambda x: 1.0 / (1.0 + np.exp(-x)))
300 objects processed in 317.375 seconds
map: 71.78%
AP@0.5: 78.38%
AP@0.5:0.05:95: 53.79%
```
### pot log
```
pot -c yolo-v4-tf-int8.json -e
05:44:19 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/graph.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping
05:44:19 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping, Set, Iterable
05:44:20 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/optimization/algorithm.py:41: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+
warnings.warn(
05:44:20 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import reload
INFO:app.run:Output log dir: ./results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20
INFO:app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : performance
stat_subset_size : 300
target_device : ANY
model_type : None
dump_intermediate_model : False
exec_log_dir : ./results/yolo-v4-tf_DefaultQuantization/2021-04-13_05-44-20
===========================================================================
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been started
Parameters to be used for conversion:
converter: mscoco_detection
annotation_file: /home/openvino/coco_dataset/annotations/instances_val2017.json
Total annotations size: 5000
100 / 5000 processed in 0.534s
200 / 5000 processed in 0.484s
300 / 5000 processed in 0.464s
400 / 5000 processed in 0.455s
500 / 5000 processed in 0.449s
600 / 5000 processed in 0.450s
700 / 5000 processed in 0.449s
800 / 5000 processed in 0.467s
900 / 5000 processed in 0.490s
1000 / 5000 processed in 0.468s
1100 / 5000 processed in 0.464s
1200 / 5000 processed in 0.453s
1300 / 5000 processed in 0.452s
1400 / 5000 processed in 0.449s
1500 / 5000 processed in 0.451s
1600 / 5000 processed in 0.448s
1700 / 5000 processed in 0.450s
1800 / 5000 processed in 0.448s
1900 / 5000 processed in 0.450s
2000 / 5000 processed in 0.450s
2100 / 5000 processed in 0.448s
2200 / 5000 processed in 0.453s
2300 / 5000 processed in 0.448s
2400 / 5000 processed in 0.455s
2500 / 5000 processed in 0.446s
2600 / 5000 processed in 0.447s
2700 / 5000 processed in 0.458s
2800 / 5000 processed in 0.450s
2900 / 5000 processed in 0.448s
3000 / 5000 processed in 0.447s
3100 / 5000 processed in 0.444s
3200 / 5000 processed in 0.442s
3300 / 5000 processed in 0.442s
3400 / 5000 processed in 0.438s
3500 / 5000 processed in 0.439s
3600 / 5000 processed in 0.437s
3700 / 5000 processed in 0.436s
3800 / 5000 processed in 0.442s
3900 / 5000 processed in 0.437s
4000 / 5000 processed in 0.432s
4100 / 5000 processed in 0.429s
4200 / 5000 processed in 0.432s
4300 / 5000 processed in 0.426s
4400 / 5000 processed in 0.424s
4500 / 5000 processed in 0.422s
4600 / 5000 processed in 0.420s
4700 / 5000 processed in 0.420s
4800 / 5000 processed in 0.417s
4900 / 5000 processed in 0.418s
5000 / 5000 processed in 0.418s
5000 objects processed in 22.341 seconds
Annotation conversion for ms_coco_detection_80_class_without_background dataset has been finished
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
05:45:06 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/back/ie_ir_ver_2/emitter.py:243: DeprecationWarning: This method will be removed in future versions. Use 'list(elem)' or iteration over elem instead.
if len(element.attrib) == 0 and len(element.getchildren()) == 0:
05:45:37 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/libs/open_model_zoo/tools/accuracy_checker/accuracy_checker/adapters/yolo.py:373: RuntimeWarning: overflow encountered in exp
prob_correct=lambda x: 1.0 / (1.0 + np.exp(-x)))
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 5000
1000 / 5000 processed in 1028.772s
2000 / 5000 processed in 1026.357s
3000 / 5000 processed in 1026.659s
4000 / 5000 processed in 1030.753s
5000 / 5000 processed in 1026.593s
5000 objects processed in 5139.134 seconds
INFO:compression.engines.ac_engine:Inference finished
INFO:app.run:map : 0.7059990099769229
INFO:app.run:AP@0.5 : 0.7724924313572912
INFO:app.run:AP@0.5:0.05:95 : 0.4788317444262568
```
#### Replace weights and cfg
I0415 13:47:35.862991 140451038899072 keras_to_tensorflow.py:183] Saved the freezed graph at /home/openvino/openvino_models/public/yolo-v4-tf/yolo-v4.pb
ls /home/openvino/openvino_models/public/yolo-v4-tf/
FP16 FP32 keras-YOLOv3-model-set yolo-v4.h5 yolo-v4.pb yolov4.weights
/home/openvino/openvino_models/public/yolo-v4-tf/keras-YOLOv3-model-set/cfg/yolov4.cfg
/usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py
--framework=tf --data_type=FP16
--output_dir=/home/openvino/openvino_models/public/yolo-v4-tf/FP16 --model_name=yolo-v4-tf
'--input_shape=[1,608,608,3]'
--input=image_input '--scale_values=image_input[255]'
--reverse_input_channels --input_model=/home/openvino/openvino_models/public/yolo-v4-tf/yolo-v4.pb
/opt/intel/openvino_2021.3.394/deployment_tools/open_model_zoo/models/public/yolo-v4-tf/model.yml
```
model_optimizer_args:
- --input_shape=[1,608,608,3]
- --input=image_input
- --scale_values=image_input[255]
- --reverse_input_channels
- --input_model=$conv_dir/yolo-v4.pb
````
Sign in with Wallet
Connect another wallet