# Use POT to Quantize deeplabv3 Public Model
###### tags: `POT`
#### https://docs.openvinotoolkit.org/latest/omz_models_model_deeplabv3.html
## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt --rm openvino/ubuntu20_data_dev:latest
```
## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,
#### 0. Download VOC2012 trainval dataset and annotation
cd /home/openvino
mkdir VOC2012
cd VOC2012
curl http://host.robots.ox.ac.uk:8080/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar -o VOCtrainval_11-May-2012.tar
tar -xf VOCtrainval_11-May-2012.tar
#### 1. Download deeplabv3
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/downloader.py -o /home/openvino/openvino_models --name deeplabv3
#### 2. Convert deeplabv3 to IR
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/converter.py -d /home/openvino/openvino_models -o /home/openvino/openvino_models --name deeplabv3
#### 3. Run Accuracy Checker on deeplabv3
accuracy_check -c deeplabv3-int8.yml -m /home/openvino/openvino_models/public/deeplabv3/FP16/
#### 4. Run POT on deeplabv3
rm voc2012_segmentation.*
pot -c deeplabv3-int8.json -e
#### 5. Copy deeplabv3 FP16-INT8 IR
mkdir /home/openvino/openvino_models/public/deeplabv3/FP16-INT8/
cp -ar results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41/optimized/* /home/openvino/openvino_models/public/deeplabv3/FP16-INT8/
## Referece
```
drwxr-xr-x 3 openvino openvino 4096 Apr 13 12:34 VOC2012
-rw-r--r-- 1 openvino openvino 584 Apr 13 12:43 deeplabv3-int8.json
-rw-r--r-- 1 openvino openvino 1090 Apr 13 12:39 deeplabv3-int8.yml
-rw-r--r-- 1 openvino openvino 688 Apr 13 12:48 voc2012_segmentation.json
-rw-r--r-- 1 openvino openvino 400414 Apr 13 12:48 voc2012_segmentation.pickle
drwxr-xr-x 3 openvino openvino 4096 Apr 13 12:48 results
results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41/optimized/deeplabv3.bin
results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41/optimized/deeplabv3.xml
results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41/optimized/deeplabv3.mapping
results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41/log.txt
drwxr-xr-x 3 openvino openvino 4096 Apr 13 12:31 openvino_models
openvino_models/public/deeplabv3/FP32/deeplabv3.bin
openvino_models/public/deeplabv3/FP32/deeplabv3.xml
openvino_models/public/deeplabv3/FP32/deeplabv3.mapping
openvino_models/public/deeplabv3/FP16-INT8/deeplabv3.bin
openvino_models/public/deeplabv3/FP16-INT8/deeplabv3.xml
openvino_models/public/deeplabv3/FP16-INT8/deeplabv3.mapping
openvino_models/public/deeplabv3/deeplabv3_mnv2_pascal_train_aug
openvino_models/public/deeplabv3/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb
openvino_models/public/deeplabv3/deeplabv3_mnv2_pascal_train_aug/model.ckpt-30000.data-00000-of-00001
openvino_models/public/deeplabv3/deeplabv3_mnv2_pascal_train_aug/model.ckpt-30000.index
openvino_models/public/deeplabv3/FP16/deeplabv3.bin
openvino_models/public/deeplabv3/FP16/deeplabv3.xml
openvino_models/public/deeplabv3/FP16/deeplabv3.mapping
```
### deeplabv3-int8.yml
```
models:
- name: deeplabv3
launchers:
- framework: dlsdk
device: CPU
adapter: segmentation
datasets:
- name: VOC2012_Segmentation
preprocessing:
- type: padding
size: 513
enable_resize: True
postprocessing:
- type: encode_segmentation_mask
apply_to: annotation
- type: extend_segmentation_mask
metrics:
- type: mean_iou
use_argmax: false
presenter: print_scalar
annotation_conversion:
converter: voc_segmentation
imageset_file: /home/openvino/VOC2012/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt
images_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/JPEGImages/
mask_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/SegmentationClass/
data_source: /home/openvino/VOC2012/VOCdevkit/VOC2012/JPEGImages/
segmentation_masks_source: /home/openvino/VOC2012/VOCdevkit/VOC2012/SegmentationClass/
annotation: voc2012_segmentation.pickle
dataset_meta: voc2012_segmentation.json
```
### deeplabv3-int8.json
```
{
"model": {
"model_name": "deeplabv3",
"model": "/home/openvino/openvino_models/public/deeplabv3/FP16/deeplabv3.xml",
"weights": "/home/openvino/openvino_models/public/deeplabv3/FP16/deeplabv3.bin"
},
"engine": {
"config": "/home/openvino/deeplabv3-int8.yml"
},
"compression": {
"algorithms": [
{
"name": "DefaultQuantization",
"params": {
"preset": "performance",
"stat_subset_size": 100
}
}
]
}
}
```
### accuracy_checker log
```
accuracy_check -c deeplabv3-int8.yml -m /home/openvino/openvino_models/public/deeplabv3/FP16/ -td CPU
Processing info:
model: deeplabv3
launcher: dlsdk
device: CPU
dataset: VOC2012_Segmentation
OpenCV version: 4.5.2-openvino
Annotation for VOC2012_Segmentation dataset will be loaded from voc2012_segmentation.pickle
Loaded dataset info:
Dataset name: VOC2012_Segmentation
Accuracy Checker version 0.8.6
Dataset size 1449
Conversion parameters:
converter: voc_segmentation
imageset_file: /home/openvino/VOC2012/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt
images_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/JPEGImages
mask_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/SegmentationClass
VOC2012_Segmentation dataset metadata will be loaded from voc2012_segmentation.json
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Found model /home/openvino/openvino_models/public/deeplabv3/FP16/deeplabv3.xml
Found weights /home/openvino/openvino_models/public/deeplabv3/FP16/deeplabv3.bin
Input info:
Layer name: mul_1/placeholder_port_1
precision: FP32
shape [1, 3, 513, 513]
Output info
Layer name: ArgMax/Squeeze
precision: I32
shape: [1, 513, 513]
1449 objects processed in 248.891 seconds
mean_iou: 66.84%
```
### pot log
```
pot -c deeplabv3-int8.json -e
12:48:40 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/graph.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping
12:48:40 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping, Set, Iterable
12:48:41 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/optimization/algorithm.py:41: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+
warnings.warn(
12:48:41 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import reload
INFO:app.run:Output log dir: ./results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41
INFO:app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : performance
stat_subset_size : 100
target_device : ANY
model_type : None
dump_intermediate_model : False
exec_log_dir : ./results/deeplabv3_DefaultQuantization/2021-04-13_12-48-41
===========================================================================
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Annotation conversion for VOC2012_Segmentation dataset has been started
Parameters to be used for conversion:
converter: voc_segmentation
imageset_file: /home/openvino/VOC2012/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt
images_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/JPEGImages
mask_dir: /home/openvino/VOC2012/VOCdevkit/VOC2012/SegmentationClass
Annotation conversion for VOC2012_Segmentation dataset has been finished
VOC2012_Segmentation dataset metadata will be saved to voc2012_segmentation.json
Converted annotation for VOC2012_Segmentation dataset will be saved to voc2012_segmentation.pickle
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
12:48:49 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/back/ie_ir_ver_2/emitter.py:243: DeprecationWarning: This method will be removed in future versions. Use 'list(elem)' or iteration over elem instead.
if len(element.attrib) == 0 and len(element.getchildren()) == 0:
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 1449
1000 / 1449 processed in 99.081s
1449 objects processed in 142.010 seconds
INFO:compression.engines.ac_engine:Inference finished
INFO:app.run:mean_iou : 0.66797033519794
```
Sign in with Wallet
Connect another wallet