# Use POT to Quantize resnet-50-tf Public Model
###### tags: `POT`
## Use OpenVINO dockerhub image
```
docker run -it -v ~/Downloads:/mnt --rm openvino/ubuntu20_data_dev:latest
```
## Run Accuracy Checker and POT
In ubuntu20_data_dev docker image,
#### 1. Download resnet-50-tf
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/downloader.py --name resnet-50-tf -o /home/openvino/openvino_models
#### 2. Convert resnet-50-tf to IR
python3 /opt/intel/openvino_2021.3.394/deployment_tools/tools/model_downloader/converter.py --name resnet-50-tf -d /home/openvino/openvino_models -o /home/openvino/openvino_models
#### 3. Run Accuracy Checker on resnet-50-tf
accuracy_check -c resnet-50-tf-int8.yml -m openvino_models/public/resnet-50-tf/FP16/
#### 4. Run POT on resnet-50-tf
rm imagenet1001.pickle
pot -c resnet-50-tf-int8.json
#### 5. Copy resnet-50-tf FP16-INT8 IR
mkdir /home/openvino/openvino_models/public/resnet-50-tf/FP16-INT8/
cp -ar results/resnet-50-tf_DefaultQuantization/2021-03-30_14-15-39/optimized/* /home/openvino/openvino_models/public/resnet-50-tf/FP16-INT8/
## Reference
```
drwxrwxr-x 2 openvino openvino 12288 Mar 30 13:08 ILSVRC2012_img_val
-rw-r--r-- 1 openvino openvino 65725 Mar 30 14:01 imagenet1001.pickle
-rw-r--r-- 1 openvino openvino 57035 Mar 30 13:59 imagenet1001.pickle.acc
-rwxr-xr-x 1 openvino openvino 602 Mar 30 13:55 resnet-50-tf-int8.json
-rw-r--r-- 1 openvino openvino 762 Mar 30 13:54 resnet-50-tf-int8.yml
drwxr-xr-x 3 openvino openvino 4096 Mar 30 13:59 results
drwxrwxrwx 1 openvino openvino 20 Mar 30 13:26 openvino_models
openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.bin
openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.xml
openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.mapping
openvino_models/public/resnet-50-tf/FP32/resnet-50-tf.bin
openvino_models/public/resnet-50-tf/FP32/resnet-50-tf.xml
openvino_models/public/resnet-50-tf/FP32/resnet-50-tf.mapping
openvino_models/public/resnet-50-tf/FP16-INT8/resnet-50-tf.bin
openvino_models/public/resnet-50-tf/FP16-INT8/resnet-50-tf.xml
openvino_models/public/resnet-50-tf/FP16-INT8/resnet-50-tf.mapping
```
Note : Copy openvino_models foler to /mnt folder. They will be accessable in ~/Downloads folder in the host and /mnt in the container.
### resnet-50-tf-int8.yml
```
models:
- name: resnet-50-tf
launchers:
- framework: dlsdk
device: CPU
adapter: classification
datasets:
- name: imagenet_1001_classes
annotation_conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
has_background: True
annotation: imagenet1001.pickle
data_source: /home/openvino/ILSVRC2012_img_val
metrics:
- name: accuracy@top1
type: accuracy
top_k: 1
- name: accuracy@top5
type: accuracy
top_k: 5
preprocessing:
- type: resize
size: 256
aspect_ratio_scale: greater
- type: crop
size: 224
```
### resnet-50-tf-int8.json
```
{
"model": {
"model_name": "resnet-50-tf",
"model": "/home/openvino/openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.xml",
"weights": "/home/openvino/openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.bin"
},
"engine": {
"config": "/home/openvino/resnet-50-tf-int8.yml"
},
"compression": {
"algorithms": [
{
"name": "DefaultQuantization",
"params": {
"preset": "performance",
"stat_subset_size": 100
}
}
]
}
}
```
### accuracy_checker log
```
accuracy_check -c resnet-50-tf-int8.yml -m openvino_models/public/resnet-50-tf/FP16/
Processing info:
model: resnet-50-tf
launcher: dlsdk
device: CPU
dataset: imagenet_1001_classes
OpenCV version: 4.5.2-openvino
Annotation conversion for imagenet_1001_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
has_background: True
Annotation conversion for imagenet_1001_classes dataset has been finished
Converted annotation for imagenet_1001_classes dataset will be saved to imagenet1001.pickle
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Found model openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.xml
Found weights openvino_models/public/resnet-50-tf/FP16/resnet-50-tf.bin
Input info:
Layer name: map/TensorArrayStack/TensorArrayGatherV3
precision: FP32
shape [1, 3, 224, 224]
Output info
Layer name: softmax_tensor
precision: FP32
shape: [1, 1001]
200 objects processed in 7.133 seconds
accuracy@top1: 78.50%
accuracy@top5: 92.00%
```
### pot log
```
pot -c resnet-50-tf-int8.json
14:15:38 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/graph.py:23: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping
14:15:38 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
from collections import Mapping, Set, Iterable
14:15:38 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:112: DeprecationWarning: invalid escape sequence \
"""
14:15:38 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:182: DeprecationWarning: invalid escape sequence \
"""
14:15:38 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseInputChannels.py:283: DeprecationWarning: invalid escape sequence \
"""
14:15:38 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/front/tf/graph_utils.py:159: DeprecationWarning: invalid escape sequence \*
"""
14:15:38 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/compress_quantized_weights.py:30: DeprecationWarning: invalid escape sequence \
"""
14:15:39 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/tools/post_training_optimization_toolkit/compression/algorithms/quantization/optimization/algorithm.py:41: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+
warnings.warn(
14:15:39 accuracy_checker WARNING: /usr/local/lib/python3.8/dist-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import reload
INFO:app.run:Output log dir: ./results/resnet-50-tf_DefaultQuantization/2021-03-30_14-15-39
INFO:app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : performance
stat_subset_size : 100
target_device : ANY
model_type : None
dump_intermediate_model : False
exec_log_dir : ./results/resnet-50-tf_DefaultQuantization/2021-03-30_14-15-39
===========================================================================
IE version: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
Annotation conversion for imagenet_1001_classes dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: /home/openvino/ILSVRC2012_img_val/val.txt
has_background: True
Annotation conversion for imagenet_1001_classes dataset has been finished
Converted annotation for imagenet_1001_classes dataset will be saved to imagenet1001.pickle
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
14:15:48 accuracy_checker WARNING: /opt/intel/openvino/deployment_tools/model_optimizer/mo/back/ie_ir_ver_2/emitter.py:243: DeprecationWarning: This method will be removed in future versions. Use 'list(elem)' or iteration over elem instead.
if len(element.attrib) == 0 and len(element.getchildren()) == 0:
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
===========================================================================
```
Sign in with Wallet
Connect another wallet