# 如何運行Intel OpenVINO Open Model Zoo（OMZ）範例於Google Colab上


Intel OpenVINO[1] 從2018年5月發行至今已更新20多個版次，目前最新版次為 2023.3 LTS。期間有很多重大革新，這也讓很多不錯的功能可能消失或以其它方式出現。如2021.4版後開始提供Jupyter Notebook (Python) 格式範例讓大家更容易學習。2023.0版後開始提供 Google Colab 格式的範例，讓大家免本地安裝可直接在雲端體驗 OpenVINO。不過也有像 Open Model Zoo (OMZ)[2] 之類方便學習、測試各種預訓練模型的項目及相關工具被移出。主要是因為自2023.1版後開始支援直接讀取更多框架模型及神經網路壓縮架構(Neural Network Compression Framework, NNCF)取代原有模型優化器(Model Optimizer, MO)。

目前 OpenVINO Notebooks[3] 範例編號101~128提供了各種常見框架轉成OpenVINO專屬 IR（XML+BIN） 格式的範例，如TensorFlow, PyTorch, PaddlePaddel 等，甚至 Hugging Face 格式。當然這裡也包含了OMZ的轉換範例（104-model-tools）。不過這個範例比較適合單獨運行、預先轉檔工作，當想像以往 OMZ Demos 下的應用範例，要整合轉檔加執行時，就會遇到一些小麻煩。

雖然這些OMZ的範例已被淡出，但如果大家想直接在 Google Colab 上直接運行時，這篇文章應該可以給大家一些幫助。以下就以OMZ 影像分類(Classification) Public Pre-Trained Models[4]為例，說明如何以 Colab 完成直接運行。

## OpenVINO Open Model Zoo (OMZ) & Tools

Intel OpenVINO 自20018發行後 Open Model Zoo (OMZ) 一直提供很豐富的預訓練模型庫，很方便進行AI應用的實驗，其中包含三大部份。
* Intel's Pre-Trained
Intel 提供之預設訓練模型，包括物件偵測、物件辨識、影像分割、姿態偵測、行為分析、語音處理、時間預測等數十種模型。此部份已直接轉成IR格式(Bin + Xml)，可直接載入運行。
* Public Pre-Trained
常用公開預訓練模型，有數十種應用及數百個模型，下載後需依原始框架格式（如TensorFlow, PyTorch, Caffe等）轉換到IR格式才能載入運行。
* Demos
提供整合型範例，使用單一或組合不同模型完成特定功能，搭配簡易顯示畫面，更方便使用者理解。

另外亦提供OMZ相關工具，方便使用者處理模型轉檔、優化及效能評估，主要包括下列四項工具。

| Tool | Command | Description |
|:-----------------|:--------------------|:---------------------------- |
| Model Downloader | `omz_downloader` | 從Open Model Zoo下載模型 |
| Model Converter | `omz_converter` | 將模型轉換成 OpenVINO IR 格式 |
| Info Dumper | `omz_info_dumper` | 列印模型完整資訊 |
| Benchmark Tool | `benchmark_app` | 比較模型推論效能 |

隨著 OpenVINO 版本的迭代更新，在2023.1版後， OMZ[4] 及相關工具[5]在官方說明文件中就移到「舊版功能(Legacy Features)」，不再持續維護。使用 `pip install openvino` 也不再預安裝OMZ相關工具，需改用 `pip install openvino-dev` 才能取得。

舊版 Open Model Zoo 相關文件及範例仍於開源於 Github[6] 上，有需要可行下載研究。另外原始模型及說明文件亦可從「官方共用區」[7]中取得。

## OpenVINO Notebooks

為了讓大家更容易使用 OpenVINO ，自2021.4版後就開始提供 Jupyter Notebook 格式的範例程式庫 [Notebooks](https://github.com/openvinotoolkit/openvino_notebooks) ，主要分為五大部份。
* 第一步(First steps)（編號001~099）
* 轉換與優化(Covert & Optimize)（編號100~199）
* 模型展示(Model Demos)（編號200~299）
* 模型訓練(Model Training)（編號300~399）
* 即時展示(Live Demos)（編號400~499）

2022.1版後部份範例已支援直接在 Binder 線上運行。 2023.0版後除增加更多範例外，亦有部份範例支援直接在 Google Colab 環境下安裝 OpenVINO 及運行。不過目前 Notebooks 範例中並未納入原先 OMZ Pri-Trained Model 及 Demos 範例，如有需要可參考下列作法。

## OMZ Classification Public Pre-Trained Models

目前 OMZ Public Pre-Trained Models提供了常見的預訓練模型有數十種應用及數百個模型，單就影像分類來看就有超過五十種，詳如文末附錄。

以往可透過Demos下的「Classification Python* Demo」[8]在命令列運行，來呼叫這五十多種影像分類模型。當它利用 omz_downloader 下載模型時會一併把對應的參數設定好。可是這樣的作法並無法適用在 Colab 環境下，所以這裡可簡單把工作流程改成下列步驟，就能完整運行影像分類。

1. 安裝openvino-dev
2. 導入必要模組
3. 下載預訓練模型
4. 模型轉換
5. 指定推論裝置
6. 將模型載入裝置並編譯
7. 下載測試資料

更完整的細部程式說明可直接點擊下列連結，進入Colab直接運行。大家可試著修改模型名稱及指定對應設定來實驗一下。
[Colab完整範例](https://colab.research.google.com/github/OmniXRI/OpenVINO2023LTS_on_Colab_Samples/blob/main/OpenVINO_2023_LTS_Demo_Classification.ipynb)

## 小結

這個範例僅示範了OMZ影像分類用法，這只是一個開始，大家可試著延伸到其它類型範例，後續會持續安排將Demos下的範例轉至Colab，讓大家更方便學習。

## 參考文獻

[1] Intel, OpenVINO Toolkit Document
https://docs.openvino.ai/

[2] Intel, OpenVINO Document - Documentation - Legacy Features - Open Model Zoo (2023.3 LTS)
https://docs.openvino.ai/2023.3/model_zoo.html

[3] Intel, Github - openvinotoolkit / openvino_notebooks
https://github.com/openvinotoolkit/openvino_notebooks

[4] Intel, OpenVINO Document - Documentation - Legacy Features - Open Model Zoo - Public Pre-Training Models (2023.3 LTS)
https://docs.openvino.ai/2023.3/omz_models_group_public.html#classification-models

[5] Intel, OpenVINO Document - Documentation - Legacy Features - Model Optimizer / Conversion API (2023.3 LTS)
https://docs.openvino.ai/2023.3/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html

[6] Intel, Github - openvinotoolkit / open_model_zoo
https://github.com/openvinotoolkit/open_model_zoo

[7] Intel, stroage for open model zoo
https://storage.openvinotoolkit.org/repositories/open_model_zoo/

[8] Intel, OpenVINO Document - Classification Python* Demo
https://docs.openvino.ai/2023.3/omz_demos_classification_demo_python.html

## 附錄 - OMZ 影像分類模型清單
[[資料來源]](https://docs.openvino.ai/2023.3/omz_models_group_public.html#classification-models), OmniXRI 整理製作
| Model Name | Implementation | OMZ Model Name | Accuracy (Top1/5) | GFlops | MParams | INPUT | OUTPUT |
| ------ | :------: | :------: | :------: | :------: | ------: | :------: | :------: |
| AlexNet | Caffe* | alexnet | 56.598%/79.812% | 1.5 | 60.965 | [1, 3, 227, 227] | [1,1000] |
| AntiSpoofNet | PyTorch* | anti-spoof-mn3 | 3.81% | 0.15 | 3.02 | [1, 3, 128, 128] | [1,2] |
| CaffeNet | Caffe* | caffenet | 56.714%/79.916% | 1.463 | 60.965 | [1, 3, 227, 227] | [1,1000] |
| ConvNeXt Tiny | PyTorch* | convnext-tiny | 82.05%/95.86% | 8.9419 | 28.5892 | [1,3,224,224] | [1,1000] |
| DenseNet 121 | Caffe* | densenet-121 | 74.42%/92.136% | 5.274 | 7.971 | [1,3,224,224] | [1,1000] |
| DenseNet 121 | TensorFlow* | densenet-121-tf | 74.46%/92.13% | 5.7287 | 7.971 | [1,224,224,3] | [1,1000] |
| DLA 34 | PyTorch* | dla-34 | 74.64%/92.06% | 6.1368 | 15.7344 | [1,3,224,224] | [1,1000] |
| EfficientNet B0 | TensorFlow* | efficientnet-b0 | 75.70%/92.76% | 0.819 | 5.268 | [1,224,224,3] | [1,1000] |
| EfficientNet B0 | PyTorch* | efficientnet-b0-pytorch | 77.70%/93.52% | 0.819 | 5.268 | [1,3,224,224] | [1,1000] |
| EfficientNet V2 B0 | PyTorch* | efficientnet-v2-b0 | 78.36%/94.02% | 1.4641 | 7.1094 | [1,3,224,224] | [1,1000] |
| EfficientNet V2 Small | PyTorch* | efficientnet-v2-s | 84.29%/97.26% | 16.9406 | 21.3816 | [1,3,224,224] | [1,1000] |
| HBONet 1.0 | PyTorch* | hbonet-1.0 | 73.1%/91.0% | 0.6208 | 4.5443 | [1,3,224,224] | [1,1000] |
| HBONet 0.25 | PyTorch* | hbonet-0.25 | 57.3%/79.8% | 0.0758 | 1.9299 | [1,3,224,224] | [1,1000] |
| Inception (GoogleNet) V1 | Caffe* | googlenet-v | 68.928%/89.144% | 3.016 | 6.619 | [1,3,224,224] | [1,1000] |
| Inception (GoogleNet) V1 | TensorFlow* | googlenet-v1-tf | 69.814%/89.6% | 3.266 | 6.999 | [1,224,224,3] | [1,1001] |
| Inception (GoogleNet) V2 | Caffe* | googlenet-v2 | 72.024%/90.844% | 4.058 | 11.185 | [1,3,224,224] | [1,1000] |
| Inception (GoogleNet) V2 | TensorFlow* | googlenet-v2-tf | 74.084%/91.798% | 4.058 | 11.185 | [1,224,224,3] | [1,1001] |
| Inception (GoogleNet) V3 | TensorFlow* | googlenet-v3 | 77.904%/93.808% | 11.469 | 23.817 | [1,299,299,3] | [1,1001] |
| Inception (GoogleNet) V3 | PyTorch* | googlenet-v3-pytorch | 77.69%/93.7% | 11.469 | 23.817 | [1,3,299,299] | [1,1000] |
| Inception (GoogleNet) V4 | TensorFlow* | googlenet-v4-tf | 80.204%/95.21% | 24.584 | 42.648 | [1,299,299,3] | [1,1001] |
| Inception-ResNet V2 | TensorFlow* | inception-resnet-v2-tf | 77.82%/94.03% | 22.227 | 30.223 | [1,299,299,3] | [1,1001] |
| LeViT 128S | PyTorch* | levit-128s | 76.54%/92.85% | 0.6177 | 8.2199 | [1,3,224,224] | [1,1000] |
| MixNet L | TensorFlow* | mixnet-l | 78.30%/93.91% | 0.565 | 7.3 | [1,224,224,3] | [1,1000] |
| MobileNet V1 0.25 128 | Caffe* | mobilenet-v1-0.25-128 | 40.54%/65% | 0.028 | 0.468 | [1,128,128,3] | [1,1001] |
| MobileNet V1 1.0 224 | Caffe* | mobilenet-v1-1.0-224 | 69.496%/89.224% | 1.148 | 4.221 | [1,3,224,224] | [1,1000] |
| MobileNet V1 1.0 224 | TensorFlow* | mobilenet-v1-1.0-224-tf | 71.03%/89.94% | 1.148 | 4.221 | [1,224,224,3] | [1,1001] |
| MobileNet V2 1.0 224 | Caffe* | mobilenet-v2 | 71.218%/90.178% | 0.876 | 3.489 | [1,3,224,224] | [1,1000] |
| MobileNet V2 1.0 224 | TensorFlow* | mobilenet-v2-1.0-224 | 71.85%/90.69% | 0.615 | 3.489 | [1,224,224,3] | [1,1001] |
| MobileNet V2 1.0 224 | PyTorch* | mobilenet-v2-pytorch | 71.81%/90.396% | 0.615 | 3.489 | [1,3,224,224] | [1,1000] |
| MobileNet V2 1.4 224 | TensorFlow* | mobilenet-v2-1.4-224 | 74.09%/91.97% | 1.183 | 6.087 | [1,224,224,3] | [1,1001] |
| MobileNet V3 Small 1.0 | TensorFlow* | mobilenet-v3-small-1.0-224-tf | 67.36%/87.44% | 0.1168 | 2.537 | [1,224,224,3] | [1,1000] |
| MobileNet V3 Small 1.0 | Paddle* | mobilenet-v3-small-1.0-paddle | 68.21%/88.04% | 0.1269 | 2.9339 | [1,3,224,224] | [1,1000] |
| MobileNet V3 Large 1.0 | TensorFlow* | mobilenet-v3-large-1.0-224-tf | 75.30%/92.62% | 0.445 | 5.4721 | [1,224,224,3] | [1,1000] |
| MobileNet V3 Large 1.0 | Paddle* | mobilenet-v3-large-1.0-224-paddle | 75.248%/92.32% | 0.4565 | 5.468 | [1,3,224,224] | [1,1000] |
| NFNet F0 | PyTorch* | nfnet-f0 | 83.34%/96.56% | 24.8053 | 71.4444 | [1,3,256,256] | [1,1000] |
| RegNetX-3.2GF | PyTorch* | regnetx-3.2gf | 78.17%/94.08% | 6.3893 | 15.2653 | [1,3,224,224] | [1,1000] |
| ResNet 26, alpha=0.25 | MXNet* | octave-resnet-26-0.25 | 76.076%/92.584% | 3.768 | 15.99 | [1,3,224,224] | [1,1000] |
| open-closed-eye-0001 | PyTorch* | open-closed-eye-0001 | 95.84% | 0.0014 | 0.0113 | [1,3,32,32] | [1,2,1,1] |
| RepVGG A0 | PyTorch* | repvgg-a0 | 72.40%/90.49% | 2.7286 | 8.3094 | [1,3,224,224] | [1,1000] |
| RepVGG B1 | PyTorch* | repvgg-b1 | 78.37%/94.09% | 23.6472 | 51.8295 | [1,3,224,224] | [1,1000] |
| RepVGG B3 | PyTorch* | repvgg-b3 | 80.50%/95.25% | 52.4407 | 110.9609 | [1,3,224,224] | [1,1000] |
| ResNeSt 50 | PyTorch* | resnest-50-pytorch | 81.11%/95.36% | 10.8148 | 27.4493 | [1,3,224,224] | [1,1000] |
| ResNet 18 | PyTorch* | resnet-18-pytorch | 69.754%/89.088% | 3.637 | 11.68 | [1,3,224,224] | [1,1000] |
| ResNet 34 | PyTorch* | resnet-34-pytorch | 73.30%/91.42% | 7.3409 | 21.7892 | [1,3,224,224] | [1,1000] |
| ResNet 50 | TensorFlow* | resnet-50-tf | 76.17%/92.98% | 8.216 | 25.53 | [1,224,224,3] | [1,1001] |
| ResNet 50 | PyTorch* | resnet-50-pytorch | 76.128%/92.858% | 8.216 | 25.53 | [1,3,224,224] | [1,1000] |
| ReXNet V1 x1.0 | PyTorch* | rexnet-v1-x1.0 | 77.86%/93.87% | 0.8325 | 4.7779 | [1,3,224,224] | [1,1000] |
| SE-Inception | Caffe* | se-inception | 75.996%/92.964% | 4.091 | 11.922 | [1,3,224,224] | [1,1000] |
| SE-ResNet 50 | Caffe* | se-resnet-50 | 77.596%/93.85% | 7.775 | 28.061 | [1,3,224,224] | [1,1000] |
| SE-ResNeXt 50 | Caffe* | se-resnext-50 | 78.968%/94.63% | 8.533 | 27.526 | [1,3,224,224] | [1,1000] |
| Shufflenet V2 x0.5 | Caffe* | shufflenet-v2-x0.5 | 58.80%/81.13% | 0.08465 | 1.363 | [1,3,224,224] | [1,1000] |
| Shufflenet V2 x1.0 | PyTorch* | shufflenet-v2-x1.0 | 69.36%/88.32% | 0.2957 | 2.2705 | [1,3,224,224] | [1,1000] |
| SqueezeNet v1.0 | Caffe* | squeezenet1.0 | 57.684%/80.38% | 1.737 | 1.248 | [1,3,227,227] | [1,1000] |
| SqueezeNet v1.1 | Caffe* | squeezenet1.1 | 58.382%/81% | 0.785 | 1.236 | [1,3,227,227] | [1,1000] |
| Swin Transformer Tiny | PyTorch* | swin-tiny-patch4-window7-224 | 81.38%/95.51% | 9.028 | 28.8173 | [1,3,224,224] | [1,1000] |
| T2T-ViT | PyTorch* | t2t-vit-14 | 81.44%/95.66% | 9.5451 | 21.5498 | [1,3,224,224] | [1,1000] |
| VGG 16 | Caffe* | vgg16 | 70.968%/89.878% | 30.974 | 138.358 | [1,3,224,224] | [1,1000] |
| VGG 19 | Caffe* | vgg19 | 71.062%/89.832% | 39.3 | 143.667 | [1,3,224,224] | [1,1000] |

---
OmniXRI 整理製作，歡迎點贊、收藏、訂閱、留言、分享