Try   HackMD

如何運行Intel OpenVINO Open Model Zoo(OMZ)範例於Google Colab上

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Intel OpenVINO[1] 從2018年5月發行至今已更新20多個版次,目前最新版次為 2023.3 LTS。期間有很多重大革新,這也讓很多不錯的功能可能消失或以其它方式出現。如2021.4版後開始提供Jupyter Notebook (Python) 格式範例讓大家更容易學習。2023.0版後開始提供 Google Colab 格式的範例,讓大家免本地安裝可直接在雲端體驗 OpenVINO。不過也有像 Open Model Zoo (OMZ)[2] 之類方便學習、測試各種預訓練模型的項目及相關工具被移出。主要是因為自2023.1版後開始支援直接讀取更多框架模型及神經網路壓縮架構(Neural Network Compression Framework, NNCF)取代原有模型優化器(Model Optimizer, MO)。

目前 OpenVINO Notebooks[3] 範例編號101~128提供了各種常見框架轉成OpenVINO專屬 IR(XML+BIN) 格式的範例,如TensorFlow, PyTorch, PaddlePaddel 等,甚至 Hugging Face 格式。當然這裡也包含了OMZ的轉換範例(104-model-tools)。不過這個範例比較適合單獨運行、預先轉檔工作,當想像以往 OMZ Demos 下的應用範例,要整合轉檔加執行時,就會遇到一些小麻煩。

雖然這些OMZ的範例已被淡出,但如果大家想直接在 Google Colab 上直接運行時,這篇文章應該可以給大家一些幫助。以下就以OMZ 影像分類(Classification) Public Pre-Trained Models[4]為例,說明如何以 Colab 完成直接運行。

OpenVINO Open Model Zoo (OMZ) & Tools

Intel OpenVINO 自20018發行後 Open Model Zoo (OMZ) 一直提供很豐富的預訓練模型庫,很方便進行AI應用的實驗,其中包含三大部份。

  • Intel's Pre-Trained
    Intel 提供之預設訓練模型,包括物件偵測、物件辨識、影像分割、姿態偵測、行為分析、語音處理、時間預測等數十種模型。此部份已直接轉成IR格式(Bin + Xml),可直接載入運行。
  • Public Pre-Trained
    常用公開預訓練模型,有數十種應用及數百個模型,下載後需依原始框架格式(如TensorFlow, PyTorch, Caffe等)轉換到IR格式才能載入運行。
  • Demos
    提供整合型範例,使用單一或組合不同模型完成特定功能,搭配簡易顯示畫面,更方便使用者理解。

另外亦提供OMZ相關工具,方便使用者處理模型轉檔、優化及效能評估,主要包括下列四項工具。

Tool Command Description
Model Downloader omz_downloader 從Open Model Zoo下載模型
Model Converter omz_converter 將模型轉換成 OpenVINO IR 格式
Info Dumper omz_info_dumper 列印模型完整資訊
Benchmark Tool benchmark_app 比較模型推論效能

隨著 OpenVINO 版本的迭代更新,在2023.1版後, OMZ[4] 及相關工具[5]在官方說明文件中就移到「舊版功能(Legacy Features)」,不再持續維護。使用 pip install openvino 也不再預安裝OMZ相關工具,需改用 pip install openvino-dev 才能取得。

舊版 Open Model Zoo 相關文件及範例仍於開源於 Github[6] 上,有需要可行下載研究。另外原始模型及說明文件亦可從「官方共用區」[7]中取得。

OpenVINO Notebooks

為了讓大家更容易使用 OpenVINO ,自2021.4版後就開始提供 Jupyter Notebook 格式的範例程式庫 Notebooks ,主要分為五大部份。

  • 第一步(First steps)(編號001~099)
  • 轉換與優化(Covert & Optimize)(編號100~199)
  • 模型展示(Model Demos)(編號200~299)
  • 模型訓練(Model Training)(編號300~399)
  • 即時展示(Live Demos)(編號400~499)

2022.1版後部份範例已支援直接在 Binder 線上運行。 2023.0版後除增加更多範例外,亦有部份範例支援直接在 Google Colab 環境下安裝 OpenVINO 及運行。不過目前 Notebooks 範例中並未納入原先 OMZ Pri-Trained Model 及 Demos 範例,如有需要可參考下列作法。

OMZ Classification Public Pre-Trained Models

目前 OMZ Public Pre-Trained Models提供了常見的預訓練模型有數十種應用及數百個模型,單就影像分類來看就有超過五十種,詳如文末附錄。

以往可透過Demos下的「Classification Python* Demo」[8]在命令列運行,來呼叫這五十多種影像分類模型。當它利用 omz_downloader 下載模型時會一併把對應的參數設定好。可是這樣的作法並無法適用在 Colab 環境下,所以這裡可簡單把工作流程改成下列步驟,就能完整運行影像分類。

  1. 安裝openvino-dev
  2. 導入必要模組
  3. 下載預訓練模型
  4. 模型轉換
  5. 指定推論裝置
  6. 將模型載入裝置並編譯
  7. 下載測試資料

更完整的細部程式說明可直接點擊下列連結,進入Colab直接運行。大家可試著修改模型名稱及指定對應設定來實驗一下。
Colab完整範例

小結

這個範例僅示範了OMZ影像分類用法,這只是一個開始,大家可試著延伸到其它類型範例,後續會持續安排將Demos下的範例轉至Colab,讓大家更方便學習。

參考文獻

[1] Intel, OpenVINO Toolkit Document
https://docs.openvino.ai/

[2] Intel, OpenVINO Document - Documentation - Legacy Features - Open Model Zoo (2023.3 LTS)
https://docs.openvino.ai/2023.3/model_zoo.html

[3] Intel, Github - openvinotoolkit / openvino_notebooks
https://github.com/openvinotoolkit/openvino_notebooks

[4] Intel, OpenVINO Document - Documentation - Legacy Features - Open Model Zoo - Public Pre-Training Models (2023.3 LTS)
https://docs.openvino.ai/2023.3/omz_models_group_public.html#classification-models

[5] Intel, OpenVINO Document - Documentation - Legacy Features - Model Optimizer / Conversion API (2023.3 LTS)
https://docs.openvino.ai/2023.3/openvino_docs_OV_Converter_UG_prepare_model_convert_model_MO_OVC_transition.html

[6] Intel, Github - openvinotoolkit / open_model_zoo
https://github.com/openvinotoolkit/open_model_zoo

[7] Intel, stroage for open model zoo
https://storage.openvinotoolkit.org/repositories/open_model_zoo/

[8] Intel, OpenVINO Document - Classification Python* Demo
https://docs.openvino.ai/2023.3/omz_demos_classification_demo_python.html

附錄 - OMZ 影像分類模型清單

[資料來源], OmniXRI 整理製作

Model Name Implementation OMZ Model Name Accuracy (Top1/5) GFlops MParams INPUT OUTPUT
AlexNet Caffe* alexnet 56.598%/79.812% 1.5 60.965 [1, 3, 227, 227] [1,1000]
AntiSpoofNet PyTorch* anti-spoof-mn3 3.81% 0.15 3.02 [1, 3, 128, 128] [1,2]
CaffeNet Caffe* caffenet 56.714%/79.916% 1.463 60.965 [1, 3, 227, 227] [1,1000]
ConvNeXt Tiny PyTorch* convnext-tiny 82.05%/95.86% 8.9419 28.5892 [1,3,224,224] [1,1000]
DenseNet 121 Caffe* densenet-121 74.42%/92.136% 5.274 7.971 [1,3,224,224] [1,1000]
DenseNet 121 TensorFlow* densenet-121-tf 74.46%/92.13% 5.7287 7.971 [1,224,224,3] [1,1000]
DLA 34 PyTorch* dla-34 74.64%/92.06% 6.1368 15.7344 [1,3,224,224] [1,1000]
EfficientNet B0 TensorFlow* efficientnet-b0 75.70%/92.76% 0.819 5.268 [1,224,224,3] [1,1000]
EfficientNet B0 PyTorch* efficientnet-b0-pytorch 77.70%/93.52% 0.819 5.268 [1,3,224,224] [1,1000]
EfficientNet V2 B0 PyTorch* efficientnet-v2-b0 78.36%/94.02% 1.4641 7.1094 [1,3,224,224] [1,1000]
EfficientNet V2 Small PyTorch* efficientnet-v2-s 84.29%/97.26% 16.9406 21.3816 [1,3,224,224] [1,1000]
HBONet 1.0 PyTorch* hbonet-1.0 73.1%/91.0% 0.6208 4.5443 [1,3,224,224] [1,1000]
HBONet 0.25 PyTorch* hbonet-0.25 57.3%/79.8% 0.0758 1.9299 [1,3,224,224] [1,1000]
Inception (GoogleNet) V1 Caffe* googlenet-v 68.928%/89.144% 3.016 6.619 [1,3,224,224] [1,1000]
Inception (GoogleNet) V1 TensorFlow* googlenet-v1-tf 69.814%/89.6% 3.266 6.999 [1,224,224,3] [1,1001]
Inception (GoogleNet) V2 Caffe* googlenet-v2 72.024%/90.844% 4.058 11.185 [1,3,224,224] [1,1000]
Inception (GoogleNet) V2 TensorFlow* googlenet-v2-tf 74.084%/91.798% 4.058 11.185 [1,224,224,3] [1,1001]
Inception (GoogleNet) V3 TensorFlow* googlenet-v3 77.904%/93.808% 11.469 23.817 [1,299,299,3] [1,1001]
Inception (GoogleNet) V3 PyTorch* googlenet-v3-pytorch 77.69%/93.7% 11.469 23.817 [1,3,299,299] [1,1000]
Inception (GoogleNet) V4 TensorFlow* googlenet-v4-tf 80.204%/95.21% 24.584 42.648 [1,299,299,3] [1,1001]
Inception-ResNet V2 TensorFlow* inception-resnet-v2-tf 77.82%/94.03% 22.227 30.223 [1,299,299,3] [1,1001]
LeViT 128S PyTorch* levit-128s 76.54%/92.85% 0.6177 8.2199 [1,3,224,224] [1,1000]
MixNet L TensorFlow* mixnet-l 78.30%/93.91% 0.565 7.3 [1,224,224,3] [1,1000]
MobileNet V1 0.25 128 Caffe* mobilenet-v1-0.25-128 40.54%/65% 0.028 0.468 [1,128,128,3] [1,1001]
MobileNet V1 1.0 224 Caffe* mobilenet-v1-1.0-224 69.496%/89.224% 1.148 4.221 [1,3,224,224] [1,1000]
MobileNet V1 1.0 224 TensorFlow* mobilenet-v1-1.0-224-tf 71.03%/89.94% 1.148 4.221 [1,224,224,3] [1,1001]
MobileNet V2 1.0 224 Caffe* mobilenet-v2 71.218%/90.178% 0.876 3.489 [1,3,224,224] [1,1000]
MobileNet V2 1.0 224 TensorFlow* mobilenet-v2-1.0-224 71.85%/90.69% 0.615 3.489 [1,224,224,3] [1,1001]
MobileNet V2 1.0 224 PyTorch* mobilenet-v2-pytorch 71.81%/90.396% 0.615 3.489 [1,3,224,224] [1,1000]
MobileNet V2 1.4 224 TensorFlow* mobilenet-v2-1.4-224 74.09%/91.97% 1.183 6.087 [1,224,224,3] [1,1001]
MobileNet V3 Small 1.0 TensorFlow* mobilenet-v3-small-1.0-224-tf 67.36%/87.44% 0.1168 2.537 [1,224,224,3] [1,1000]
MobileNet V3 Small 1.0 Paddle* mobilenet-v3-small-1.0-paddle 68.21%/88.04% 0.1269 2.9339 [1,3,224,224] [1,1000]
MobileNet V3 Large 1.0 TensorFlow* mobilenet-v3-large-1.0-224-tf 75.30%/92.62% 0.445 5.4721 [1,224,224,3] [1,1000]
MobileNet V3 Large 1.0 Paddle* mobilenet-v3-large-1.0-224-paddle 75.248%/92.32% 0.4565 5.468 [1,3,224,224] [1,1000]
NFNet F0 PyTorch* nfnet-f0 83.34%/96.56% 24.8053 71.4444 [1,3,256,256] [1,1000]
RegNetX-3.2GF PyTorch* regnetx-3.2gf 78.17%/94.08% 6.3893 15.2653 [1,3,224,224] [1,1000]
ResNet 26, alpha=0.25 MXNet* octave-resnet-26-0.25 76.076%/92.584% 3.768 15.99 [1,3,224,224] [1,1000]
open-closed-eye-0001 PyTorch* open-closed-eye-0001 95.84% 0.0014 0.0113 [1,3,32,32] [1,2,1,1]
RepVGG A0 PyTorch* repvgg-a0 72.40%/90.49% 2.7286 8.3094 [1,3,224,224] [1,1000]
RepVGG B1 PyTorch* repvgg-b1 78.37%/94.09% 23.6472 51.8295 [1,3,224,224] [1,1000]
RepVGG B3 PyTorch* repvgg-b3 80.50%/95.25% 52.4407 110.9609 [1,3,224,224] [1,1000]
ResNeSt 50 PyTorch* resnest-50-pytorch 81.11%/95.36% 10.8148 27.4493 [1,3,224,224] [1,1000]
ResNet 18 PyTorch* resnet-18-pytorch 69.754%/89.088% 3.637 11.68 [1,3,224,224] [1,1000]
ResNet 34 PyTorch* resnet-34-pytorch 73.30%/91.42% 7.3409 21.7892 [1,3,224,224] [1,1000]
ResNet 50 TensorFlow* resnet-50-tf 76.17%/92.98% 8.216 25.53 [1,224,224,3] [1,1001]
ResNet 50 PyTorch* resnet-50-pytorch 76.128%/92.858% 8.216 25.53 [1,3,224,224] [1,1000]
ReXNet V1 x1.0 PyTorch* rexnet-v1-x1.0 77.86%/93.87% 0.8325 4.7779 [1,3,224,224] [1,1000]
SE-Inception Caffe* se-inception 75.996%/92.964% 4.091 11.922 [1,3,224,224] [1,1000]
SE-ResNet 50 Caffe* se-resnet-50 77.596%/93.85% 7.775 28.061 [1,3,224,224] [1,1000]
SE-ResNeXt 50 Caffe* se-resnext-50 78.968%/94.63% 8.533 27.526 [1,3,224,224] [1,1000]
Shufflenet V2 x0.5 Caffe* shufflenet-v2-x0.5 58.80%/81.13% 0.08465 1.363 [1,3,224,224] [1,1000]
Shufflenet V2 x1.0 PyTorch* shufflenet-v2-x1.0 69.36%/88.32% 0.2957 2.2705 [1,3,224,224] [1,1000]
SqueezeNet v1.0 Caffe* squeezenet1.0 57.684%/80.38% 1.737 1.248 [1,3,227,227] [1,1000]
SqueezeNet v1.1 Caffe* squeezenet1.1 58.382%/81% 0.785 1.236 [1,3,227,227] [1,1000]
Swin Transformer Tiny PyTorch* swin-tiny-patch4-window7-224 81.38%/95.51% 9.028 28.8173 [1,3,224,224] [1,1000]
T2T-ViT PyTorch* t2t-vit-14 81.44%/95.66% 9.5451 21.5498 [1,3,224,224] [1,1000]
VGG 16 Caffe* vgg16 70.968%/89.878% 30.974 138.358 [1,3,224,224] [1,1000]
VGG 19 Caffe* vgg19 71.062%/89.832% 39.3 143.667 [1,3,224,224] [1,1000]

OmniXRI 整理製作,歡迎點贊、收藏、訂閱、留言、分享