# AI Appliance Performance Qualification Tool ###### tags: `Intel AI Sales Kit` - Table of Content [ToC] ### Step 0. Install required tools and libraries for turbostat in terminal 1 ``` sudo modprobe msr sudo apt install linux-tools-common linux-tools-`uname -r` ``` ### Step 1. Run turbostat in 1st terminal in terminal 1 ``` sudo turbostat -Summary --interval 5 --show Bzy_MHz,GFXMHz,PkgTmp,PkgWatt,CoreTmp,CorWatt,GFXWatt ``` ### Step 2a. Get YOLO v3 FP16-INT8 model in terminal 2 ``` docker pull your_dockerhub_id/openvino:2021.3_developer_models docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -v ~/Downloads:/mnt --device /dev/dri:/dev/dri --group-add=$(stat -c "%g" /dev/dri/render*) --rm your_dockerhub_id/openvino:2021.3_developer_models cp -ar /opt/intel/openvino_models/ /mnt exit ``` ### !!! Please change your_dockerhub_id of "your_dockerhub_id/openvino:2021.3_developer_models" in the command line to one of following (sertek, synnexgrp, wt1com, wpig) !!! ### Step 2b. Run benchmark_app in tgl container in terminal 2 ``` docker pull openvino/ubuntu20_data_dev:2021.4_tgl docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -v ~/Downloads:/mnt --device /dev/dri:/dev/dri --group-add=$(stat -c "%g" /dev/dri/render*) --rm openvino/ubuntu20_data_dev:2021.4_tgl python3 /opt/intel/openvino_2021.4.582/deployment_tools/tools/benchmark_tool/benchmark_app.py -m /mnt/openvino_models/public/yolo-v3-tf/FP16-INT8/yolo-v3-tf.xml -d CPU python3 /opt/intel/openvino_2021.4.582/deployment_tools/tools/benchmark_tool/benchmark_app.py -m /mnt/openvino_models/public/yolo-v3-tf/FP16/yolo-v3-tf.xml -d CPU python3 /opt/intel/openvino_2021.4.582/deployment_tools/tools/benchmark_tool/benchmark_app.py -m /mnt/openvino_models/public/yolo-v3-tf/FP16-INT8/yolo-v3-tf.xml -d GPU python3 /opt/intel/openvino_2021.4.582/deployment_tools/tools/benchmark_tool/benchmark_app.py -m /mnt/openvino_models/public/yolo-v3-tf/FP16/yolo-v3-tf.xml -d GPU ``` ### Step 3. Make sure yolo-v3-tf (FP16-INT8) on GPU achieve 59 (±5%) FPS (i7-1185G7E) I7-1185G7E : 59 FPS (±5%), I5-1145G7E : 47 FPS (±5%) , I3-1115G4E : 28 FPS (±5%) , Celeron 6305E : 28 FPS (±5%) ### Step 4. If FPS doesn't meet, Check if PkgWatt is around 28/40 W (TGL NUC or 28 W for others accoding to CPU/System specification). ![](https://i.imgur.com/7KnUb9T.png) ### Step 5. If PkgWatt doesn't meet, Check Reference BIOS Setting for Best CPU & GPU Performance ![](https://i.imgur.com/FNiMGnD.png) ## Install OS and tool (skip this if your system have Ubuntu 20.04.2 installed) #### Download and Install Ubuntu 20.04.2.0 LTS URL to donwload Ubuntu 20.04.2.0 LTS ISO image: https://ubuntu.com/download/desktop/thank-you?version=20.04.2.0&architecture=amd64 #### Install Ubuntu 20.04 https://phoenixnap.com/kb/install-ubuntu-20-04 #### Install Docker Utility ``` sudo apt update sudo apt-get remove docker docker-engine docker.io containerd runc sudo apt install curl curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh sudo usermod -aG docker $USER ## Need to logout or reboot to run docker as non-root user docker run hello-world ``` If you see error messages below, reboot your system. ``` openvino@openvino-TGL:~$ docker run hello-world docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create: dial unix /var/run/docker.sock: connect: permission denied. See 'docker run --help'. ``` #### Use POT to Quantize yolo-v3-tf Public Model https://hackmd.io/lKCVqs5xQ3WSFvl33wLnjQ #### Use POT to Quantize yolo-v4-tf Public Model https://hackmd.io/mQHIp8vgTViF9Ao4mnI5rw