# HDDL plugin in OpenVINO 2021.3/Ubuntu 20.04.2/Kernel 5.8/TGL

- Table of Content
[ToC]

## BKC

CPU : 11th Gen Intel(R) Core(TM) i7-1185G7E @ 2.80GHz
OS : 20.04.2 LTS (Focal Fossa)
Kernel :5.8.0-48-generic
OpenVINO : openvino_2021.3.394

## BKM

sudo apt update
sudo apt upgrade
tar -xzf l_openvino_toolkit_p_2021.3.394.tgz 
cd l_openvino_toolkit_p_2021.3.394/
sudo -E ./install.sh 
source /opt/intel/openvino_2021/bin/setupvars.sh 
cd /opt/intel/openvino_2021/install_dependencies/
./install_NCS_udev_rules.sh 
sudo apt install curl
sudo -E ./install_NEO_OCL_driver.sh

cd /opt/intel/openvino_2021.3.394/inference_engine/external/hddl/
./install_IVAD_VPU_dependencies.sh 

cd /opt/intel/openvino_2021/deployment_tools/model_downloader
pip3 install -r requirements.in 
python3 downloader.py -o ~/openvino_models --name  person-vehicle-bike-detection-2000
python3 object_detection_sample_ssd.py -m ~/openvino_models/intel/person-vehicle-bike-detection-2000/FP16/person-vehicle-bike-detection-2000.xml -i /opt/intel/openvino_2021/deployment_tools//demo/car_1.bmp -d HDDL

### [logs]

python3 /opt/intel/openvino_2021.3.394/inference_engine/samples/python/object_detection_sample_ssd/object_detection_sample_ssd.py -m /home/openvino/openvino_models/intel/person-vehicle-bike-detection-2000/FP16/person-vehicle-bike-detection-2000.xml -i /opt/intel/openvino_2021/deployment_tools/demo/car_1.bmp -d HDDL
/opt/intel/openvino_2021.3.394/python/python3.8/ngraph/utils/types.py:37: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  (NgraphType.boolean, np.bool),
[ INFO ] Loading Inference Engine
[ INFO ] Loading network:
	/home/openvino/openvino_models/intel/person-vehicle-bike-detection-2000/FP16/person-vehicle-bike-detection-2000.xml
[ INFO ] Device info:
        HDDL
        MKLDNNPlugin version ......... 2.1
        Build ........... 2021.3.0-2787-60059f2c755-releases/2021/3
[ INFO ] File was added: 
[ INFO ]         /opt/intel/openvino_2021/deployment_tools/demo/car_1.bmp
[ WARNING ] Image /opt/intel/openvino_2021/deployment_tools/demo/car_1.bmp is resized from (637, 749) to (256, 256)
[ INFO ] Preparing input blobs
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the device
[13:32:26.0232][10273]I[ClientManager.cpp:159] client(id:2) registered: clientName=HDDLPlugin socket=2
[13:32:26.4558][10274]I[GraphManager.cpp:491] Load graph success, graphId=1 graphName=person-vehicle-bike-detection-2000
[ INFO ] Creating infer request and starting inference
[ INFO ] Processing output blobs
/opt/intel/openvino_2021.3.394/inference_engine/samples/python/object_detection_sample_ssd/object_detection_sample_ssd.py:171: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  imid = np.int(proposal[0])
[0,0] element, prob = 1.000000    (232,122)-(510,467) batch id : 0 WILL BE PRINTED!
[1,0] element, prob = 0.962402    (633,227)-(746,615) batch id : 0 WILL BE PRINTED!
[ INFO ] Image out.bmp created!
[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
[13:32:26.5096][10273]I[ClientManager.cpp:189] client(id:2) unregistered: clientName=HDDLPlugin socket=2
[13:32:26.5202][10274]I[GraphManager.cpp:539] graph(1) destroyed

### [log of install_IVAD_VPU_dependencies.sh]

Ubuntu20.04
[sudo] password for openvino: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libboost-program-options1.71.0 is already the newest version (1.71.0-6ubuntu6).
libusb-1.0-0 is already the newest version (2:1.0.23-2build1).
libusb-1.0-0 set to manually installed.
libjson-c4 is already the newest version (0.13.1+dfsg-7ubuntu0.3).
libjson-c4 set to manually installed.
libssl1.1 is already the newest version (1.1.1f-1ubuntu2.2).
libssl1.1 set to manually installed.
The following additional packages will be installed:
  udev
The following NEW packages will be installed:
  libelf-dev
The following packages will be upgraded:
  libudev1 udev
2 upgraded, 1 newly installed, 0 to remove and 43 not upgraded.
Need to get 1504 kB of archives.
After this operation, 388 kB of additional disk space will be used.
Get:1 http://tw.archive.ubuntu.com/ubuntu focal-updates/main amd64 udev amd64 245.4-4ubuntu3.5 [1366 kB]
Get:2 http://tw.archive.ubuntu.com/ubuntu focal-updates/main amd64 libudev1 amd64 245.4-4ubuntu3.5 [81.2 kB]
Get:3 http://tw.archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB]
Fetched 1504 kB in 1s (2350 kB/s)    
(Reading database ... 189136 files and directories currently installed.)
Preparing to unpack .../udev_245.4-4ubuntu3.5_amd64.deb ...
Unpacking udev (245.4-4ubuntu3.5) over (245.4-4ubuntu3.4) ...
Preparing to unpack .../libudev1_245.4-4ubuntu3.5_amd64.deb ...
Unpacking libudev1:amd64 (245.4-4ubuntu3.5) over (245.4-4ubuntu3.4) ...
Setting up libudev1:amd64 (245.4-4ubuntu3.5) ...
Selecting previously unselected package libelf-dev:amd64.
(Reading database ... 189136 files and directories currently installed.)
Preparing to unpack .../libelf-dev_0.176-1.1build1_amd64.deb ...
Unpacking libelf-dev:amd64 (0.176-1.1build1) ...
Setting up udev (245.4-4ubuntu3.5) ...
update-initramfs: deferring update (trigger activated)
Setting up libelf-dev:amd64 (0.176-1.1build1) ...
Processing triggers for systemd (245.4-4ubuntu3.4) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for initramfs-tools (0.136ubuntu6.3) ...
update-initramfs: Generating /boot/initrd.img-5.8.0-48-generic
Copying to /etc/udev/rules.d/98-hddlbsl.rules
Running install for component drv_vsc
if [[ "" != "" ]]; then sudo rmmod myd_vsc; fi;
make[1]: Entering directory '/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc'
rm -f *.o *.ko *.order *.mod.c *.symvers .*.cmd
rm -rf .tmp_versions
make[1]: Leaving directory '/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc'
make[1]: Entering directory '/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc'
make -C /lib/modules/`uname -r`/build M=$PWD ccflags-y+=
make[2]: Entering directory '/usr/src/linux-headers-5.8.0-48-generic'
  AR      /opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc/built-in.a
  CC [M]  /opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc/myd_vsc.o
  MODPOST /opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc/Module.symvers
  CC [M]  /opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc/myd_vsc.mod.o
  LD [M]  /opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc/myd_vsc.ko
make[2]: Leaving directory '/usr/src/linux-headers-5.8.0-48-generic'
make[1]: Leaving directory '/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/drivers/drv_vsc'
mkdir -p /lib/modules/5.8.0-48-generic/kernel/drivers/myd/
cp myd_vsc.ko /lib/modules/5.8.0-48-generic/kernel/drivers/myd/myd_vsc.ko
   Driver myd_vsc.ko is installed to /lib/modules/5.8.0-48-generic/kernel/drivers/myd/myd_vsc.ko 
sudo depmod
filename:       /lib/modules/5.8.0-48-generic/kernel/drivers/myd/myd_vsc.ko
license:        GPL
srcversion:     904DA28840EA0B1F5366D9C
alias:          usb:v03E7pF63Bd*dc*dsc*dp*ic*isc*ip*in*
depends:        
retpoline:      Y
name:           myd_vsc
vermagic:       5.8.0-48-generic SMP mod_unload modversions 
mkdir -p /etc/modules-load.d/
   /etc/modules-load.d/myd_vsc.conf is created for auto-load at boot time 
'/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/../97-myriad-usbboot.rules' -> '/etc/udev/rules.d/97-myriad-usbboot.rules'
'/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/etc/udev/rules.d/97-myriad-usbboot.rules' -> '/etc/udev/rules.d/97-myriad-usbboot.rules'
'/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/etc/udev/rules.d/99-hddl-ion.rules' -> '/etc/udev/rules.d/99-hddl-ion.rules'
'/opt/intel/openvino_2021.3.394/deployment_tools/inference_engine/external/hddl/etc/udev/rules.d/99-myriad-vsc.rules' -> '/etc/udev/rules.d/99-myriad-vsc.rules'
=======================================
Install HDDL dependencies sucessful
Please reboot

## Use HDDL plugin on NCS2

Change "abort_if_hw_reset_failed" of
/opt/intel/openvino_2021/deployment_tools/inference_engine/external/hddl/config/hddl_autoboot.config from true to false. Like below.

  "autoboot_settings":
  {
    "work_mode":                   "scan",                                      // mode of monitoring devices, options: {"scan", "hotplug"}
    "startup_wait_timeout":        20000,                                      // wait timeout for autoboot startup loading all firmwares to devices in milliseconds
    "abort_if_hw_reset_failed":    false,                                        // abort if hardware reset of devices failed, e.g. because of lack of SMBus on PCIe support
    "abort_if_device_num_not_met": false,                                       // abort if still not all devices successfully load firmwares after timeout
    "total_device_num":            0                                            // total number of myriad devices to be used. If 0 autoboot will try to find enough devices until startup_wait_timeout expires
  },

# HDDL plugin in OpenVINO 2021.3 & Ubuntu 20.04 container ( Host OS Ubuntu 20.04.2 and Kernel 5.8. CPU is TGL)

Ref : https://github.com/openvinotoolkit/docker_ci/blob/master/install_guide_vpu_hddl.md

#### Download HDDL driver package from

https://storage.openvinotoolkit.org/drivers/vpu/hddl/2021.3/hddl_ubuntu18_1636.tgz

#### Install HDDL driver and configuration files

sudo apt install build-essential
./install_IVAD_VPU_dependencies.sh
ar -xzf hddl_ubuntu18_1636.tgz 
hddl_ubuntu18_1636/
cd hddl/
export HDDL_INSTALL_DIR=`pwd`
./setupvars.sh 
./install_IVAD_VPU_dependencies.s
Ubuntu20.04
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libboost-program-options1.71.0 is already the newest version (1.71.0-6ubuntu6).
libelf-dev is already the newest version (0.176-1.1build1).
libusb-1.0-0 is already the newest version (2:1.0.23-2build1).
libjson-c4 is already the newest version (0.13.1+dfsg-7ubuntu0.3).
libssl1.1 is already the newest version (1.1.1f-1ubuntu2.3).
libudev1 is already the newest version (245.4-4ubuntu3.5).
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
Copying to /etc/udev/rules.d/98-hddlbsl.rules
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Running install for component drv_vsc
if [[ "" != "" ]]; then sudo rmmod myd_vsc; fi;
make[1]: Entering directory '/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc'
rm -f *.o *.ko *.order *.mod.c *.symvers .*.cmd
rm -rf .tmp_versions
make[1]: Leaving directory '/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc'
make[1]: Entering directory '/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc'
make -C /lib/modules/`uname -r`/build M=$PWD ccflags-y+=
make[2]: Entering directory '/usr/src/linux-headers-5.8.0-43-generic'
  AR      /home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc/built-in.a
  CC [M]  /home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc/myd_vsc.o
  MODPOST /home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc/Module.symvers
  CC [M]  /home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc/myd_vsc.mod.o
  LD [M]  /home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc/myd_vsc.ko
make[2]: Leaving directory '/usr/src/linux-headers-5.8.0-43-generic'
make[1]: Leaving directory '/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/drivers/drv_vsc'
mkdir -p /lib/modules/5.8.0-43-generic/kernel/drivers/myd/
cp myd_vsc.ko /lib/modules/5.8.0-43-generic/kernel/drivers/myd/myd_vsc.ko
   Driver myd_vsc.ko is installed to /lib/modules/5.8.0-43-generic/kernel/drivers/myd/myd_vsc.ko 
sudo depmod
filename:       /lib/modules/5.8.0-43-generic/kernel/drivers/myd/myd_vsc.ko
license:        GPL
srcversion:     904DA28840EA0B1F5366D9C
alias:          usb:v03E7pF63Bd*dc*dsc*dp*ic*isc*ip*in*
depends:        
retpoline:      Y
name:           myd_vsc
vermagic:       5.8.0-43-generic SMP mod_unload 
mkdir -p /etc/modules-load.d/
   /etc/modules-load.d/myd_vsc.conf is created for auto-load at boot time 
'/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/../97-myriad-usbboot.rules' -> '/etc/udev/rules.d/97-myriad-usbboot.rules'
'/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/etc/udev/rules.d/97-myriad-usbboot.rules' -> '/etc/udev/rules.d/97-myriad-usbboot.rules'
'/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/etc/udev/rules.d/99-hddl-ion.rules' -> '/etc/udev/rules.d/99-hddl-ion.rules'
'/home/openvino/Downloads/hddl_ubuntu18_1636/hddl/etc/udev/rules.d/99-myriad-vsc.rules' -> '/etc/udev/rules.d/99-myriad-vsc.rules'
=======================================
Install HDDL dependencies sucessful
Please reboot

#### Run docker

docker pull openvino/ubuntu20_data_dev
docker run -it -v ~/Downloads:/mnt --device /dev/dri:/dev/dri --device-cgroup-rule='c 189:* rmw' --device-cgroup-rule='c 180:* rmw' -v /dev:/dev -v /var/tmp:/var/tmp -u root --ipc=host --net host --rm openvino/ubuntu20_data_dev:latest

apt update
apt install libusb-1.0-0 libboost-program-options1.71.0 libssl1.1 libudev1 libjson-c4 libelf-dev

python3 /opt/intel/openvino_2021.3.394/inference_engine/samples/python/object_detection_sample_ssd/object_detection_sample_ssd.py -m /home/openvino/openvino_models/intel/person-vehicle-bike-detection-2000/FP16/person-vehicle-bike-detection-2000.xml -i /opt/intel/openvino_2021/deployment_tools/demo/car_1.bmp -d HDDL

Log when run in container

python3 /opt/intel/openvino/deployment_tools/tools/benchmark_tool/benchmark_app.py -m /mnt/openvino_models/public/yolo-v3-tf/FP16/yolo-v3-tf.xml -t 1 -d HDDL
[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README. 
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         API version............. 2.1.2021.3.0-2787-60059f2c755-releases/2021/3
[ INFO ] Device info
         HDDL
         HDDLPlugin.............. version 2.1
         Build................... 2021.3.0-2787-60059f2c755-releases/2021/3

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Read network took 52.93 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[Step 7/11] Loading the model to the device
[12:08:24.9972][314]I[main.cpp:246] ## HDDL_INSTALL_DIR: /opt/intel/openvino/deployment_tools/inference_engine/external/hddl
[12:08:24.9972][314]I[main.cpp:248] Config file '/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/config/hddl_service.config' has been loaded
[12:08:24.9974][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_service_alive.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:24.9974][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_service_ready.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:24.9975][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_service_failed.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:24.9975][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_start_exit.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:24.9976][314]I[AutobootStarter.cpp:156] Info: No running autoboot process. Start autoboot daemon...
[12:08:25.0003][316]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_autoboot_alive.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:25.0003][316]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_autoboot_ready.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:25.0003][316]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_autoboot_start_exit.mutex owner: user-'no_change', group-'users', mode-'0660'
[12:08:25.0004][316]I[FileHelper.cpp:268] Set file:/tmp/hddl_autoboot_device.map owner: user-'no_change', group-'users', mode-'0660'
[12:08:25.0004][316]I[AutoBoot.cpp:308] [Firmware Config] deviceName=default deviceNum=0 firmwarePath=/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib/mvnc/usb-ma2x8x.mvcmd
[E:] [BSL] found 0 ioexpander device
[E:] [BSL] No device found
[12:08:25.0024][316]W[AutoBoot.cpp:476] Warn: HDDL hardware reset failed, run without hardware reset support.
[12:08:26.0026][319]I[AutoBoot.cpp:197] Start boot device 1.3-ma2480
[12:08:26.2272][319]I[AutoBoot.cpp:199] Device 1.3-ma2480 boot success, firmware=/opt/intel/openvino/deployment_tools/inference_engine/external/hddl/lib/mvnc/usb-ma2x8x.mvcmd
[12:08:46.2287][314]I[AutobootStarter.cpp:85] Info: Autoboot is running.
[12:08:46.2298][314]W[ConfigParser.cpp:269] Warning: Cannot find key, path=scheduler_config.max_graph_per_device subclass=0, use default value: 1.
[12:08:46.2298][314]W[ConfigParser.cpp:291] Warning: Cannot find key, path=scheduler_config.use_sgad_by_default subclass=0, use default value: false.
[12:08:46.2298][314]I[DeviceSchedulerFactory.cpp:56] Info: ## DeviceSchedulerFacotry ## Created Squeeze Device-Scheduler2.
[12:08:46.2299][314]I[DeviceManager.cpp:551] ## SqueezeScheduler created ##
[12:08:46.2299][314]I[DeviceManager.cpp:649] times 0: try to create worker on device(1.3)
[12:08:48.2332][314]I[DeviceManager.cpp:670] [SUCCESS] times 0: create worker on device(1.3)
[12:08:48.2333][314]I[DeviceManager.cpp:719] worker(Wt1.3) created on device(1.3), type(0)
[12:08:48.2333][314]I[DeviceManager.cpp:145] DEVICE FOUND : 1
[12:08:48.2333][314]I[DeviceManager.cpp:146] DEVICE OPENED : 1
[12:08:48.2334][314]I[DeviceManagerCreator.cpp:81] New device manager(DeviceManager0) created with subclass(0), deviceCount(1)
[12:08:48.2337][314]I[TaskSchedulerFactory.cpp:45] Info: ## TaskSchedulerFactory ## Created Polling Task-Scheduler.
[12:08:48.2341][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_snapshot.sock owner: user-'no_change', group-'users', mode-'0660'
[12:08:48.2344][314]I[FileHelper.cpp:268] Set file:/var/tmp/hddl_service.sock owner: user-'no_change', group-'users', mode-'0660'
[12:08:48.2345][314]I[MessageDispatcher.cpp:87] Message Dispatcher initialization finished
[12:08:48.2345][314]I[main.cpp:106] SERVICE IS READY ...
[12:08:48.3053][331]I[ClientManager.cpp:159] client(id:1) registered: clientName=HDDLPlugin socket=2
[12:08:52.5178][332]I[GraphManager.cpp:491] Load graph success, graphId=1 graphName=yolo-v3-tf
[ INFO ] Load network took 27532.42 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'input_1' precision U8, dimensions (NCHW): 1 3 416 416
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'input_1' with random values (image is expected)
[ INFO ] Infer Request 1 filling
[ INFO ] Fill input 'input_1' with random values (image is expected)
[ INFO ] Infer Request 2 filling
[ INFO ] Fill input 'input_1' with random values (image is expected)
[ INFO ] Infer Request 3 filling
[ INFO ] Fill input 'input_1' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 4 inference requests, limits: 1000 ms duration)
[ INFO ] First inference took 514.97 ms
[Step 11/11] Dumping statistics report
Count:      8 iterations
Duration:   1971.92 ms
Latency:    900.22 ms
Throughput: 4.06 FPS
[12:08:55.0439][331]I[ClientManager.cpp:189] client(id:1) unregistered: clientName=HDDLPlugin socket=2
[12:08:55.0611][332]I[GraphManager.cpp:539] graph(1) destroyed