###### tags: `Tutorial`
# Greengrass + Openvino on Ubuntu
## 全新安裝
參考此說明修改 : https://software.intel.com/en-us/articles/get-started-with-the-openvino-toolkit-and-aws-greengrass
1. 安裝openvino(參考intel網站)
2. 轉換IR
3. 將IR壓縮成zip上傳到S3上
4. 設定AWS Greengrass
5. 啟動AWS Greengrass
6. 將位於
`/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/inference_engine/samples/python_samples/greengrass_samples` 下的 `greengrass_object_detection_sample_ssd.py`與greengrasssdk合併打包成zip檔上傳到Lambda上
7. 修改`greengrass_object_detection_sample_ssd.py`檔案 :
* 將
```python
n, c, h, w = net.inputs[input_blob]
```
修改為
```python
n, c, h, w = net.inputs[input_blob].shape
```
* 新增以下程式碼位於`frame_count += 1`之後
```python
cv2.imshow('Greengrass_Openvino', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
```
* 新增文字訊息
將
```python
cv2.putText(frame, ("label:"+ str(int(obj[1]))+", class:"+ str(classlabel)+ ", confidence"+ str(round(obj[2], 2))), (xmin, ymin - 10), cv2.FONT_HERSHEY_DUPLEX, 0.6, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, ("label:"+ str(int(obj[1]))+", class:"+ str(classlabel)+ ", confidence"+ str(round(obj[2], 2))), (xmin, ymin - 10), cv2.FONT_HERSHEY_DUPLEX, 0.6, (0, 72, 97), 1, cv2.LINE_AA)
```
插入在`object_id += 1` 之後
然後將
```python
cv2.putText(frame, ("inference_fps:"+ str(inference_fps)), (int(20), int(20)), cv2.FONT_HERSHEY_DUPLEX, 0.6, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, ("inference_fps:"+ str(inference_fps)), (int(20), int(20)), cv2.FONT_HERSHEY_DUPLEX, 0.6, (1, 66, 0), 1, cv2.LINE_AA)
```
插入在`frame_count += 1` 與 `cv2.imshow('Greengrass_Openvino', frame)` 之間
然後將
```python
inference_fps = 0
```
插入在`while (cap.isOpened()):` 之前
然後將
```python
inference_fps = res_json["inference_fps"]
```
插入在`res_json["inference_fps"] = frame_count / inf_seconds` 之後
* 完整Lambda Function :
```python
"""
BSD 3-clause "New" or "Revised" license
Copyright (C) 2018 Intel Coporation.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
import sys
import os
import cv2
import numpy as np
import greengrasssdk
import boto3
import timeit
import datetime
import json
from collections import OrderedDict
from openvino.inference_engine import IENetwork, IEPlugin
# Specify the delta in seconds between each report
reporting_interval = 1.0
# Parameters for IoT Cloud
enable_iot_cloud_output = True
# Parameters for Kinesis
enable_kinesis_output = False
kinesis_stream_name = ""
kinesis_partition_key = ""
kinesis_region = ""
# Parameters for S3
enable_s3_jpeg_output = False
s3_bucket_name = "ssd_test"
# Parameters for jpeg output on local disk
enable_local_jpeg_output = False
# Create a Greengrass Core SDK client for publishing messages to AWS Cloud
client = greengrasssdk.client("iot-data")
# Create an S3 client for uploading files to S3
if enable_s3_jpeg_output:
s3_client = boto3.client("s3")
# Create a Kinesis client for putting records to streams
if enable_kinesis_output:
kinesis_client = boto3.client("kinesis", "us-west-2")
# Read environment variables set by Lambda function configuration
PARAM_MODEL_XML = os.environ.get("PARAM_MODEL_XML")
PARAM_INPUT_SOURCE = os.environ.get("PARAM_INPUT_SOURCE")
PARAM_DEVICE = os.environ.get("PARAM_DEVICE")
PARAM_OUTPUT_DIRECTORY = os.environ.get("PARAM_OUTPUT_DIRECTORY")
PARAM_CPU_EXTENSION_PATH = os.environ.get("PARAM_CPU_EXTENSION_PATH")
PARAM_LABELMAP_FILE = os.environ.get("PARAM_LABELMAP_FILE")
PARAM_TOPIC_NAME = os.environ.get("PARAM_TOPIC_NAME", "intel/faas/ssd")
def report(res_json, frame):
now = datetime.datetime.now()
date_prefix = str(now).replace(" ", "_")
if enable_iot_cloud_output:
data = json.dumps(res_json)
client.publish(topic=PARAM_TOPIC_NAME, payload=data)
if enable_kinesis_output:
kinesis_client.put_record(StreamName=kinesis_stream_name, Data=json.dumps(res_json), PartitionKey=kinesis_partition_key)
if enable_s3_jpeg_output:
temp_image = os.path.join(PARAM_OUTPUT_DIRECTORY, "inference_result.jpeg")
cv2.imwrite(temp_image, frame)
with open(temp_image) as file:
image_contents = file.read()
s3_client.put_object(Body=image_contents, Bucket=s3_bucket_name, Key=date_prefix + ".jpeg")
if enable_local_jpeg_output:
cv2.imwrite(os.path.join(PARAM_OUTPUT_DIRECTORY, date_prefix + ".jpeg"), frame)
def greengrass_object_detection_sample_ssd_run():
client.publish(topic=PARAM_TOPIC_NAME, payload="OpenVINO: Initializing...")
model_bin = os.path.splitext(PARAM_MODEL_XML)[0] + ".bin"
# Plugin initialization for specified device and load extensions library if specified
plugin = IEPlugin(device=PARAM_DEVICE, plugin_dirs="")
if "CPU" in PARAM_DEVICE:
plugin.add_cpu_extension(PARAM_CPU_EXTENSION_PATH)
# Read IR
net = IENetwork(model=PARAM_MODEL_XML, weights=model_bin)
assert len(net.inputs.keys()) == 1, "Sample supports only single input topologies"
assert len(net.outputs) == 1, "Sample supports only single output topologies"
input_blob = next(iter(net.inputs))
out_blob = next(iter(net.outputs))
# Read and pre-process input image
n, c, h, w = net.inputs[input_blob].shape
cap = cv2.VideoCapture(PARAM_INPUT_SOURCE)
exec_net = plugin.load(network=net)
del net
client.publish(topic=PARAM_TOPIC_NAME, payload="Starting inference on %s" % PARAM_INPUT_SOURCE)
start_time = timeit.default_timer()
inf_seconds = 0.0
frame_count = 0
labeldata = None
if PARAM_LABELMAP_FILE is not None:
with open(PARAM_LABELMAP_FILE) as labelmap_file:
labeldata = json.load(labelmap_file)
inference_fps = 0
while (cap.isOpened()):
ret, frame = cap.read()
if not ret:
break
frameid = cap.get(cv2.CAP_PROP_POS_FRAMES)
initial_w = cap.get(3)
initial_h = cap.get(4)
in_frame = cv2.resize(frame, (w, h))
in_frame = in_frame.transpose((2, 0, 1)) # Change data layout from HWC to CHW
in_frame = in_frame.reshape((n, c, h, w))
# Start synchronous inference
inf_start_time = timeit.default_timer()
res = exec_net.infer(inputs={input_blob: in_frame})
inf_seconds += timeit.default_timer() - inf_start_time
# Parse detection results of the current request
res_json = OrderedDict()
frame_timestamp = datetime.datetime.now()
object_id = 0
for obj in res[out_blob][0][0]:
if obj[2] > 0.5:
xmin = int(obj[3] * initial_w)
ymin = int(obj[4] * initial_h)
xmax = int(obj[5] * initial_w)
ymax = int(obj[6] * initial_h)
cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), (255, 165, 20), 4)
obj_id = "Object" + str(object_id)
classlabel = labeldata[str(int(obj[1]))] if labeldata else ""
res_json[obj_id] = {"label": int(obj[1]), "class": classlabel, "confidence": round(obj[2], 2), "xmin": round(obj[3], 2), "ymin": round(obj[4], 2), "xmax": round(obj[5], 2), "ymax": round(obj[6], 2)}
object_id += 1
cv2.putText(frame, ("label:"+ str(int(obj[1]))+", class:"+ str(classlabel)+ ", confidence"+ str(round(obj[2], 2))), (xmin, ymin - 10), cv2.FONT_HERSHEY_DUPLEX, 0.6, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, ("label:"+ str(int(obj[1]))+", class:"+ str(classlabel)+ ", confidence"+ str(round(obj[2], 2))), (xmin, ymin - 10), cv2.FONT_HERSHEY_DUPLEX, 0.6, (0, 72, 97), 1, cv2.LINE_AA)
frame_count += 1
cv2.putText(frame, ("inference_fps:"+ str(inference_fps)), (int(20), int(20)), cv2.FONT_HERSHEY_DUPLEX, 0.6, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, ("inference_fps:"+ str(inference_fps)), (int(20), int(20)), cv2.FONT_HERSHEY_DUPLEX, 0.6, (1, 66, 0), 1, cv2.LINE_AA)
cv2.imshow('Greengrass_Openvino', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# Measure elapsed seconds since the last report
seconds_elapsed = timeit.default_timer() - start_time
if seconds_elapsed >= reporting_interval:
res_json["timestamp"] = frame_timestamp.isoformat()
res_json["frame_id"] = int(frameid)
res_json["inference_fps"] = frame_count / inf_seconds
inference_fps = res_json["inference_fps"]
start_time = timeit.default_timer()
report(res_json, frame)
frame_count = 0
inf_seconds = 0.0
client.publish(topic=PARAM_TOPIC_NAME, payload="End of the input, exiting...")
del exec_net
del plugin
greengrass_object_detection_sample_ssd_run()
def function_handler(event, context):
client.publish(topic=PARAM_TOPIC_NAME, payload='HANDLER_CALLED!')
return
```
8. 參照AWS GG使用手冊發布並與Greengress 群組連接
9. 設定AWS GG Lambda:
* 執行身分與Lambda 生命週期:
* 環境變數
* | Key | Value |
| -------- | -------- |
| LD_LIBRARY_PATH | /opt/intel/computer_vision_sdk/opencv/share/OpenCV/3rdparty/lib:/opt/intel/computer_vision_sdk/opencv/lib:/opt/intel/opencl:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/external/cldnn/lib:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/external/mkltiny_lnx/lib:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/lib/ubuntu_16.04/intel64:/opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/model_optimizer_caffe/bin:/opt/intel/computer_vision_sdk/openvx/lib |
| PYTHONPATH | /opt/intel/computer_vision_sdk/python/python2.7/ubuntu16/ |
| PARAM_CPU_EXTENSION_PATH | /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/lib/ubuntu_16.04/intel64/libcpu_extension_avx2.so |
| PARAM_MODEL_XML | /aimodel/SqueezeNetSSD-5Class.xml |
| PARAM_INPUT_SOURCE | /dev/video0 |
| PARAM_DEVICE | CPU |
| PARAM_OUTPUT_DIRECTORY | /home/iei/dl/img |
| PARAM_NUM_TOP_RESULTS | 3 |
| PARAM_TOPIC_NAME | intel/faas/ssd |
| DISPLAY | unix:0 |
| XAUTHORITY | /X11A/.Xauthority |
* 設定資源
* Name | Resource Type | Local Path | Access
------------- |------------- | ------------- | -------------
Webcam | Device | `/dev/video0` | Read and Write
DataDir | Volume | `/opt/intel/computer_vision_sdk/` | Read and Write
OpenVINOPath | Volume | `/opt/intel/computer_vision_sdk/` | Read-Only
GPU | Device | `/dev/dri/renderD128` | Read and Write
X11 | Volume | `/tmp/.X11-unix/` | Read and Write
X11A | Volume | Source : `/home/iei/.Xauthority` Destination : `/X11A/.Xauthority` | Read and Write
* 設定 Machine Learning Resour
* 選擇上傳的IR .zip檔案
* Path設定為 : `/aimodel/`
* 與[Lambda Function]綁定
* Read-Only
10. 設定Subscribe
* **Source** : [Lambda Function]
* **Destination** : IoT Cloud
* **Topic** : intel/faas/ssd
11. Depory
12. 開機自啟動，參閱 : https://docs.aws.amazon.com/zh_tw/greengrass/latest/developerguide/gg-core.html#start-on-boot
## 已安裝完啟動步驟
### Host 步驟
* Username/Password : iei/123
* 正常開機會自啟動
* 若沒自啟動輸入以下command :
```bash
sudo systemctl restart greengrass.service
```
* 查閱啟動狀況輸入以下command :
```bash
sudo systemctl status greengrass.service
```
* 上述命令啟動失敗輸入以下命令手動啟動並查看啟動狀況 :
```bash
sudo /greengrass/ggc/core/greengrassd start
```
* 關閉Greengrass Core :
```bash
sudo systemctl stop greengrass.service
```
### AWS IoT 步驟
* 進入IoT Greengrass頁面
* 切換到Test頁面，Subscribe `intel/faas/ssd`
* 若成功啟動即會看到資訊
Sign in with Wallet
Connect another wallet