---
# System prepended metadata

title: MTK推論測試

---

指定tflite模型用NPU推論
```
sudo python3 /usr/share/benchmark_dla/benchmark.py --file "model.tflite" --target mdla3.0
```
![image](https://hackmd.io/_uploads/BkimIX8xyx.png)

檢查模型輸入輸出資訊(限用tflite)
```ncc-tflite --show-io-info model.tflite```

```
# of input tensors: 1
[0]: normalized_input_image_tensor
 ├ Type: kTfLiteUInt8 (要找這個type)
 ├ AllocType: kTfLiteArenaRw
 ├ **Shape: {1,300,300,3}** (要找這個大小)
 ├ Scale: 0.0078125
 ├ ZeroPoint: 128
 └ Bytes: 270000
 
# of output tensors: **12 **(要找這個數量)

如果type是kTfLiteUInt8的話 
recommand_compile_options = '--relax-fp32'
```

tflite --> dla
sudo ncc-tflite -arch mdla3.0 /usr/share/benchmark_dla/ssd_mobilenet_v1_coco_quantized.tflite  -o /usr/share/benchmark_dla/ssd_mobilenet_v1_coco_quantized.dla
```
sudo ncc-tflite -arch mdla3.0 model.tflite -o model.dla
```
Inference(會有Error)
```
neuronrt -m hw -a model.dla -c 100 -b 100 -i input.bin -o output.bin

ex: 
neuronrt -m hw -a yolov8n_float32-mdla3.0.dla -c 100 -b 100 -i yolov8n_image.bin -o output_yolov8n.bin
```

bechmark_resized調整(使用resized生成的bin):
```
# gen rand input.bin
input_bin = 'yolov8n_image.bin'
# gen_input_bin(modeil_io, input_bin)
```
註解後可以保留dla檔 但生成得dla會在給的模型的目錄下
```
if not cache_dla:
    os.remove(dla)
```




## 前處理
```
# 讀取圖片
    image = cv2.imread(image_path, cv2.IMREAD_COLOR)
    print("image size =>", image.shape)

    image_height, image_width = image.shape[:2]

    # 調整圖片大小以符合模型的輸入尺寸 (H, W, C)
    resized = cv2.resize(image, (model_io.intput_size[1], model_io.intput_size[2]))
    print("resize => ", resized.shape)  # (高, 寬, 顏色通道)

    # # 如果模型的輸入類型是 float32，將圖片pixel歸一化為 [0, 1] 但大小相同
    # if model_io.intput_type == 'kTfLiteFloat32':
    input_image = resized / 255.0
    print("normalize => ", input_image.shape)

    input_image = input_image.transpose(2, 0, 1)  # (顏色通道, 高, 寬) = (2, 0, 1)
    print("transpose => ", input_image.shape)

    input_tensor = input_image[np.newaxis, :, :, :].astype(np.float32)
    print(
        "input_tensor => ", input_tensor.shape
    )  # (1, 3, 640, 640) [1張, 顏色通道, 高, 寬]
    
    # 將數據保存為二進制文件
    input_tensor.tofile(filename)
```

`first_box = output_data[0][:, 0]  # 這將返回第一個候選框的 84 個數值`

## 後處理
問題:
這是output.bin的size: (705600,)
reshape後的size(1, 84, 8400)
不知道數據的順序有沒有還原成原本模型的output?
或是當初模型轉tflite有問題?或是推論有問題?
```
model_io = query_tflite_model_io_info('yolov8n_saved_model/yolov8n_float32.tflite')
conf_thresold = 0.7

# 讀取圖片
image_path = 'images/bus.jpg'  # 替換為你的圖片路徑
image = cv2.imread(image_path)

# 檢查圖片是否成功讀取
if image is None:
    print("Error: Unable to load image.")
else:
    # 獲取圖片大小
    height, width, channels = image.shape  # channels 是顏色通道數量
    print(f"Image Size: Width={width}, Height={height}, Channels={channels}")

# 正確的輸出形狀
output_shape = (1, 84, 8400)  # 確保這是正確的整數元組

# 讀取 output_0.bin 並將其轉換為 numpy array
output_data = np.fromfile('output_0.bin', dtype=np.float32)

# print(output_data.shape)

# 將數據調整為模型的輸出形狀
output_data = output_data.reshape(output_shape)
# print(output_data.shape)

predictions = np.squeeze(output_data).T
# print(predictions.shape)

scores = np.max(predictions[:, 4:], axis=1)
# print("scores max => ", scores.shape)

# 過濾分數
predictions = predictions[scores > conf_thresold, :]
scores = scores[scores > conf_thresold]

### 過濾出座標，並重組框框
boxes = predictions[scores > conf_thresold, :4]
boxes = np.array(
    [
        (
            box[0] - box[2] * 0.5,
            box[1] - box[3] * 0.5,
            box[2],
            box[3],
        )
        for box in boxes
    ],
    dtype=np.float32,
)
# print("boxes => ", boxes.shape)

# 獲取置信度最高的類
class_ids = np.argmax(predictions[:, 4:], axis=1)
# print("class => ", class_ids.shape)

scale_x = width / model_io.intput_size[2]
scale_y = height / model_io.intput_size[1]
input_shape = np.array([scale_x, scale_y, scale_x, scale_y])
boxes = np.multiply(boxes, input_shape, dtype=np.float32).astype(np.int32)
print("boxes: ", boxes.shape)
indices = cv2.dnn.NMSBoxes(boxes, scores, conf_thresold, 0.45, 0.5)  # type: ignore
image_output = (boxes, scores, class_ids, indices)
print(image_output)
```