指定tflite模型用NPU推論 ``` sudo python3 /usr/share/benchmark_dla/benchmark.py --file "model.tflite" --target mdla3.0 ``` ![image](https://hackmd.io/_uploads/BkimIX8xyx.png) 檢查模型輸入輸出資訊(限用tflite) ```ncc-tflite --show-io-info model.tflite``` ``` # of input tensors: 1 [0]: normalized_input_image_tensor ├ Type: kTfLiteUInt8 (要找這個type) ├ AllocType: kTfLiteArenaRw ├ **Shape: {1,300,300,3}** (要找這個大小) ├ Scale: 0.0078125 ├ ZeroPoint: 128 └ Bytes: 270000 # of output tensors: **12 **(要找這個數量) 如果type是kTfLiteUInt8的話 recommand_compile_options = '--relax-fp32' ``` tflite --> dla sudo ncc-tflite -arch mdla3.0 /usr/share/benchmark_dla/ssd_mobilenet_v1_coco_quantized.tflite -o /usr/share/benchmark_dla/ssd_mobilenet_v1_coco_quantized.dla ``` sudo ncc-tflite -arch mdla3.0 model.tflite -o model.dla ``` Inference(會有Error) ``` neuronrt -m hw -a model.dla -c 100 -b 100 -i input.bin -o output.bin ex: neuronrt -m hw -a yolov8n_float32-mdla3.0.dla -c 100 -b 100 -i yolov8n_image.bin -o output_yolov8n.bin ``` bechmark_resized調整(使用resized生成的bin): ``` # gen rand input.bin input_bin = 'yolov8n_image.bin' # gen_input_bin(modeil_io, input_bin) ``` 註解後可以保留dla檔 但生成得dla會在給的模型的目錄下 ``` if not cache_dla: os.remove(dla) ``` ## 前處理 ``` # 讀取圖片 image = cv2.imread(image_path, cv2.IMREAD_COLOR) print("image size =>", image.shape) image_height, image_width = image.shape[:2] # 調整圖片大小以符合模型的輸入尺寸 (H, W, C) resized = cv2.resize(image, (model_io.intput_size[1], model_io.intput_size[2])) print("resize => ", resized.shape) # (高, 寬, 顏色通道) # # 如果模型的輸入類型是 float32，將圖片pixel歸一化為 [0, 1] 但大小相同 # if model_io.intput_type == 'kTfLiteFloat32': input_image = resized / 255.0 print("normalize => ", input_image.shape) input_image = input_image.transpose(2, 0, 1) # (顏色通道, 高, 寬) = (2, 0, 1) print("transpose => ", input_image.shape) input_tensor = input_image[np.newaxis, :, :, :].astype(np.float32) print( "input_tensor => ", input_tensor.shape ) # (1, 3, 640, 640) [1張, 顏色通道, 高, 寬] # 將數據保存為二進制文件 input_tensor.tofile(filename) ``` `first_box = output_data[0][:, 0] # 這將返回第一個候選框的 84 個數值` ## 後處理 問題: 這是output.bin的size: (705600,) reshape後的size(1, 84, 8400) 不知道數據的順序有沒有還原成原本模型的output? 或是當初模型轉tflite有問題?或是推論有問題? ``` model_io = query_tflite_model_io_info('yolov8n_saved_model/yolov8n_float32.tflite') conf_thresold = 0.7 # 讀取圖片 image_path = 'images/bus.jpg' # 替換為你的圖片路徑 image = cv2.imread(image_path) # 檢查圖片是否成功讀取 if image is None: print("Error: Unable to load image.") else: # 獲取圖片大小 height, width, channels = image.shape # channels 是顏色通道數量 print(f"Image Size: Width={width}, Height={height}, Channels={channels}") # 正確的輸出形狀 output_shape = (1, 84, 8400) # 確保這是正確的整數元組 # 讀取 output_0.bin 並將其轉換為 numpy array output_data = np.fromfile('output_0.bin', dtype=np.float32) # print(output_data.shape) # 將數據調整為模型的輸出形狀 output_data = output_data.reshape(output_shape) # print(output_data.shape) predictions = np.squeeze(output_data).T # print(predictions.shape) scores = np.max(predictions[:, 4:], axis=1) # print("scores max => ", scores.shape) # 過濾分數 predictions = predictions[scores > conf_thresold, :] scores = scores[scores > conf_thresold] ### 過濾出座標，並重組框框 boxes = predictions[scores > conf_thresold, :4] boxes = np.array( [ ( box[0] - box[2] * 0.5, box[1] - box[3] * 0.5, box[2], box[3], ) for box in boxes ], dtype=np.float32, ) # print("boxes => ", boxes.shape) # 獲取置信度最高的類 class_ids = np.argmax(predictions[:, 4:], axis=1) # print("class => ", class_ids.shape) scale_x = width / model_io.intput_size[2] scale_y = height / model_io.intput_size[1] input_shape = np.array([scale_x, scale_y, scale_x, scale_y]) boxes = np.multiply(boxes, input_shape, dtype=np.float32).astype(np.int32) print("boxes: ", boxes.shape) indices = cv2.dnn.NMSBoxes(boxes, scores, conf_thresold, 0.45, 0.5) # type: ignore image_output = (boxes, scores, class_ids, indices) print(image_output) ```