# Xavier Use gstreamer + opencv Decode Encode Video

Code放在公司的gitlab上: http://192.168.33.240/xwater8/opencv_gstreamer_encode_decode

文章放在HackMD: https://hackmd.io/@xwater8/SJQXejDJF

## 實驗
在XavierAGX上同時解碼三部影片，以及同時輸出三部影片，統計處理500張Frame的平均時間

硬體: XavierAGX(16GB)

影片:
InputVideo: H.264 Video, Resolution: 1920x1080, **Process 3 Videos at the same time**.

OutputVideo: H.264 Video, Resolution: 1920x1080, **Process 3 Video at the same time**.

Total: The total time of process 500 frames in the video.

| Reader+Writer | AvgRead(sec/3frame) | AvgWrite(sec/3frame) | Total(sec) | comment |
|:---------------------------------:|:-------------------:|:--------------------:|:----------:|:--------------:|
| GStreamer-R + GStreamer-W | 0.025 | 0.0076 | 16.64 | CPU usage low |
| GStreamer-R + Opencv-W | 0.0098 | 0.0817 | 47.66 | |
| OpenCV-R + GStreamer-W | 0.0195 | 0.0054 | 12.57 | |
| OpenCV(thread)-R + GStreamer-W | 0.00046 | 0.0206 | 10.677 | CPU usage full |
| GStreamer(thread)-R + GStreamer-W | 0.000045 | 0.0079 | 4.06 | CPU usage 80% |

### 分析
> 1.OpenCV(thread)-R + GStreamerW: 因為CPU的利用率近乎滿載，所以影響到GStreamerW的輸出速度
> 2.GStreamer(thread): 解碼速度太快，可能有重複的Frame被解碼，所以輸出影片的時候影片看起來特別慢(同樣一張Frame可能有兩三張)。

### 注意事項
> 如果你使用rtsp串流作為輸入的話，顯示部份請將**nv3dsink sync=0**，否則看起來會非常delay。
> 如果你的Video畫面看起來是黑色的，或是每個row之間的Pixel看起來沒有對齊，請檢查你的VideoWriter解析度和Image解析度是否一樣。
> 有嘗試過在VideoWriter中，順便使用nvvidconv來resize圖片到指定解析度，但是失敗了。

### 參考資料:
https://blog.csdn.net/zong596568821xp/article/details/80306987

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/accelerated_gstreamer.html

https://developer.download.nvidia.com/embedded/L4T/r32_Release_v1.0/Docs/Accelerated_GStreamer_User_Guide.pdf?FJ0PgsovdxlD1ulMqnkeiWejcdISj9W4QIexZhQN6jm8MPucsOBoy-VYiigFZwbXYIvdYm5s7b67r6__hPwV3BGgKNRrlK8solnX-E61ksDF4GR1Lm690o0hP8mAbseKHSv-MEwr6-dKSgy-3Y_66QxR33huFMm3kb5v7GZh6zADwxu3lJc

Gstreamer replace cv2.imshow
https://forums.developer.nvidia.com/t/displaying-to-the-screen-with-opencv-and-gstreamer/140648/4

How to use tee element to create branch
http://divac76.blogspot.com/2020/10/gst-launch-10pipeline.html