Gstreamer framerate

1. Build Gstreamer

Steps

# install a few dependencies
sudo pip3 install ninja
sudo apt-get install cmake meson flex bison
sudo apt-get install libglib2.0-dev libjpeg-dev libx264-dev
sudo apt-get install libgtk2.0-dev libcanberra-gtk* libgtk-3-dev

# for omx
sudo apt-get install libglfw3-dev libgles2-mesa-dev

# needed for alsasrc, alsasink (currently not needed)
sudo apt-get install libasound2-dev

# start building
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/opt/vc/lib/pkgconfig/
meson builddir -D omx=enabled -D gst-omx:target=rpi
ninja -C builddir

Problems

# "config.h" not found
create an empty one.

# library 'GLESv2' not found
sudo apt-get install libglfw3-dev libgles2-mesa-dev

# 'OMX_IndexConfigBrcmVideoRequestIFrame' undeclared
ln -s /opt/vc/include/IL/OMX_Index.h gst-omx/omx/openmax/OMX_Index.h 

2. Raspberry Pi Build Image

1. 用 Win32DiskImager 將SD卡內的資料轉成.img檔
2. 但此時檔案大小會跟SD卡容量相等，需再使用 pishrink 對此.img檔進行壓縮
3. 還原系統: 使用Raspberry Pi Imager，在 CHOOSE OS 中點選"Use Custom"並選取此.img檔

3. Test System (original，改omx264enc)

進入編譯好的gstreamer環境 (進入後即可使用 gst-launch-1.0 等相關指令):

ninja -C builddir devenv

rpi 發送端:

gst-launch-1.0 v4l2src device=/dev/video0  !  video/x-raw,width=1280,height=720,framerate=60/1 ! videoconvert ! 'video/x-raw, framerate=60/1,width=(int)1280, height=(int)720, format=(string)I420' ! queue ! omxh264enc target-bitrate=2000000 control-rate=variable ! h264parse ! rtph264pay  ! queue ! udpsink host=<receiving-ip-addr> port=5001

rpi3不太支援 v4l2src，暫時改成rpicamsrc:

gst-launch-1.0 rpicamsrc bitrate=2000000 !  video/x-raw,width=1280,height=720,framerate=60/1 ! videoconvert ! 'video/x-raw, framerate=60/1,width=(int)1280, height=(int)720, format=(string)I420' ! queue ! omxh264enc target-bitrate=2000000 control-rate=variable ! h264parse ! rtph264pay  ! queue ! udpsink host=<receiving-ip-addr> port=5001

接收端:

gst-launch-1.0 -v udpsrc port=5001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

4. videorate

官方文件
原始程式碼
降fps反而提升CPU使用率問題

• drop-only=true 開啟後，程式不會duplicate，只會drop frame，因此可以吧處理延遲降到0。
• in, out, drop 這幾個參數只是用來 log 已讀進、讀出、丟掉了幾張照片。其餘的參數保持預設值就好。

測試指令：

gst-launch-1.0 v4l2src device=/dev/video0  !  video/x-raw,width=1280,height=720,framerate=60/1 ! videoconvert ! videorate drop-only=true ! 'video/x-raw, framerate=15/1,width=(int)1280, height=(int)720, format=(string)I420' ! queue ! omxh264enc target-bitrate=2000000 control-rate=variable ! h264parse ! rtph264pay ! queue ! udpsink host=<receiving-ip-addr> port=5001