Try   HackMD

使用GStreamer Python API解析RTP與RTCP封包

Streaming 系列筆記

更多deepstream的實戰應用:

如何使用GStreamer元件解析RTP與RTCP內的資料

主要依賴gstreamer中的rtpbin元素實踐

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

RTP bin結合了rtpsession、rtpssrcdemux、rtpjitterbuffer和rtpptdemux的功能，將它們集成為一個元素。它允許多個RTP會話，這些會話將使用RTCP SR封包進行同步。

rtpbin配置了一些請求端口(request pads)，這些端口定義了要啟動的功能，類似於rtpsession元素。

  • 使用rtpbin進行RTP接收與發送

    • 接收RTP封包
      • 需請求一個recv_rtp_sink_%u端口
        • 必須在端口名稱中指定會話號碼。
      • 在recv_rtp_sink_%u端口上接收的數據將在rtpsession管理器中進行處理，經過驗證後轉發到rtpssrcdemux元素。
      • 每個RTP流都是基於SSRC進行解多路復用的，然後發送到rtpjitterbuffer。在從抖動緩衝區釋放數據包後，它們將被轉發到rtpptdemux元素。
      • rtpptdemux元素將根據有效載荷類型解多路復用數據包，並將會話號碼、SSRC和載荷類型分別作為端口名稱，在rtpbin上創建一個唯一的pad recv_rtp_src_%u_%u_%u。
    • 發送RTP封包
      • 需請求(request)一個send_rtp_sink_%u端口
      • 它將自動創建一個send_rtp_src_%u端口。如果未提供會話號碼，則將返回最低可用會話的端口。
      • 會話管理器將修改RTP封包中的SSRC為其自己的SSRC，然後在更新其內部狀態後將封包轉發到send_rtp_src_%u端口。

  • 使用rtpbin進行RTCP接收與發送

    • 接收RTCP封包
      需請求(request)一個recv_rtcp_sink_%u端口。必須在端口名稱中指定會話號碼。
    • 發送RTCP封包
      如果希望會話管理器生成並發送RTCP封包，需請求一個帶有會話號碼的send_rtcp_src_%u端口。在此端口上推送的封包包含應發送給會話中所有參與者的SR/RR RTCP報告。

使用get-internal-session屬性來訪問rtpbin的內部統計信息。此動作信號提供對RTPSession對象的訪問，RTPSession對象進一步提供了獲取內部源和其他源的動作信號。

Gstreamer範例Pipeline

  • 範例1：單純使用rtpbin接收RTP packets
    接收來自端口5000的RTP數據，並發送到rtpbin中的session 0

    

    ​​​​​gst-launch-1.0 udpsrc port=5000 caps="application/x-rtp, ..." ! .recv_rtp_sink_0 \
​​​​​    rtpbin ! rtptheoradepay ! theoradec ! xvimagesink

  • 範例2： 視訊、音訊分別接收並發送RTP與RTCP packets

    • RTP packets管道

      • 將來自v4l2src的H263video編碼並打包，從audiotestsrc生成AMR audio並打包。
      • video
        • 發送到rtpbin中的session 0，然後至udpsink的port 5000
      • audio
        • 發送到rtpbin中的session 1，然後至udpsink的port 5002

    • RTCP packets管道

      • video
        • session 0的RTCP packets會被發送到port 5000+1，在port 5005上接收
      • audio
        • session 1的RTCP packets會被發送到port 5002+1。在port 5007上接收
        • 在port 5007上接收
          由於發送者的RTCP數據包應該盡快發送並且不參與在port 5005上接收，因此在udpsink上配置sync=false和async=false
    








    ​​​​ gst-launch-1.0 rtpbin name=rtpbin \
​​​​         v4l2src ! videoconvert ! ffenc_h263 ! rtph263ppay ! rtpbin.send_rtp_sink_0 \
​​​​                   rtpbin.send_rtp_src_0 ! udpsink port=5000                            \
​​​​                   rtpbin.send_rtcp_src_0 ! udpsink port=5001 sync=false async=false    \
​​​​                   udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0                           \
​​​​         audiotestsrc ! amrnbenc ! rtpamrpay ! rtpbin.send_rtp_sink_1                   \
​​​​                   rtpbin.send_rtp_src_1 ! udpsink port=5002                            \
​​​​                   rtpbin.send_rtcp_src_1 ! udpsink port=5003 sync=false async=false    \
​​​​                   udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

  • 範例3： 視訊、音訊分別接收並發送RTP與RTCP packets並撥放

    • RTP packets管道
      • video
        • 使用udpsrc接收port 5000上的H263視頻，通過rtpbin的recv_rtp_sink_0發送到session 0，解包、解碼並顯示影片
      • audio
        • 接收port 5002上的AMR音檔，通過rtpbin的recv_rtp_sink_1發送到session 1，解包、解碼並播放
    • RTCP packets管道
      這些RTCP 封包的資料將經過以下管道用於會話管理和同步
      • video
        • 通過rtpbin的recv_rtcp_sink_0以port 5000+1接收session 0的RTCP封包
        • 再發送session 0的RTCP報告到port 5005
      • audio
        • 通過rtpbin的recv_rtcp_sink_0以port 5002+1接收session 1的RTCP封包
        • 再發送session 1的RTCP報告到port 5007
    










    ​​​​ gst-launch-1.0 -v rtpbin name=rtpbin                                          \
​​​​     udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-1998" \
​​​​             port=5000 ! rtpbin.recv_rtp_sink_0                                \
​​​​         rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink                    \
​​​​      udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                               \
​​​​      rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false        \
​​​​     udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)AMR,encoding-params=(string)1,octet-align=(string)1" \
​​​​             port=5002 ! rtpbin.recv_rtp_sink_1                                \
​​​​         rtpbin. ! rtpamrdepay ! amrnbdec ! alsasink                           \
​​​​      udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
​​​​      rtpbin.send_rtcp_src_1 ! udpsink port=5007 sync=false async=false

對Deepstream 使用uridecodebin 進行客製修改

  • 對應Deepstream中使用uridecodebin元素作為接收rtsp stream的工具，因此必須根據使用的插件內容來發送訊號、訪問RTCP SR、並捕獲、解析RTCP 封包

    • 當輸入來源為"rtsp://…"時 ，uridecodebin內部完整的的element組成
    • 當資料來源是rtsp時，decodebin內部會調用rtspsrc來解析

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More → 

    decodebin
    rtph264depay
    h264parse
    capsfilter
    nvv4l2decoder

  • 範例中的Video Decode Pipeline

    • rtspsrc 負責連接 RTSP 服務器並讀取數據。rtspsrc就像一個實時信號源，因此只會在播放狀態下生成數據。

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More →

    • rtspsrc 在內部使用rtpbin實例化一個 "RTP 會話管理器"(RTP session manager)元件，負責與 rtsp server雙向對話，取得RTCP訊息

  • decodebin與rtspsrc

    特性 decodebin rtspsrc
    定義 - decodebin 是一個自動解碼器元件 (element) - rtspsrc 是一個用於接收 RTSP 流的元件 (element)
    功能 - 自動檢測和使用適當的解碼器來解碼多媒體數據 - 處理 RTSP 協議，從 RTSP 伺服器接收數據
    - 可以處理音頻和視頻數據 - 可以接收 RTP 和 RTCP 數據
    用途 - 用於動態解碼多媒體數據，不需要事先知道數據的格式 - 用於從 RTSP 伺服器接收多媒體流
    連接性 - 通常連接到數據源 (例如 filesrc 或 rtspsrc) 來接收未解碼的多媒體數據 - 通常連接到解碼器 (例如 decodebin) 或其他處理 RTP 數據的元件，以獲取解碼後的多媒體數據
    特殊功能 - 當它接收到數據時，它會動態地選擇和更改解碼器 - 支持 RTSP 的各種功能，如認證、選擇媒體流、控制播放速度等

實作

實作-解析RTCPBuffer

  • def on_receiving_rtcp_callback
    • rtcp_packet.sr_get_sender_info()裡面包含了:
      • ssrc、ntptime、rtptime、packet_count、octet_count等資料
    • 記得要用GstRtp.RTCPBuffer.unmap(rtcp_buffer)釋放記憶體
    ​​​​    # Process RTCP Packets  ---------------- 
​​​​    def on_receiving_rtcp_callback(session, buffer: Gst.Buffer):
​​​​        """
​​​​        Fetch the NTP and RTP reference Timestamp in the RTCP Sender Report
​​​​        Map RTCP and RTP packet data to calculate latency.
​​​​        """
​​​​        rtcp_buffer = GstRtp.RTCPBuffer()
​​​​        if not GstRtp.RTCPBuffer.map(buffer, Gst.MapFlags.READ, rtcp_buffer):
​​​​            logging.warning("\t\t[on_receiving_rtcp_callback] : Unable to map RTCP buffer")
​​​​            return Gst.PadProbeReturn.PASS
​​​​        try:
​​​​            rtcp_packet = GstRtp.RTCPPacket()
​​​​            if not rtcp_buffer.get_first_packet(rtcp_packet):
​​​​                logging.warning("\t\t[on_receiving_rtcp_callback] : Unable to get the first RTCP packet")
​​​​                return Gst.PadProbeReturn.PASS
​​​​            while rtcp_packet:
​​​​                if rtcp_packet.get_type() == GstRtp.RTCPType.SR:
​​​​                    rtcp_data = *rtcp_packet.sr_get_sender_info()
​​​​                if not rtcp_packet.move_to_next():
​​​​                    break
​​​​        finally:  # Add finally block to ensure buffer is unmapped
​​​​            GstRtp.RTCPBuffer.unmap(rtcp_buffer)

​​​​        return Gst.PadProbeReturn.PASS

實作-解析RTPBuffer

  • def rtp_depay_sink_pad_buffer_probe

    • 這個探針函式要靜態插在第一個接收RTP的元件前，名稱通常是rtpdepay
    • 記得同樣要在最後面用.unmap()釋放記憶體
    ​​​​# Process RTP Packets and Calculate RTSP Latency ----------------------------- 

​​​​def rtp_depay_sink_pad_buffer_probe(
​​​​    self, pad: Gst.Pad, info: Gst.PadProbeInfo, u_data: Any
​​​​) -> Gst.PadProbeReturn:
​​​​    """
​​​​    Handle the sink pad buffer probe for RTP buffer in depay.

​​​​    Parameters:
​​​​    - pad (Gst.Pad): The pad to fetch the buffer from.
​​​​    - info (Gst.PadProbeInfo): Information about the buffer.
​​​​    """
​​​​    buffer = info.get_buffer()
​​​​    res, rtp_buffer = GstRtp.RTPBuffer.map(buffer, Gst.MapFlags.READ)
​​​​    try:
​​​​        rtp_data = [
​​​​            rtp_buffer.get_timestamp(),
​​​​            rtp_buffer.get_ssrc(),
​​​​            rtp_buffer.get_payload_type(),
​​​​            rtp_buffer.get_marker(),
​​​​            rtp_buffer.get_seq()
​​​​        ]
​​​​    finally:
​​​​        rtp_buffer.unmap()
​​​​    return Gst.PadProbeReturn.OK

如何在Gstreamer uridecodebin 中使用

Sample code modified from NVIDIA-AI-IOT/deepstream_python_apps/deepstream_test1_rtsp_in_rtsp_out.py

由於uridecodebin內部會自動建立接收rtsp等相關的元件，且內部會使用高階APIrtspsrc，因此需要建立一個manager的callback函式，讓rtspsrc內部rtpbin的連接到RTPsession物件，透過rtspsrc.connect("new manager", new_manager_callback)這樣的方式連接

  • def new_manager_callback
    對應前面所提：rtspsrc 在內部使用rtpbin實例化一個 “RTP 會話管理器”(RTP session manager)元件，負責與 rtsp server雙向對話，取得RTCP訊息

    • 先確認rtpsrc中的元件為rtpbin
    • 為了使用rtpbin來作為RTCP接收器，使用get_request_pad動態建立recv_rtcp_sink_%u接口
    • 確認後發送"get-internal-session"訊號，這個操作可以訪問rtpbin內部的RTPSession object以取得rtpbin內部統計資料，透過連接RTPSession也可以進一步檢索內部Buffer及其他資料源

      GStreamer/rtpbin

      • use rtpbin as an RTCP receiver, request a recv_rtcp_sink_%u pad. The session number must be specified in the pad name.
      • Access to the internal statistics of rtpbin is provided with the get-internal-session property. This action signal gives access to the RTPSession object which further provides action signals to retrieve the internal source and other sources.
    ​​​​def new_manager_callback (rtspsrc, manager, index, *args):
​​​​    '''
​​​​    Manage RTP sessions for accessing RTCP packets
​​​​    '''
​​​​    element_name = manager.get_factory().get_name()
​​​​    # print(f"\t[new_manager_callback] : List element_name '{element_name}-{index}' in Decodebin ")
​​​​    if element_name != "rtpbin":
​​​​        # print(f"\t[new_manager_callback] : Manager is of type {element_name}, not rtspsrc. ")
​​​​        return

​​​​    # Establish RTP session in src pad for fetching RTCP packets
​​​​    sinkpad = manager.get_request_pad(f"recv_rtcp_sink_{index}") 
​​​​    session = manager.emit("get-internal-session", 0)
​​​​    session.connect("on-receiving-rtcp", on_receiving_rtcp_callback)

  • 上面的manager的相關操作等價於下面的操作方式:

    ​​​​# Receive RTCP Packets -------------------------         
​​​​# Configure rtpbin to handle the RTP/RTCP session
​​​​recv_rtcp_sink_pad = Gst.Element.get_request_pad(rtpbin, f"recv_rtcp_sink_{index}")
​​​​session = rtpbin.emit("get-internal-session", 0)
​​​​session.connect("on-receiving-rtcp", on_receiving_rtcp_callback)

  • def decodebin_child_added

    • 找出uridecodebin中rtspsrc元件，準備連接建立"RTP session manager"以接收RTCP資料 
      ​​​​​​​​[decodebin_child_added] : Element type of 'source-00': 'rtspsrc'
​​​​​​​​[decodebin_child_added] : Successfully connected 'on-receiving-rtcp' with 'source-00'.
    • 找出uridecodebin中，名稱有"depay"的元件，插入解析RTP Buffer的探針函示(rtp_depay_sink_pad_buffer_probe)以獲取資料
    ​​​​def decodebin_child_added(child_proxy, Object, name, user_data, index):
​​​​    print("Decodebin child added:", name, "\n")
​​​​    if name.find("decodebin") != -1:
​​​​        Object.connect("child-added", new_manager_callback, user_data)

​​​​    if "source" in name:
​​​​        element_type = Object.get_factory().get_name()
​​​​        if element_type == "rtspsrc":
​​​​            try:
​​​​                # Use functools.partial to freeze the 'index' parameter for the callback
​​​​                # Object.connect("new-manager", new_manager_callback)
​​​​                Object.connect("new-manager", partial(new_manager_callback, index=index))
​​​​                
​​​​    # Add Probe for *depay, which is used to extract video from RTP packets.
​​​​    # uridecodebin is a high-level element that dynamically creates its internal pipeline, including elements like rtph264depay,
​​​​    # based on the media it's given. This means that the rtph264depay element may not exist until the pipeline is set to the PAUSED
​​​​    # or PLAYING state and has detected that the incoming stream is H.264 encoded.
​​​​    if ("depay") in name:
​​​​        element_type = Object.get_factory().get_name()
​​​​        depay_sinkpad = Object.get_static_pad("sink")
​​​​        if depay_sinkpad:
​​​​            depay_sinkpad.add_probe(
​​​​                Gst.PadProbeType.BUFFER,
​​​​                rtp_depay_sink_pad_buffer_probe,
​​​​                0,
​​​​            )

  • def create_source_bin

    • 使用partial(decodebin_child_added, index=index)連結decodebin_child_added函式，方便傳遞index參數進入、增加可讀性
    ​​​​def create_source_bin(index, uri):
​​​​    bin_name = "source-bin-%02d" % index
​​​​    nbin = Gst.Bin.new(bin_name)
​​​​    ...
​​​​    uri_decode_bin.connect("child-added", partial(decodebin_child_added, index=index), nbin)

​​​​    ...
​​​​    return nbin

Reference

Blog and Forum

2022/10。qorela.com。Absolute Timestamp of Arbitrary Frame in RTSP Stream with GStreamer

1. 使用rtspsrc元素來接收rtsp的uri，並以此發送"new-manager"訊號(emit signal)給rtpbin
  • 首先，需要捕獲並連接 rtpbin 元素的創建信號，以便訪問它。為此，我們從第 5 行的管道中提取了 rtspsrc 元素。
  • 在創建管理器後，該元素會產生一個 "new-manager "信號
  • 對於 rtpbin 元素，回調函數與該信號相連，如第 6 行所示。我們將使用在該函數中獲得的會話信息訪問 RTCP SR。
  • 訪問 RTP 數據包中的時間值
  • 在 rtph264depay 元素的 sink pad 上添加一個探針
  • 第 8 行中的 rtph264depay 元素和第 9 行中該元素的 sink pad 已被提取
  • 通過在第 10 行獲得的pad(即rtph264depay的輸入sink pad)上添加探針來連接該功能








pipeline = Gst.parse_launch("rtspsrc name = rtspsrc locationmrtsp://some.server/url | rtph264depay nane=depay ! appsink nane=sink") 

rtspsrc = pipeline.get_by_name('rtspsrc') 
rtspsrc.connect("new-manager" ‚ new_manager_callback) 

depay = pipeline.get_by_name('depay') 
sinkpad = depay.get_static_pad('sink') 
probeID = sinkpad.add_probe(Gst.PadProbeType.BUFFER, calculate_tinestamp)
2. rtpbin收到訊號，透過def new_manager_callbacK函式獲得的對話訊息訪問"RTCP SR"
  • 完成上述設置後，當創建 rtpbin 元素時，它將作為管理器參數傳遞給我們的回調函數
  • 使用會話捕獲傳入的 RTCP 數據包。如第 14 行所述，為了到達會話，應請求該會話的 pad。
  • 如第 15 行所述，應發出內部會話。我們應通過一個新的回調函數連接到 "on-receiving-rtcp "信號。
  • 這樣，每次有新的 RTCP 數據包到達時，都會調用該回調函數







def new_manager_callback ( rtspsrc, manager): 
    sinkpad = manager.get_request_pad("recv_rtp_sink_0") 
    session = manager.emit("get-internal-session", 0)
    if session:
        session.connect("on-receiving-rtcp", on_receiving_rtcp_callback)
    else:
        print("Failed to get RTPSession from the manager")
3. 建立def on_receiving_rtcp_callback，使用GstRtp.RTCPBuffer捕獲、解析RTCP 數據封包
  • def on_receiving_rtcp_callback 函數將傳遞一個 Gst.Buffer 類型的緩沖區
  • 該緩沖區必須映射到第 21 行中的 RTCP 緩沖區，這樣才能訪問 RTCP 數據包。
  • 除了發送方報告外，RTCP 還可以接收其他數據包。因此，應從接收到的數據包中提取帶有發送方報告的數據包。
  • 我通過 RTCP 緩沖區獲取第一個數據包，如第 23 行中的 get_first_packet。隨後的數據包也可通過第 30 行獲取。與第 27 行一樣，可以通過數據包訪問發送方報告本身。
  • 到達 GStreamer 緩沖區的發送方報告示例為"(ssrc=1644170567, ntptime=16645962074316441600, rtptime=2847690, packet_count=475, octet_count=623890)"。應保存此報告中的 NTP 和 RTP 時間值，以便日後使用。













def on_receiving_rtcp_callback (session, buffer: Gst.Buffer): 
  rtcp_buffer = GstRtp.RTCPBuffer() 
  res = GstRtp.RTCPBuffer.map (buffer, Gst.MapFlags.READ, rtcp_buffer) 
  rtcp_packet = GstRtp.RTCPPacket() 
  packet = rtcp_buffer.get_first _packet ( rtcp_packet) 

  while True: 
    if rtcp_packet.get_type() == GstRtp.RTCPType.SR : 
      si = rtcp_packet.sr_get_sender_info() 
      ntp_time = si[1] 
      rtp_time = si[2] 
    if rtcp_packet.move_to_next () ==False: 
      break
4. def calculate_timestamp

FRAME_TS= LAST_NTP + (RTP — LAST_RTP) / 90000

Where

  • RTP is the frame timestamp received via RTP header
  • LAST_NTP is NTP clock time received via RTCP Sender Report
  • LAST_RTP is a reference RTP time received via RTCP Sender Report
  • 90000 is required because the RTP timestamps increment with 90 kHz per RTSP specification. That means the RTP timestamp increments by 90000 every second. This value may vary for the different clock rates.








def calculate_timestamp(pad, info): 
    res, rtpBuff = GstRtp.RTPBuffer.map(info.get_buffer(), Gst.MapFlags.READ) 
    tv = timevalue() 
    last_rtcp_ntp_time = c_uint64(ntp_timestamp) 
    LibNTP.ntp2tv(byref(last_rtcp_ntp_time), byref(tv)) 
    rtp_diff = float(rtpBuff.get_timestamp() - rtp_timestamp) / 90000.0 
    timestamp = float(tv.tv_sec) + float(tv.tv_usec) / 1000000.0 + rtp_diff 
    return Gst.PadProbeReturn.OK

2020。Sending and Receiving custom RTCP packets/events with GStreamer

1.Get the RTPSession object from the RtpBin

g_signal_emit_by_name (rtpbin, "get-internal-session", id, &session);

2.Attach to the "on-receiving-rtcp" signal (or to the more specialized ones):

g_signal_connect_after (session, "on-receiving-rtcp",
    G_CALLBACK (on_rtcp_callback), my_callback_data);

3.Look for the RTCP message you want

gst_rtcp_buffer_map (buffer, GST_MAP_READ, &rtcp_buffer));
new_packet = gst_rtcp_buffer_get_first_packet (&rtcp_buffer, &rtcp_packet);

while (gst_rtcp_packet_move_to_next (&rtcp_packet) {
  type = gst_rtcp_packet_get_type (&rtcp_packet);
  switch (type) {
  ...
  }
}

2020。Antonio Ospite。collabora.com。High bitrate video streaming with GStreamer's RTP elements

Document

gstreamer/documentation/additional/RTP and RTSP support

Most of GStreamer's key RTP components live in gst-plugins-good:

  • The rtpmanager plugin contains elements like rtpbin and rtpjitterbuffer
  • The rtp plugin contains RTP payloading and depayloading elements for many different codecs and container formats
  • rtpbin is the high-level RTP component and supports sending and receiving, just sending or just receiving data, with and without RTCP support. This is the bin that does it all: it adapts dynamically to your needs based on the requested pads; it also contains an rtpjitterbuffer.

gstreamer/rtpmanager/rtpbin

To use rtpbin as an RTP receiver, request a recv_rtp_sink_%u pad. The session number must be specified in the pad name. Data received on the recv_rtp_sink_%u pad will be processed in the rtpsession manager and after being validated forwarded on rtpssrcdemux element. …
Access to the internal statistics of rtpbin is provided with the get-internal-session property. This action signal gives access to the RTPSession object which further provides action signals to retrieve the internal source and other sources.

  • get-internal-session 
    ​​​​g_signal_emit_by_name (rtpbin, "get-internal-session", id, &ret);

gstreamer/rtsp/rtspsrc

rtspsrc will internally instantiate an RTP session manager element that will handle the RTCP messages to and from the server, jitter removal, packet reordering along with providing a clock for the pipeline. This feature is implemented using the gstrtpbin element.

To also use rtpbin as an RTCP receiver, request a recv_rtcp_sink_%u pad. The session number must be specified in the pad name.

  • new-manager 
    ​​​​new_manager_callback (GstElement * rtspsrc,
​​​​                      GstElement * manager,
​​​​                      gpointer udata)

gstreamer/rtpmanager/RTPSession

  • on-receiving-rtcp
    This signal is emitted when receiving an RTCP packet before it is handled by the session. It can be used to extract custom information from RTCP packets. 
    ​​​​def on_receiving_rtcp_callback (session, buffer, udata):
​​​​    #python callback for the 'on-receiving-rtcp' signal

gstreamer/rtplib/GstRTPBuffer

  • GstRtp.RTPBuffer.get_timestamp
    Get the timestamp of the RTP packet in buffer.

    

    ​​​​def GstRtp.RTPBuffer.get_timestamp (self):
​​​​    #python wrapper for 'gst_rtp_buffer_get_timestamp'

  • GstRtp.RTPBuffer.get_marker
    Check if the marker bit is set on the RTP packet in buffer.

    

    ​​​​def GstRtp.RTPBuffer.get_marker (self):
​​​​    #python wrapper for 'gst_rtp_buffer_get_marker'

  • GstRtp.RTPBuffer.get_ssrc
    Get the SSRC of the RTP packet in buffer.

    ​​​​def GstRtp.RTPBuffer.get_ssrc (self):
​​​​    #python wrapper for 'gst_rtp_buffer_get_ssrc'

gstreamer/rtplib/GstRTCPBuffer

GstRTCPBuffer 的c++原始程式碼

The GstRTPCBuffer helper functions makes it easy to parse and create regular Gst.Buffer objects that contain compound RTCP packets. These buffers are typically of 'application/x-rtcp' Gst.Caps.

An RTCP buffer consists of 1 or more GstRtp.RTCPPacket structures that you can retrieve with GstRtp.RTCPBuffer.get_first_packet. GstRtp.RTCPPacket acts as a pointer into the RTCP buffer; you can move to the next packet with GstRtp.RTCPPacket.move_to_next.

  • source code

  • GstRtp.RTCPBuffer.get_first_packet
    Initialize a new GstRtp.RTCPPacket pointer that points to the first packet in rtcp.

    

    ​​​​def GstRtp.RTCPBuffer.get_first_packet (self, packet):
​​​​    #python wrapper for 'gst_rtcp_buffer_get_first_packet'

  • GstRtp.RTCPBuffer.map
    The resulting RTCP buffer state is stored in rtcp.

    ​​​​def GstRtp.RTCPBuffer.map (buffer, flags, rtcp):
​​​​    #python wrapper for 'gst_rtcp_buffer_map'
​​​​Open buffer for reading or writing, depending on flags.

  • sr_get_sender_info

    ​​​​def GstRtp.RTCPPacket.sr_get_sender_info (self):
​​​​    #python wrapper for 'gst_rtcp_packet_sr_get_sender_info'

    ​​​​gst_rtcp_packet_sr_get_sender_info (GstRTCPPacket * packet,
​​​​                                    guint32 * ssrc,
​​​​                                    guint64 * ntptime,
​​​​                                    guint32 * rtptime,
​​​​                                    guint32 * packet_count,
​​​​                                    guint32 * octet_count)

    • source code : gst-libs/gst/rtp/gstrtcpbuffer.c/gst_rtcp_packet_sr_get_sender_info

      









































      ​​​​​​​​/**
​​​​​​​​ * gst_rtcp_packet_sr_get_sender_info:
​​​​​​​​ * @packet: a valid SR #GstRTCPPacket
​​​​​​​​ * @ssrc: result SSRC
​​​​​​​​ * @ntptime: result NTP time
​​​​​​​​ * @rtptime: result RTP time
​​​​​​​​ * @packet_count: result packet count
​​​​​​​​ * @octet_count: result octet count
​​​​​​​​ *
​​​​​​​​ * Parse the SR sender info and store the values.
​​​​​​​​ */
​​​​​​​​void
​​​​​​​​gst_rtcp_packet_sr_get_sender_info (GstRTCPPacket * packet, guint32 * ssrc,
​​​​​​​​    guint64 * ntptime, guint32 * rtptime, guint32 * packet_count,
​​​​​​​​    guint32 * octet_count)
​​​​​​​​{
​​​​​​​​  guint8 *data;

​​​​​​​​  g_return_if_fail (packet != NULL);
​​​​​​​​  g_return_if_fail (packet->type == GST_RTCP_TYPE_SR);
​​​​​​​​  g_return_if_fail (packet->rtcp != NULL);
​​​​​​​​  g_return_if_fail (packet->rtcp->map.flags & GST_MAP_READ);

​​​​​​​​  data = packet->rtcp->map.data;

​​​​​​​​  /* skip header */
​​​​​​​​  data += packet->offset + 4;
​​​​​​​​  if (ssrc)
​​​​​​​​    *ssrc = GST_READ_UINT32_BE (data);
​​​​​​​​  data += 4;
​​​​​​​​  if (ntptime)
​​​​​​​​    *ntptime = GST_READ_UINT64_BE (data);
​​​​​​​​  data += 8;
​​​​​​​​  if (rtptime)
​​​​​​​​    *rtptime = GST_READ_UINT32_BE (data);
​​​​​​​​  data += 4;
​​​​​​​​  if (packet_count)
​​​​​​​​    *packet_count = GST_READ_UINT32_BE (data);
​​​​​​​​  data += 4;
​​​​​​​​  if (octet_count)
​​​​​​​​    *octet_count = GST_READ_UINT32_BE (data);
​​​​​​​​}
Last changed by 
 
YH Hsu
www.linkedin.com/in/yung-hui-hsu-226b50a6
0
2
2291

Read more

[GenAI][AI Agents] Long-Term Agentic Memory With LangGraph - Baseline Email Assistant

Large Language Models with Semantic Search。大型語言模型與語義搜索

Apr 1, 2025
[GenAI][AI Agents] Long-Term Agentic Memory With LangGraph - Introduction to Agent Memory

Long-Term Agentic Memory With LangGraph

Apr 1, 2025
[AI Agents in LangGraph](https://learn.deeplearning.ai/courses/ai-agents-in-langgraph/lesson/1/introduction)

LanaGraph Components

Mar 19, 2025
AI / ML領域相關學習筆記入口頁面

Mar 19, 2025
Read more from YH Hsu
Published on HackMD