TODOs

Audio Quality

  • Prometheus for monitoring audio quality (what happened during bad audio?)
    • Include the new latency metrics (lossrate, late packets) (Gary)
    • Jitter buffer latency (figure out whether this is the main problem) - audio_jitter_buffer_diff
    • Display with Grafana (Likai)
      • Setup Grafana
      • Setup Prometheus
      • Configure query parameters on Grafana
  • Better JitterBuffer Adjust Algorithm (Chengw)
  • Find a proper maximum latency(500ms)
  • Set REMB based on audio stats
  • Send Lossrate to enable inbandfec
  • Find out why recoverable is 0. Not relate to whether there is FEC or not.
  • Check whether RTP FEC is useful or not
  • Figure out whether audio quality sometime degrades when there're videos.
  • Send a public stream to all silent peers
  • Experimenting sending audio via data channel
  • Add receiver report in Audio track

Video Quality

  • REMB to sender (need algorithm to calculate REMB) (Gary)

    • Do not send video streams if REMB is low (to prioritise audio)
    • Auto downgrade

  • Measure video stats (Cheng Wei)

  • Send PLI to sender

  • Accept PLI from receiver and forward to the sender with a threshold?

  • Calculate Sequence number of combined track correctly (need sophisticated algo) (Likai)

  • Browser side select resolution based on network condition (Likai)

Performance Issues

  • Compare H264 and VP8 encoding/decoding on Mac and iPhone (Likai)
    • Mac
    • iPhone
  • New API to forward RTP packet to a peer with different rtpmap.

Product Stage

  • Speaker mode and grid mode

  • Voice Activity Detection

    • Put the voice energy in the packet
    • The client should display the speaker with the most energy as the speaker
    • Limit the switching frequency (stay at least 3 seconds)

  • real stopping video (similar to stop audio)

  • grid mode we can select or pin a peer

  • Server-side video quality switching

    As a sender

    • limit quality of video sent based on REMB received
      • distribute video bandwidth based on REMB
      • incorporate audio energy into calculation

    As a receiver

    • should send correct REMB to sender (need to decide which metric to use)
    • Separate REMB calculation algo into different interface.
    • need to auto switch when one of the 3 streams stops.

  • Port over Seameet features

    • multiple rooms support

  • Bug fixes

    • Pausing video when using Safari causes audio packets being dropped by the server
    • Disabling Video will not turn off the camera on indicator
    • No uplink REMB estimation for muted clients
    • A few short spining in Safari
    • Lose local video on iPhone when the screen is locked and then wakes up
    • Uplink REMB estimation on iOS Safari
    • Cannot establish RTCConnection if there's no mic or camera
    • Cannot remove disconnected users if that user hasn't established RTCConnection
Select a repo