Simulcast Improvements

Simulcast has landed! This a list of improvements I think would be great to see before /v3 is tagged.

Support SSRC Based Simulcast
FireFox uses SSRCes instead of rid and mid. We should support this as well.
Refactor/Cleanup RTPReceiver
Rename trackStreams -> readerStreams. A lot more stuff will fall into place when multiple onTrack paths are removed.
Update examples/simulcast to respond to remote Session Description
Currently the example is tightly coupled with the frontend code. It would be nice if the backend dynamically constructed tracks depending on how many Simulcast layers the offerer is sending.
Implementing ReadRTCP for Simulcast Streams
Update Read to allow users to pass in the SSRC as well? Maybe we don't break the API and assume readerStreams[0] if no SSRC is passed? 6540022
Remove multiple OnTrack paths
Remove the goroutine from startReceiver. Everything should instead just react it incoming SSRCes. We could rename handleUndeclaredSSRC to be something like onNewIncomingSSRC.
E2E Tests
Currently we just have unit testing, but no examples of E2E testing. Would be fine to send packets manually now. If easy might be worth adding proper Simulcast client support.
Fix Simulcast Probing deadlock
Currently if a sender doesn't send enough packets handleUndeclaredSSRC can hang forever. This prevents probing taking place on other streams. We should fire a goroutine per stream we want to probe. 70fb90c

@jbrady42

@jbrady42 do you have any bandwidth to help with these? (if not that is ok!)

Fantastic work on the Simulcast so far, I was super excited to merge it :) I tried to cleanup stuff as I went.

I am taking this, will work on this during the weekend!

Fix Simulcast routine leak

Issue/sub-issue

tweaking the simulcast example so it doesn't feed the media back to the browser causes an issue where OnTrack has blank Rids for the simulcast tracks

Correct output from simulcast

Connection State has changed connected
Track has started
Sending pli for stream with rid: "f", ssrc: 610171300
Sending remb for stream with rid: "f", ssrc: 610171300
Track has started
Track has started
Sending pli for stream with rid: "f", ssrc: 610171300
Sending remb for stream with rid: "f", ssrc: 610171300
Sending pli for stream with rid: "h", ssrc: 4056720671

simulcast that demos bug with blank/empty Rids:

Connection State has changed connected
Track has started  rid is
Track has started  rid is
Sending pli for stream with rid: "", ssrc: 133249352
Sending remb for stream with rid: "", ssrc: 133249352
Sending pli for stream with rid: "", ssrc: 3288342995
Sending remb for stream with rid: "", ssrc: 3288342995
Track has started  rid is
Sending pli for stream with rid: "", ssrc: 133249352
Sending remb for stream with rid: "", ssrc: 133249352

Repo with issue, last commit is changes to realize the bug:
(look at last commit to see my small changes vs. the vanilla simulcast)
https://github.com/cameronelliott/webrtc.empty-rids

Jsfiddle needed for issue: (also in the repo linked)
https://jsfiddle.net/camfoo/p2xn4kw5/2/

@cameronelliott

@cameronelliott
Hi! I'm not familiar with open source and English isn't my first language, so please understand.. 😢

I think that happens when there is only one media on SDP.
In case of simulcast, on sdp only one media(video) is described but actually more than one(depending on how many encodings)
Check this peerconnection code.
It doesn't create ssrcs we need(it is optional according to this issue).

If you disabled above peerConnection code, it gets proper rid and creates ssrc.

Add E2E RID Simulcast test

Implement SSRC Based Simulcast

Improve SSRC Based Simulcast test

Move Simulcast SDP Modification into helper

Add E2E RID Simulcast test

Implement SSRC Based Simulcast

Improve SSRC Based Simulcast test

Move Simulcast SDP Modification into helper

Navigation Menu

Repository navigation

Simulcast Improvements #1345

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Simulcast Improvements #1345

Description

Activity

Sean-Der commented on Jul 24, 2020

Sean-Der commented on Oct 3, 2020

cameronelliott commented on Jan 5, 2021

jongwow commented on Feb 15, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions