Skip to content

Support simulcast from "SIM" ssrc-group - #3396

Open
fancycode wants to merge 6 commits into
pion:mainfrom
fancycode:simulcast-ssrc-group-sim
Open

Support simulcast from "SIM" ssrc-group#3396
fancycode wants to merge 6 commits into
pion:mainfrom
fancycode:simulcast-ssrc-group-sim

Conversation

@fancycode

Copy link
Copy Markdown

Required to correctly handle offer SDPs like this (taken from Nextcloud Talk with Chrome):

v=0
o=- 3304229686190011165 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1 2
a=extmap-allow-mixed
a=msid-semantic: WMS 3b43bf3b-262d-45df-a91d-c6f4dcf00b4e
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 9 0 8 13 110 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Ezq+
a=ice-pwd:3lSMQj0krWe/iGb7ex4U+/2d
a=ice-options:trickle
a=fingerprint:sha-256 B1:4D:7F:6B:5A:88:67:5D:D1:9E:D2:1C:FB:22:71:21:9B:09:B4:8D:C0:52:DC:6F:C9:A7:2D:D2:08:44:B8:70
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=sendonly
a=msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e cac04ecb-ee49-46c0-a5b9-189a33774647
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:126 telephone-event/8000
a=ssrc:1575337190 cname:4Cqdy6kA30DYfwYF
a=ssrc:1575337190 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e cac04ecb-ee49-46c0-a5b9-189a33774647
m=video 9 UDP/TLS/RTP/SAVPF 96 97 103 104 107 108 109 114 115 116 117 118 39 40 45 46 98 99 100 101 119 120 121
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Ezq+
a=ice-pwd:3lSMQj0krWe/iGb7ex4U+/2d
a=ice-options:trickle
a=fingerprint:sha-256 B1:4D:7F:6B:5A:88:67:5D:D1:9E:D2:1C:FB:22:71:21:9B:09:B4:8D:C0:52:DC:6F:C9:A7:2D:D2:08:44:B8:70
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:103 H264/90000
a=rtcp-fb:103 goog-remb
a=rtcp-fb:103 transport-cc
a=rtcp-fb:103 ccm fir
a=rtcp-fb:103 nack
a=rtcp-fb:103 nack pli
a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:104 rtx/90000
a=fmtp:104 apt=103
a=rtpmap:107 H264/90000
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:108 rtx/90000
a=fmtp:108 apt=107
a=rtpmap:109 H264/90000
a=rtcp-fb:109 goog-remb
a=rtcp-fb:109 transport-cc
a=rtcp-fb:109 ccm fir
a=rtcp-fb:109 nack
a=rtcp-fb:109 nack pli
a=fmtp:109 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:114 rtx/90000
a=fmtp:114 apt=109
a=rtpmap:115 H264/90000
a=rtcp-fb:115 goog-remb
a=rtcp-fb:115 transport-cc
a=rtcp-fb:115 ccm fir
a=rtcp-fb:115 nack
a=rtcp-fb:115 nack pli
a=fmtp:115 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:116 rtx/90000
a=fmtp:116 apt=115
a=rtpmap:117 H264/90000
a=rtcp-fb:117 goog-remb
a=rtcp-fb:117 transport-cc
a=rtcp-fb:117 ccm fir
a=rtcp-fb:117 nack
a=rtcp-fb:117 nack pli
a=fmtp:117 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:118 rtx/90000
a=fmtp:118 apt=117
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=fmtp:45 level-idx=5;profile=0;tier=0
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:119 red/90000
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=119
a=rtpmap:121 ulpfec/90000
a=ssrc:1109168805 cname:4Cqdy6kA30DYfwYF
a=ssrc:1109168805 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc:3192552408 cname:4Cqdy6kA30DYfwYF
a=ssrc:3192552408 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc:3046756662 cname:4Cqdy6kA30DYfwYF
a=ssrc:3046756662 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc:1315488727 cname:4Cqdy6kA30DYfwYF
a=ssrc:1315488727 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc:274207894 cname:4Cqdy6kA30DYfwYF
a=ssrc:274207894 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc:547186941 cname:4Cqdy6kA30DYfwYF
a=ssrc:547186941 msid:3b43bf3b-262d-45df-a91d-c6f4dcf00b4e a1c1d396-6eda-4d34-984a-f5f2eb70ece7
a=ssrc-group:SIM 1109168805 3046756662 274207894
a=ssrc-group:FID 1109168805 3192552408
a=ssrc-group:FID 3046756662 1315488727
a=ssrc-group:FID 274207894 547186941
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:Ezq+
a=ice-pwd:3lSMQj0krWe/iGb7ex4U+/2d
a=ice-options:trickle
a=fingerprint:sha-256 B1:4D:7F:6B:5A:88:67:5D:D1:9E:D2:1C:FB:22:71:21:9B:09:B4:8D:C0:52:DC:6F:C9:A7:2D:D2:08:44:B8:70
a=setup:actpass
a=mid:2
a=sctp-port:5000
a=max-message-size:262144

Without this change, only a single track with the first SSRC is signaled from PeerConnection.OnTrack.

@codecov

codecov Bot commented Mar 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.56338% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.41%. Comparing base (9654161) to head (3637ef3).

Files with missing lines Patch % Lines
rtpreceiver.go 0.00% 22 Missing ⚠️
peerconnection.go 33.33% 2 Missing and 2 partials ⚠️
sdp.go 95.34% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3396      +/-   ##
==========================================
- Coverage   85.66%   85.41%   -0.25%     
==========================================
  Files          81       81              
  Lines        9819     9880      +61     
==========================================
+ Hits         8411     8439      +28     
- Misses        992     1024      +32     
- Partials      416      417       +1     
Flag Coverage Δ
go 85.41% <60.56%> (-0.25%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fancycode
fancycode force-pushed the simulcast-ssrc-group-sim branch from 281474f to 4a5e4c3 Compare March 19, 2026 19:28
@fancycode

Copy link
Copy Markdown
Author

Updated the commit message to fix the metadata linter.

Should I try to refactor trackDetailsFromSDP to reduce the complexity? I see that there already are some nolint flags.

@fancycode
fancycode force-pushed the simulcast-ssrc-group-sim branch from 4a5e4c3 to 3637ef3 Compare May 3, 2026 19:10
@fancycode

Copy link
Copy Markdown
Author

Rebased and added method to read RTCP from simulcast tracks based on SSRC (instead of RID).

Anything else I should do to help get this landed? Thanks!

@JoTurk
JoTurk self-requested a review May 3, 2026 19:12
@JoTurk

JoTurk commented May 3, 2026

Copy link
Copy Markdown
Member

@fancycode I'll take a look. thank you.

@fancycode

Copy link
Copy Markdown
Author

Friendly ping. Should I rebase to latest main?

@JoTurk

JoTurk commented Aug 17, 2026

Copy link
Copy Markdown
Member

@fancycode I'm so sorry, I got busy with the DTLS 1.3 project, I'll try to get this reviewed after my work today.

@fancycode

Copy link
Copy Markdown
Author

No worries, just wanted to check if there is something I could do to help.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for SSRC-based simulcast SDP groups and per-encoding RTCP access.

Changes:

  • Parses SIM SSRC groups with RTX/FEC mappings.
  • Adds SSRC-specific simulcast RTCP read APIs.
  • Updates receiver detection and SDP tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
sdp.go Parses SIM groups and encoding repair SSRCs.
sdp_test.go Tests SIM, RTX, and FEC parsing.
rtpreceiver.go Adds SSRC-specific RTCP reads.
peerconnection.go Recognizes multiple repair SSRCs.
errors.go Adds an SSRC lookup error.
constants.go Adds SIM semantics and updated guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdp.go
}
}

tracksInMediaSection = []trackDetails{simulcastTrack}
Comment thread rtpreceiver.go
Comment on lines +329 to +330
// ReadSimulcastSSRC reads incoming RTCP for this RTPReceiver for given SSRC.
func (r *RTPReceiver) ReadSimulcastSSRC(b []byte, ssrc SSRC) (n int, a interceptor.Attributes, err error) {
Comment thread sdp.go Outdated
continue
}

ssrcs = append(ssrcs, SSRC(ssrc))
Comment thread sdp.go Outdated
Comment on lines +263 to +267
for rtx, base := range rtxRepairFlows {
baseSsrc := SSRC(base) //nolint:gosec // G115
if pos := slices.Index(ssrcs, baseSsrc); pos != -1 {
repairSsrc := SSRC(rtx) //nolint:gosec // G115
simulcastTrack.rtxSsrc[pos] = &repairSsrc
Comment thread sdp_test.go Outdated
Comment on lines +737 to +741
tracks := trackDetailsFromSDP(log, descr)
assert.Equal(t, 4, len(tracks))
assert.Equal(t, []SSRC{3000, 4000, 5000}, tracks[0].ssrcs)
assert.Equal(t, []SSRC{3000, 5000}, tracks[1].ssrcs)
assert.Equal(t, []SSRC{3000, 4000, 5000}, tracks[2].ssrcs)
@JoTurk

JoTurk commented Aug 17, 2026

Copy link
Copy Markdown
Member

@fancycode can you please look at copilot comments? just mark anything you think is made-up as resolved. and I'll try to review it tonight, thank you.

@fancycode
fancycode force-pushed the simulcast-ssrc-group-sim branch from 3637ef3 to f147538 Compare August 17, 2026 21:26
@fancycode

Copy link
Copy Markdown
Author

Thanks, I rebased and addressed all issues found by Copilot but didn't add a end-to-end test as suggested in #3396 (comment).
My PR only adds parsing of SIM ssrc-groups but not writing, and I couldn't figure out how to create a proper offer manually that will also send ssrc-based simulcast streams.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants