SFU integration: renegotiation client + iceServers passthrough#551
SFU integration: renegotiation client + iceServers passthrough#551HexaField wants to merge 4 commits into
Conversation
✅ Deploy Preview for fluxsocial-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for fluxdocs failed. Why did it fail? →
|
📝 WalkthroughWalkthroughIntroduces SFU (Selective Forwarding Unit) support to WebRTC calls with a new SfuManager module for topology resolution and participant tracking, UI components for SFU indicators and quality selection, and video layout enhancements to automatically switch to Focused layout when participants exceed 8. Changes
Sequence Diagram(s)sequenceDiagram
participant Caller as Caller/Local
participant SfuMgr as SfuManager
participant Nbhood as NeighbourhoodProxy
participant GraphQL as GraphQL API
participant RemoteGW as Remote Gateway/SFU
Caller->>SfuMgr: join(localStream)
SfuMgr->>SfuMgr: Create RTCPeerConnection
SfuMgr->>SfuMgr: Add local tracks (simulcast)
SfuMgr->>GraphQL: callJoin(roomId, offer)
GraphQL->>RemoteGW: Forward offer
RemoteGW->>RemoteGW: Process offer, generate answer
RemoteGW->>GraphQL: Return answer + participants
GraphQL->>SfuMgr: Receive answer + remote participants
SfuMgr->>SfuMgr: Set remote description
SfuMgr->>SfuMgr: Track incoming remote streams
SfuMgr->>Caller: Emit participant-joined, stream-added
Caller->>SfuMgr: leave()
SfuMgr->>GraphQL: callLeave(roomId)
SfuMgr->>SfuMgr: Close peer connection
SfuMgr->>Caller: Emit participant-left
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (4)
app/src/components/call/controls/QualitySelector.vue (2)
26-26:QualityPreferencetype duplicated - consider sharing with SfuManager.The
QualityPreferencetype is defined here locally, butSfuManager.setQualityPreferencealso expects the same union type. Consider exporting this type from the webrtc package to ensure consistency.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/src/components/call/controls/QualitySelector.vue` at line 26, The QualityPreference union is duplicated here; instead export the shared type from the webrtc package and import it where needed so SfuManager.setQualityPreference and this component use the same type. Update the declaration of QualityPreference to remove the local definition, import the exported type (e.g., QualityPreference) from the webrtc package, and adjust any references in QualitySelector.vue and SfuManager to use the single exported symbol to keep types consistent.
7-19: Dropdown lacks click-outside-to-close behavior.The dropdown opens on button click but doesn't close when clicking outside. This is a common UX pattern that users expect.
♻️ Suggested approach using a click-outside directive or composable
+import { onMounted, onUnmounted } from 'vue'; + +const dropdownRef = ref<HTMLElement | null>(null); + +function handleClickOutside(event: MouseEvent) { + if (dropdownRef.value && !dropdownRef.value.contains(event.target as Node)) { + isOpen.value = false; + } +} + +onMounted(() => document.addEventListener('click', handleClickOutside)); +onUnmounted(() => document.removeEventListener('click', handleClickOutside));And in template:
- <div class="quality-selector" v-if="showSelector"> + <div class="quality-selector" v-if="showSelector" ref="dropdownRef">🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/src/components/call/controls/QualitySelector.vue` around lines 7 - 19, The dropdown in QualitySelector.vue uses isOpen and selectQuality but never closes when clicking outside; add a click-outside handler (either a v-click-outside directive or a small composable) that listens for document click events and sets isOpen = false when the click target is outside the component root; register the listener on mount and remove it on unmount (or use the directive lifecycle) and ensure the root element or the element wrapping the template (the element that currently contains the v-if="isOpen") is used to detect "outside" so selectQuality and options behavior remains unchanged.packages/webrtc/src/SfuManager.ts (1)
102-107: Missingoff()method to unsubscribe event listeners.The event system provides
on()but no way to remove listeners. This can cause memory leaks when consumers need to clean up subscriptions.♻️ Proposed addition
on(event: SfuEvent, callback: SfuEventCallback): void { if (!this.callbacks.has(event)) { this.callbacks.set(event, []); } this.callbacks.get(event)!.push(callback); } + + off(event: SfuEvent, callback: SfuEventCallback): void { + const cbs = this.callbacks.get(event); + if (cbs) { + const idx = cbs.indexOf(callback); + if (idx !== -1) cbs.splice(idx, 1); + } + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/webrtc/src/SfuManager.ts` around lines 102 - 107, The SfuManager currently exposes on(event: SfuEvent, callback: SfuEventCallback) but lacks a corresponding off to remove listeners, which can leak memory; add an off(event: SfuEvent, callback?: SfuEventCallback) method on the SfuManager that locates the callbacks array in this.callbacks for the given event, and if a callback is provided removes only that function (filtering or splicing) and if no callback is provided clears the entire array (or deletes the map entry); ensure you handle the case where the event key is missing and after removing the last listener delete the map entry to keep this.callbacks clean.app/src/components/call/widgets/SfuSettingsPanel.vue (1)
37-43: Designated peer may go offline before save - no validation.The dropdown shows online agents at mount time, but there's no refresh mechanism or validation that the selected peer is still online when saving. Consider either refreshing the list periodically or validating at save time.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@app/src/components/call/widgets/SfuSettingsPanel.vue` around lines 37 - 43, The designatedPeer dropdown uses the initial members list but lacks validation on save; update the SfuSettingsPanel.vue to (1) refresh or re-fetch the members list before persisting and/or periodically (e.g., on mount and before save) so the options reflect current online agents, and (2) validate in the save handler (the component's save/submit method) that config.designatedPeer still exists in members and is online—if not, clear config.designatedPeer or surface a validation error and prevent save. Reference the select binding config.designatedPeer and the members array to implement these checks and the re-fetch call.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@app/src/components/call/composables/useVideoLayout.ts`:
- Around line 88-96: The watcher that auto-switches layout ignores user choice:
modify the logic around the watch of peers.value.length so it checks a "user
selected" flag before calling uiStore.setVideoLayout; e.g., add or use a boolean
like userSelectedVideoLayout (or a method on uiStore such as
uiStore.isUserSelectedLayout()) and only call
uiStore.setVideoLayout(videoLayoutOptions[2]) when that flag is false, and
ensure any UI action that sets a layout (the existing setter used by users /
uiStore.setVideoLayout) toggles that flag to true so subsequent peer-count
changes do not override manual selections.
In `@app/src/components/call/widgets/SfuSettingsPanel.vue`:
- Around line 81-84: The onMounted handler currently swallows errors in two
empty catch blocks; update the catches for the calls to
props.neighbourhood.sfuConfig() and props.neighbourhood.onlineAgents() to log
the thrown errors (including context) instead of ignoring them — e.g., catch
(err) { console.error("Failed to load SFU config", err) } and catch (err) {
console.error("Failed to fetch online agents", err) } while leaving the existing
assignments to config and members unchanged; locate these in the onMounted block
and replace the empty catch bodies accordingly.
In `@packages/webrtc/src/SfuManager.ts`:
- Around line 124-130: Hardcoded TURN/STUN credentials are present in the
RTCPeerConnection iceServers config in SfuManager.ts (the const pc creation);
replace the embedded usernames/credentials and server URLs by reading them from
configuration/environment variables (e.g., process.env.TURN_URL,
process.env.TURN_USERNAME, process.env.TURN_CREDENTIAL and fallbacks for STUN),
and construct the iceServers array dynamically so SfuManager (the
RTCPeerConnection instantiation) uses the provided config at runtime rather than
hardcoded values; ensure sensible fallback behavior or skip TURN entry if env
vars are not present.
- Around line 181-186: The ice-gathering await in SfuManager (inside the join()
flow) can hang indefinitely because the Promise waiting on pc.iceGatheringState
=== "complete" has no timeout; update that Promise to include a configurable
timeout (e.g., default ~5–10s) so it resolves or rejects after the timeout, and
ensure you clear the timeout and remove/clear pc.onicegatheringstatechange when
finished to avoid leaks; reference the existing Promise block that checks
pc.iceGatheringState and pc.onicegatheringstatechange and add the timeout,
cleanup, and a clear resolution path on timeout so join() cannot hang forever.
- Around line 216-220: The destroy() implementation only closes the local
PeerConnection and clears callbacks/participants but never notifies the SFU
server; update destroy() to invoke the existing leave() (or callLeave) flow
before closing this.state.peerConnection so the server session is cleaned up —
call and await this.leave() (or call this.callLeave() if leave is internal) and
handle/rethrow/log errors, then proceed to close this.state.peerConnection,
clear this.callbacks and this.state.participants to avoid orphaned sessions.
- Around line 161-170: The participant DID is incorrectly set to the browser
MediaStream id in the pc.ontrack handler—update the SFU signaling and SfuManager
to use a real participant-to-stream mapping: extend the CallSession response (or
SDP/track metadata) to include a mapping of streamId (or trackId) →
participantId, store that mapping on the SfuManager (e.g., this.session or
this.state as streamIdToParticipantId), and change the pc.ontrack flow that
constructs SfuParticipantState to lookup the real DID via that mapping (use the
new mapping key when creating the SfuParticipantState in SfuManager’s pc.ontrack
handler instead of stream.id); ensure fallback logging if a mapping is missing
so you can detect unmapped streams.
---
Nitpick comments:
In `@app/src/components/call/controls/QualitySelector.vue`:
- Line 26: The QualityPreference union is duplicated here; instead export the
shared type from the webrtc package and import it where needed so
SfuManager.setQualityPreference and this component use the same type. Update the
declaration of QualityPreference to remove the local definition, import the
exported type (e.g., QualityPreference) from the webrtc package, and adjust any
references in QualitySelector.vue and SfuManager to use the single exported
symbol to keep types consistent.
- Around line 7-19: The dropdown in QualitySelector.vue uses isOpen and
selectQuality but never closes when clicking outside; add a click-outside
handler (either a v-click-outside directive or a small composable) that listens
for document click events and sets isOpen = false when the click target is
outside the component root; register the listener on mount and remove it on
unmount (or use the directive lifecycle) and ensure the root element or the
element wrapping the template (the element that currently contains the
v-if="isOpen") is used to detect "outside" so selectQuality and options behavior
remains unchanged.
In `@app/src/components/call/widgets/SfuSettingsPanel.vue`:
- Around line 37-43: The designatedPeer dropdown uses the initial members list
but lacks validation on save; update the SfuSettingsPanel.vue to (1) refresh or
re-fetch the members list before persisting and/or periodically (e.g., on mount
and before save) so the options reflect current online agents, and (2) validate
in the save handler (the component's save/submit method) that
config.designatedPeer still exists in members and is online—if not, clear
config.designatedPeer or surface a validation error and prevent save. Reference
the select binding config.designatedPeer and the members array to implement
these checks and the re-fetch call.
In `@packages/webrtc/src/SfuManager.ts`:
- Around line 102-107: The SfuManager currently exposes on(event: SfuEvent,
callback: SfuEventCallback) but lacks a corresponding off to remove listeners,
which can leak memory; add an off(event: SfuEvent, callback?: SfuEventCallback)
method on the SfuManager that locates the callbacks array in this.callbacks for
the given event, and if a callback is provided removes only that function
(filtering or splicing) and if no callback is provided clears the entire array
(or deletes the map entry); ensure you handle the case where the event key is
missing and after removing the last listener delete the map entry to keep
this.callbacks clean.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a118d53c-76ae-4915-a6bf-1f6b82055648
📒 Files selected for processing (6)
app/src/components/call/composables/useVideoLayout.tsapp/src/components/call/controls/QualitySelector.vueapp/src/components/call/widgets/SfuIndicator.vueapp/src/components/call/widgets/SfuSettingsPanel.vuepackages/webrtc/src/SfuManager.tspackages/webrtc/src/index.ts
8411a7e to
cac44af
Compare
7347b24 to
5ef8e99
Compare
Replaces the stale feat/sfu-integration with a clean forward-port on top of dev. This commit is the foundation — pure WebRTC + SDK plumbing. The store / UI / settings panel forward-port follows in later commits on this branch (those require heavy mechanical conflict resolution against dev's webrtcStore.ts and the AD4M launcher settings shell). Contents: - packages/webrtc/src/SfuManager.ts (~410 LOC): - SfuManager class + resolveTopology() standalone helper. - Connects via RTCPeerConnection with simulcast (high/medium/low). - Stream correlation via streamMapping order + knownParticipantDids. - Cascade failover on ICE connection state. - Server-initiated SDP renegotiation handling. - Event-emitter API matching the existing mesh WebRTCManager. - packages/webrtc/src/SfuManager.test.ts: vitest suite covering resolveTopology table (mesh/sfu/cascaded across config+participant cross-product) and the SfuManager event surface. SDK call shape adjustments for the WS RPC migration: - Imports `SfuConfig` + `CallSessionInfo` from top-level @coasys/ad4m (was deep-import from NeighbourhoodClient.ts before the apollo→ws migration). - `neighbourhood.sfuConfig()` → `sfuConfig(neighbourhoodUrl)`. - `resolveTopology(neighbourhood, count)` → `resolveTopology(neighbourhood, neighbourhoodUrl, count)`. Test calls updated. - `callAnswerServerOffer(roomId, sdp)` → `callAnswerServerOffer( neighbourhoodUrl, roomId, sdp)`. Still to forward-port on this branch (separate commits): - app/src/stores/webrtcStore.ts SFU integration (~500 LOC diff vs dev; heavy mechanical resolution). - Call-window UI components (SfuIndicator, SfuSettingsPanel, QualitySelector) — settings shell may have moved. - TURN credential fetch + redirect handling pathways. - Build script extension for ad4m --features. Companion PRs: - coasys/ad4m feat/embedded-sfu (forward-ported executor + WS RPC). - coasys/ad4m-wind-tunnel feat/sfu-webrtc-scenarios (new harness + scenarios). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
f76d3cc to
694c575
Compare
Adds the complete scenario matrix for the WebRTC + SFU work:
mesh fundamentals, SFU topology, mid-call transitions, faults, and
scale. 25 scenarios total, each runs end-to-end against an ad4m
executor with the embedded SFU service (coasys/ad4m feat/embedded-sfu).
## Harness additions
- src/peer.ts: WebRtcPeer wrapper around @roamhq/wrtc. Synthetic
media (per-peer tone audio + counter video), 1 Hz getStats()
sampling, EventEmitter for ICE state / remote-track / stats.
Constructor takes iceTransportPolicy ('all' | 'relay') and
recvSlots (pre-allocates N recv-only audio + video transceivers
in the initial offer). Stats aggregation now resolves the
nominated candidate-pair's local/remote candidate type.
- src/mesh.ts: MeshHost — each participant maintains N-1 separate
RTCPeerConnections, one per remote. Single-PC peers can't act as
both offerer and answerer in mesh > 2 (DTLS role conflict).
connectAll() pairs every host's a-side and b-side connections.
- src/net.ts: tc qdisc netem wrapper for F1/F2 packet loss.
Detects sudo -n availability; no-ops on macOS.
- src/cascade.ts: startCluster() spawns N ad4m executors on
consecutive ports, calls sfu.enableCascade on each.
cluster.announceCount() pushes per-node count updates via
sfu.cascadeAnnounce. Binary path overridable via AD4M_EXECUTOR_BIN.
- src/peer-server.ts: lightweight HTTP harness for W1M's
multi-machine path.
- src/run-webrtc.ts: standalone runner. Generates an agent before
T/M/F/S scenarios (SFU handlers require ctx.user_did).
Recognises which scenarios need an executor and gates accordingly.
120 s race fallback on the slow agent.generate path.
- src/client.ts: exposes a public call() method for raw WS RPC
dispatch so scenarios can hit handlers the typed client doesn't
wrap.
## Scenarios
| Bucket | Scenarios |
|---|---|
| W mesh fundamentals | W1, W1M, W2, W3, W4, W5 |
| T SFU topology | T1, T2, T3, T4, T5 |
| M mid-call transitions | M1, M2, M3, M4 |
| F faults | F1, F2, F3, F4, F5, F6, F7 |
| S scale | S1, S2, S3 |
W5/F3 use coturn (set TURN_URL + creds). F1/F2 use tc qdisc netem
on Linux. T3/T4/M3/F4/S2/S3 spawn multi-executor clusters via
src/cascade.ts. W1M needs AD4M_REMOTE_PEER_URL pointed at a
remote peer-server.
## Headline scaling
Per-host upload, single-host loopback runs:
mesh N=2: ~100 KB
mesh N=3: ~200 KB (2.00x)
mesh N=4: ~300 KB (3.00x)
SFU N=5: ~188 KB
SFU N=10: ~188 KB (sd <1 KB)
SFU N=20: ~186 KB (sd ~3 KB, 0 packets lost)
Mesh grows O(N-1); SFU stays flat O(1). M1 mesh→SFU promotion
drops per-host upload to 0.32x of the mesh phase (68% saving).
## Known limitations
- T4/S2 distribution is skewed without real-time gossip — peers
all land, but stale cross-node counts in the static cascade view
cause uneven landings.
- The SFU's server-pushed SDP renegotiation isn't wired into the
wind tunnel client; T1/T2/S1 use recvSlots=N-1 as a workaround
so downloadMean reflects forwarded media instead of 0.
## Companion PRs
- coasys/ad4m#712 — executor + SFU module + WS RPC handlers + TS
SDK + boot fixes + cascade admin endpoints.
- coasys/flux#551 — SfuManager + tests + webrtcStore integration
+ UI components.
…itative SfuConfig.iceServers (from the neighbourhood Social DNA) is the production-grade source for TURN credentials. When set, SfuManager maps the list straight into RTCConfiguration.iceServers — host apps can rotate creds server-side without a client redeploy. Resolution order: 1. sfuConfig.iceServers (production) 2. iceConfig param (test override) 3. DEFAULT_ICE_SERVERS (public STUN fallback)
The previous SfuManager called subscribeCallRenegotiationOffer with
a payload shape `{reason, sdpOffer, roomId}` that didn't match the
server-side `SfuCallRenegotiationOffer` struct. The ad4m SDK now
exposes the matching surface; align Flux to it:
- Use the `{targetDid, neighbourhoodUrl, roomName, sdpOffer}` payload
produced by `crate::sfu::types::SfuCallRenegotiationOffer`.
- Defensive double-filter on neighbourhoodUrl + roomName in case
the events_ws per-DID fanout ever regresses.
- Track the unsubscribe handle in `renegotiationUnsubscribe` and
release it in `leave()` so listeners don't accumulate across
join/leave cycles.
Summary
Wires Flux's
SfuManagerto the renegotiation pipeline shipped oncoasys/ad4m#712:subscribeCallRenegotiationOfferis now consumed via the SDK's typed surface. Payload aligned tocrate::sfu::types::SfuCallRenegotiationOffer({targetDid, neighbourhoodUrl, roomName, sdpOffer}).neighbourhoodUrl+roomNamein the subscriber so any future events_ws fanout regression can't push a stale renegotiation into the wrong room.renegotiationUnsubscribeand released inleave()so listeners don't leak across join/leave cycles.SfuManagerconstructor takes an optionalSfuConfig— when present, itsiceServersare authoritative over the per-appiceConfig, so the SFU host can rotate TURN creds.Dependencies
coasys/ad4m#712must be merged first — exposesNeighbourhoodProxy.subscribeCallRenegotiationOfferand the renegotiation payload shape.Test plan
🤖 Generated with Claude Code