feat(spf): capability probing#1676
Draft
cjpillsbury wants to merge 11 commits into
Draft
Conversation
Add canPlayTrack in media/dom/capabilities.ts — the synchronous codec half of capability probing. Builds a track's MIME codec string and checks MediaSource.isTypeSupported, memoized by MIME (codec support is static per environment). Tracks with no mimeType or no declared CODECS are unprobeable and pass through as playable; the late createSourceBuffer check stays their backstop. The DOM-free CanPlayTrack predicate type lives in media/types so DOM-free behaviors can consume it while the DOM implementation stays in media/dom. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…witching
Add excludeUnplayableTracks to track-switching's hard-constraints
pre-pass, pooled beside excludeFailedCdns and shared by switchVideoTrack
/ switchAudioTrack. It reads an injected canPlayTrack probe and drops
undecodable renditions before the rule chain, so an unplayable variant
(e.g. HEVC on a browser without it) is pruned upstream instead of
surviving to fail late at createSourceBuffer.
Surface the Phase-6-partial not-ready state: per-type
noPlayable{Video,Audio}Tracks flags fire when a non-empty candidate set
prunes to empty (every rendition undecodable, or every CDN cooled down).
Cause-agnostic by design; the variant passes its own signal in so the
helper stays free of the slot's key, and the flag is a required signal
on the behavior's declared state (an optional one collapses under the
composition's conflict check).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Default config.canPlayTrack to the DOM canPlayTrack in finalConfig (the
seam consumers override to force-exclude a codec) and expose per-type
noPlayable{Video,Audio}Tracks on SimpleHlsEngineState. Codec filtering
is now live in createSimpleHlsEngine: mixed-codec sources select a
decodable rendition, and an all-undecodable source surfaces the
no-playable flag with no pick.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Advance status to partial / definition to sketched. Mark the codec primitive + multivariant CODECS filtering implemented and unsupported-case surfacing partial; add Implementation surface + Verification sections; record the resolved open questions (config-predicate vs slot-writer split, lazy+memoized probing, cause-agnostic no-playable surfacing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
parseCodecs only matched mp4a.* as audio, so a CODECS string like "avc1.640020,ac-3" (Mux's 5.1 surround shape) parsed with no audio codec — the audio rendition ended up with codecs: [], which capability probing treats as unprobeable and passes through. The undecodable AC-3 5.1 track then stayed selected and its audio buffer failed to materialize (video-only playback, audio silently dropped). Recognize ac-3 / ec-3 / ac-4 / opus / flac / dts / alac / vorbis (case-insensitive) as audio. Smoke-tested against a real AC-3 5.1 Mux stream in Chromium: the AC-3 rendition is now pruned and AAC stereo is selected, so audio plays. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ivariant parser
HLS lists one video rendition across several EXT-X-STREAM-INF entries —
one per audio group it can pair with — all sharing the same media-
playlist URI. The parser built one video track per STREAM-INF, so a
source like Mux's enable_51_surround (7 renditions × {5.1, stereo})
surfaced 14 video tracks for 7 real renditions, doubling ABR's candidate
set with identical-URI duplicates.
De-duplicate video tracks by playlist URI, accumulating every advertised
audio group into VideoTrack.audioGroupIds (was the single audioGroupId,
read by nothing downstream). Keep the lowest combined BANDWIDTH across
duplicates as the closest proxy to video-only, which is what ABR ranks
on. Redundant-stream renditions live at distinct per-CDN URIs, so they
correctly stay separate — only the same-URI cross-product merges.
Verified on the live AC-3 5.1 Mux stream: 14 -> 7 video tracks, each
carrying [audio-51-0, audio-hi-0]; capability probing + playback
unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The multivariant parser hardcoded channels: 2 on every audio rendition, so a 5.1 surround track (CHANNELS="6") was modelled as stereo. Read the attribute via getInt (its leading-integer parse also handles spatial forms like "16/JOC"), falling back to the stereo default when absent. Verified on the live AC-3 5.1 Mux stream: the surround rendition now reports channels: 6. This is the signal 5.1-surround-selection will key off to prefer the multichannel rendition when the environment supports it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ert them unplayable HLS non-fMP4 renditions carry the same CODECS (avc1/mp4a) as fMP4 but no EXT-X-MAP init segment. The parser hardcoded the fMP4 default video/mp4 / audio/mp4, so a TS or raw-AAC source was mislabeled — a TS segment chokes createSourceBuffer, and a raw .aac audio track claims to be playable fMP4 and then fails downstream, all with no upstream signal. Detect per media playlist: no EXT-X-MAP + a recognized segment extension (.ts -> video/mp2t, .aac -> audio/aac; high precision, fMP4 always has the map). resolve-track propagates the MIME to every rendition of the SAME type (applyContainerMimeType) — one resolved playlist relabels the type, pruned from a single fetch. Same-type only, never cross audio<->video (mixed-container sources exist, e.g. muxed-TS video + raw-.aac audio), which also keeps per-type resolutions' writes disjoint (no race). canPlayTrack asserts both video/mp2t and audio/aac unsupported (without consulting isTypeSupported, which false-positives on Chromium for both), so a non-fMP4 source surfaces noPlayable* (loud, observable) instead of stalling deep in the pipeline. TS can't be played here at all (no transmux). Raw AAC is a TEMPORARY limitation: the browser genuinely decodes it (Chrome/Safari), but our segment loader / append pipeline assumes an init segment; making it play (drop that assumption + bare-MIME probe/projection) is deliberately out of scope here and noted for follow-up. Validated in the SPF sandbox against Apple bipbop_4x3 (muxed-TS video + raw-.aac audio): video -> noPlayableVideoTracks, audio -> noPlayableAudioTracks; an audio-only .aac source likewise surfaces noPlayableAudioTracks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…bility-probing Generalize the container-detection notes from MPEG-TS to non-fMP4 (TS + raw ADTS AAC), both detected and asserted unplayable. Implementation surface + Verification updated; resolve the container-detection-scope open question (per-track-type, marked-unplayable, probe not trusted for either); record that raw AAC is browser-supported and playable once the pipeline's init-segment assumption is removed (follow-up under container-support). Apple bipbop smoke test noted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
createHlsAudioOnlyEngine composes switchAudioTrack (and its excludeUnplayableTracks constraint) but didn't default canPlayTrack or expose noPlayableAudioTracks, so capability probing — codec filtering and the TS / raw-AAC unplayable detection — was inert there. A raw-AAC audio-only source (the exact case the audio-only engine handles) would slip through unpruned instead of surfacing noPlayable. Mirror the default engine: add canPlayTrack to the audio-only config, default it to the DOM canPlayTrack in finalConfig, and add the noPlayableAudioTracks state slot. Adapters already forward ...config, so no adapter change. Test asserts the default prunes a raw-AAC source and surfaces noPlayableAudioTracks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The per-type noPlayableVideoTracks/noPlayableAudioTracks flags were write-only: nothing in spf/core/html/react/sandbox read them. They also stored a derivation (hadCandidates && candidates.length === 0) in a slot an effect wrote, rather than a computed — and dragged the `N extends NoPlayableKey = never` generic through setupTrackSwitching to keep the slot off the rule/base views. Remove the slots, the generic, the entry-cleanup, and the effect write from track-switching plus the two engine state shapes. An emptied candidate set still makes no pick; the late createSourceBuffer check stays as the structural backstop. Coverage of the surviving behavior is preserved: a behavior-level "makes no pick when the constraint prunes every rendition" test, plus the engine/audio-only integration tests re-pointed from the flag to the no-pick outcome. Update capability-probing.md: the no-playable surfacing moves from "Phase 6 partial — implemented" to removed/deferred, and two open questions are recorded — (1) surfacing "nothing playable" as a future fatal-vs-recoverable error (computed vs a dedicated behavior), and (2) sharing the constraint pre-pass with user-selection logic so a track list doesn't offer unplayable renditions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Base branch: targets
feat/spf-multi-cdn-failover(the sticky-CDN-pick PR), notmain — will repoint to main once that lands. The Files changed tab is
already scoped to just this PR's content.