Skip to content

Add support for Dash.js rendered subtitles #383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 55 commits into from
May 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
dd29347
Feat: Support MSE Subtitles for Low Latency
ryanmccartney Sep 18, 2024
f9b1227
Add tests for new subtitle type
ryanmccartney Sep 18, 2024
80d4851
Subtitles: low-latency docs
ryanmccartney Sep 19, 2024
0d68c69
Subtitles: low-latency docs
ryanmccartney Sep 23, 2024
033bcc3
Merge branch 'master' of https://github.com/ryanmccartney/bigscreen-p…
ryanmccartney Sep 23, 2024
6279f29
Don't require a captions object when dashSubtitles override is used
ryanmccartney Sep 24, 2024
db5e2d6
Add tests for DashSubtitles
ryanmccartney Sep 27, 2024
40d95d8
Merge branch 'bbc:master' into dash-subtitles
ryanmccartney Sep 27, 2024
3f65696
Rename DashSubs to EmbeddedSubs
ryanmccartney Oct 10, 2024
2e5d782
Merge branch 'master' into dash-subtitles
tom-coward Oct 23, 2024
4c6edf7
Fix merge (missing })
tom-coward Oct 23, 2024
485ca68
Wait for media player to be ready before enabling embedded subtitles
tom-coward Oct 29, 2024
591a553
Pass IMSCjs style options through BSP to embeddedsubtitles
ryanmccartney Feb 6, 2025
56c8386
Add customisation to embedded subtitles
ryanmccartney Feb 9, 2025
d6b0cc1
Merge branch 'master' of https://github.com/bbc/bigscreen-player into…
ryanmccartney Mar 4, 2025
629821a
Merge branch 'master' into embedded-subtitles
ryanmccartney Apr 28, 2025
6a605b8
Alias smp-imsc in dev and build
ryanmccartney Apr 28, 2025
4e08ddc
Add TTML Rendering Div if embedded
ryanmccartney Apr 28, 2025
04aceba
Add example subtitle using smp-imsc to embedded subs
ryanmccartney Apr 28, 2025
a78255b
Fix imports
ShiningTrapez May 1, 2025
74364cf
Change call order
ryanmccartney May 2, 2025
793a99d
Move div creation to mse-strategy
ryanmccartney May 2, 2025
e132fa3
Add tests for div creation
ryanmccartney May 2, 2025
ce6d72b
Add observer and cleanup afterwards
ryanmccartney May 2, 2025
15c4063
temp autoStart for tests
ryanmccartney May 2, 2025
49bebb4
Handle enable subtitles on start
ryanmccartney May 2, 2025
3cbb3b6
Fix Autostart
ShiningTrapez May 6, 2025
7aa6213
Improved isSubtitlesAvailable for embedded subs
ryanmccartney May 6, 2025
0a2f49d
Merge branch 'master' into embedded-subtitles
ryanmccartney May 6, 2025
f3cd971
Always start embedded subs strategy in embedded mode
ryanmccartney May 6, 2025
094e3c9
Apply sub styles immediately
ryanmccartney May 6, 2025
abaf8ce
Apply customisation immediately test
ShiningTrapez May 7, 2025
0eab88c
Wait for media player before starting subs
ryanmccartney May 7, 2025
559e373
Check if text track has id 888 before starting it
ryanmccartney May 8, 2025
19306e8
Add a debug line for text tracks
ryanmccartney May 8, 2025
6cafbff
fix: use Dash.js apis
eirikbjornr May 8, 2025
189d442
chore: undo changes to dev runner
eirikbjornr May 8, 2025
dfc8206
Don't double transform styles
ryanmccartney May 12, 2025
885a021
Tidy up customise function
ryanmccartney May 12, 2025
2e4d075
console.log style options
ryanmccartney May 12, 2025
1d5ca46
console.log style options
ryanmccartney May 12, 2025
c3a0a23
Remove console.log
ryanmccartney May 12, 2025
763f0a2
Update dash.js version from branch
ryanmccartney May 12, 2025
44a34ae
Update Dash.js version
ryanmccartney May 12, 2025
d8657cb
Update Dash.js version from branch
ryanmccartney May 12, 2025
674c51a
Attempt to fix Race Condition
ShiningTrapez May 14, 2025
288f74f
Revert "Attempt to fix Race Condition"
ShiningTrapez May 14, 2025
b8309e2
Printf Debugging
ShiningTrapez May 14, 2025
4f981c4
Printf Debugging++
ShiningTrapez May 14, 2025
616943c
Log the element
ShiningTrapez May 14, 2025
f1c23af
Remove logging
ryanmccartney May 14, 2025
cc81b69
Update Dash.js back to release
ryanmccartney May 14, 2025
40f667e
Merge main
ShiningTrapez May 15, 2025
f3572a3
Attach Subs Element before source
ShiningTrapez May 15, 2025
0e2ae01
Set up subtitles element in `setUpMediaPlayer`
ShiningTrapez May 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions docs/tutorials/Subtitles.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ You provide subtitles to BigscreenPlayer by setting `media.captions` in the `.in

```js
// 1️⃣ Add an array of caption blocks to your playback data.
playbackData.media.captions = [/* caption blocks... */];
playbackData.media.captions = [
/* caption blocks... */
]

// 2️⃣ Pass playback data that contains captions to the player.
player.init(document.querySelector("video"), playbackData, /* other opts */);
player.init(document.querySelector("video"), playbackData /* other opts */)
```

1. `media.captions` MUST be an array containing at least one object.
Expand All @@ -34,7 +36,7 @@ const captions = [
{ url: "https://some.cdn/subtitles.xml" },
{ url: "https://other.cdn/subtitles.xml" },
/* ... */
];
]
```

Subtitles delivered as a whole do not require any additional metadata in the manifest to work.
Expand All @@ -49,13 +51,15 @@ const captions = [
{
url: "https://some.cdn/subtitles/$segment$.m4s",
segmentLength: 3.84,
cdn: "default",
},
{
url: "https://other.cdn/subtitles/$segment$.m4s",
segmentLength: 3.84,
cdn: "default",
},
/* ... */
];
]
```

The segment number is calculated from the presentation timeline. You MUST ensure your subtitle segments are enumerated to match your media segments and you account for offsets such as:
Expand All @@ -73,12 +77,24 @@ You can style the subtitles by setting `media.subtitleCustomisation` in the `.in

```js
// 1️⃣ Create an object mapping out styles for your subtitles.
playbackData.media.subtitleCustomisation = { lineHeight: 1.5, size: 1 };
playbackData.media.subtitleCustomisation = { lineHeight: 1.5, size: 1 }

// 2️⃣ Pass playback data that contains subtitle customisation (and captions) to the player.
player.init(document.querySelector("video"), playbackData, /* other opts */);
player.init(document.querySelector("video"), playbackData /* other opts */)
```

### Low Latency Streams

When using Dash.js with a low-latency MPD segments are delivered using Chunked Transfer Encoding (CTE) - the default side chain doesn't allow for delivery in this case.

Whilst it is possible to collect chunks as they are delivered, wait until a full segment worth of subtitles have been delivered and pass these to the render function this breaks the low-latency workflow.

An override has been added to allow subtitles to be rendered directly by Dash.js instead of the current side-chain.

Subtitles can be enabled and disabled in the usual way using the `setSubtitlesEnabled()` function. However, they are signalled and delivered by the chosen MPD.

Using Dash.js subtitles can be enabled using `window.bigscreenPlayer.overrides.embeddedSubtitles = true`.

##  Design

### Why not include subtitles in the manifest?
Expand Down
35 changes: 11 additions & 24 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"@babel/plugin-transform-runtime": "^7.23.9",
"@babel/preset-env": "^7.23.8",
"@babel/preset-typescript": "^7.23.3",
"@rollup/plugin-alias": "^5.1.0",
"@rollup/plugin-alias": "^5.1.1",
"@rollup/plugin-babel": "^6.0.4",
"@rollup/plugin-commonjs": "^25.0.7",
"@rollup/plugin-inject": "^5.0.5",
Expand Down Expand Up @@ -64,7 +64,7 @@
"typescript-eslint": "^7.2.0"
},
"dependencies": {
"dashjs": "github:bbc/dash.js#smp-v4.7.3-6",
"dashjs": "github:bbc/dash.js#smp-v4.7.3-7",
"smp-imsc": "github:bbc/imscJS#v1.0.3"
},
"repository": {
Expand Down
4 changes: 4 additions & 0 deletions rollup.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import PackageJSON from "./package.json" assert { type: "json" }

import alias from "@rollup/plugin-alias"
import replace from "@rollup/plugin-replace"
import typescript from "@rollup/plugin-typescript"
import { dts } from "rollup-plugin-dts"
Expand All @@ -10,6 +11,9 @@ export default [
external: [/^dashjs/, "smp-imsc", "tslib"],
output: [{ dir: "dist/esm", format: "es" }],
plugins: [
alias({
entries: [{ find: "imsc", replacement: "smp-imsc" }],
}),
replace({
preventAssignment: true,
__VERSION__: () => PackageJSON.version,
Expand Down
4 changes: 4 additions & 0 deletions rollup.dev.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import PackageJSON from "./package.json" assert { type: "json" }

import alias from "@rollup/plugin-alias"
import babel from "@rollup/plugin-babel"
import commonjs from "@rollup/plugin-commonjs"
import resolve from "@rollup/plugin-node-resolve"
Expand All @@ -20,6 +21,9 @@ export default {
format: "es",
},
plugins: [
alias({
entries: [{ find: "imsc", replacement: "smp-imsc" }],
}),
replace({
preventAssignment: true,
__VERSION__: () => PackageJSON.version,
Expand Down
29 changes: 15 additions & 14 deletions src/bigscreenplayer.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,13 +114,6 @@ function BigscreenPlayer() {
!initialPresentationTime &&
initialPresentationTime !== 0

readyHelper = ReadyHelper(
initialPresentationTime,
mediaSources.time().manifestType,
PlayerComponent.getLiveSupport(),
_callbacks.playerReady
)

playerComponent = PlayerComponent(
playbackElement,
{ media, enableAudioDescribed, initialPlaybackTime: initialPresentationTime },
Expand All @@ -130,13 +123,21 @@ function BigscreenPlayer() {
callAudioDescribedCallbacks
)

subtitles = Subtitles(
playerComponent,
enableSubtitles,
playbackElement,
media.subtitleCustomisation,
mediaSources,
callSubtitlesCallbacks
readyHelper = ReadyHelper(
initialPresentationTime,
mediaSources.time().manifestType,
PlayerComponent.getLiveSupport(),
() => {
_callbacks.playerReady()
subtitles = Subtitles(
playerComponent,
enableSubtitles,
playbackElement,
media.subtitleCustomisation,
mediaSources,
callSubtitlesCallbacks
)
}
)
}

Expand Down
50 changes: 48 additions & 2 deletions src/playbackstrategy/msestrategy.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ function MSEStrategy(

let mediaPlayer
let mediaElement
let subtitleElement
let subtitlesEnabled = false
const manifestType = mediaSources.time().manifestType

const playerSettings = Utils.merge(
Expand All @@ -37,12 +39,15 @@ function MSEStrategy(
},
streaming: {
blacklistExpiryTime: mediaSources.failoverResetTime(),
lastMediaSettingsCachingInfo: { enabled: false },
buffer: {
bufferToKeep: 4,
bufferTimeAtTopQuality: 12,
bufferTimeAtTopQualityLongForm: 15,
},
lastMediaSettingsCachingInfo: { enabled: false },
text: {
defaultEnabled: false,
},
},
},
customPlayerSettings
Expand Down Expand Up @@ -108,6 +113,7 @@ function MSEStrategy(
STREAM_INITIALIZED: "streamInitialized",
FRAGMENT_CONTENT_LENGTH_MISMATCH: "fragmentContentLengthMismatch",
QUOTA_EXCEEDED: "quotaExceeded",
TEXT_TRACKS_ADDED: "allTextTracksAdded",
CURRENT_TRACK_CHANGED: "currentTrackChanged",
}

Expand Down Expand Up @@ -555,6 +561,13 @@ function MSEStrategy(
}
}

function setUpSubtitleElement(playbackElement) {
subtitleElement = document.createElement("div")
subtitleElement.id = "bsp_subtitles"
subtitleElement.style.position = "absolute"
playbackElement.appendChild(subtitleElement, playbackElement.firstChild)
}

function setUpMediaElement(playbackElement) {
mediaElement = mediaKind === MediaKinds.AUDIO ? document.createElement("audio") : document.createElement("video")

Expand All @@ -578,6 +591,7 @@ function MSEStrategy(

function setUpMediaPlayer(presentationTimeInSeconds) {
const dashSettings = getDashSettings(playerSettings)
const embeddedSubs = window.bigscreenPlayer?.overrides?.embeddedSubtitles ?? false
const protectionData = mediaSources.currentProtectionData()

mediaPlayer = MediaPlayer().create()
Expand All @@ -589,6 +603,11 @@ function MSEStrategy(

mediaPlayer.initialize(mediaElement, null)

if (embeddedSubs) {
setUpSubtitleElement(playbackElement)
mediaPlayer.attachTTMLRenderingDiv(subtitleElement)
}

modifySource(presentationTimeInSeconds)
}

Expand Down Expand Up @@ -670,10 +689,15 @@ function MSEStrategy(
mediaPlayer.on(DashJSEvents.GAP_JUMP, onGapJump)
mediaPlayer.on(DashJSEvents.GAP_JUMP_TO_END, onGapJump)
mediaPlayer.on(DashJSEvents.QUOTA_EXCEEDED, onQuotaExceeded)
mediaPlayer.on(DashJSEvents.TEXT_TRACKS_ADDED, handleTextTracks)
mediaPlayer.on(DashJSEvents.MANIFEST_LOADING_FINISHED, manifestLoadingFinished)
mediaPlayer.on(DashJSEvents.CURRENT_TRACK_CHANGED, onCurrentTrackChanged)
}

function handleTextTracks() {
mediaPlayer.enableText(subtitlesEnabled)
}

function manifestLoadingFinished(event) {
manifestLoadCount++
manifestRequestTime = event.request.requestEndDate.getTime() - event.request.requestStartDate.getTime()
Expand All @@ -700,6 +724,10 @@ function MSEStrategy(
: { start: 0, end: getDuration() }
}

function customiseSubtitles(options) {
return mediaPlayer && mediaPlayer.updateSettings({ streaming: { text: { imsc: { options } } } })
}

function getDuration() {
const duration = mediaPlayer && mediaPlayer.isReady() && mediaPlayer.duration()

Expand Down Expand Up @@ -777,6 +805,11 @@ function MSEStrategy(
)
}

function isSubtitlesAvailable() {
const textTracks = mediaPlayer.getTracksFor("text")
return (textTracks && textTracks.length > 0) ?? false
}

function isTrackAudioDescribed(track) {
return (
track.roles.includes("alternate") &&
Expand Down Expand Up @@ -855,9 +888,13 @@ function MSEStrategy(
mediaElement.removeEventListener("ratechange", onRateChange)

DOMHelpers.safeRemoveElement(mediaElement)

mediaElement = undefined
}

if (subtitleElement) {
DOMHelpers.safeRemoveElement(subtitleElement)
subtitleElement = undefined
}
}

function getSafelySeekableRange() {
Expand Down Expand Up @@ -952,9 +989,17 @@ function MSEStrategy(
getCurrentTime,
isAudioDescribedAvailable,
isAudioDescribedEnabled,
isSubtitlesAvailable,
setAudioDescribedOn,
setAudioDescribedOff,
getDuration,
setSubtitles: (state) => {
subtitlesEnabled = state ?? false

if (mediaPlayer) {
mediaPlayer.enableText(subtitlesEnabled)
}
},
getPlayerElement: () => mediaElement,
tearDown,
reset: () => {
Expand All @@ -964,6 +1009,7 @@ function MSEStrategy(
},
isEnded: () => isEnded,
isPaused,
customiseSubtitles,
pause,
play: () => mediaPlayer.play(),
setCurrentTime,
Expand Down
Loading