feat: Add subtitle sync ui for easy alignment

[armanckeser]

Changes

• Chrome reactive change

chrome_reactive.mp4

• Firefox before and after (pressed playback and stopped for the actual subtitles to re-render)

• WebOS Browser

• Adds a dynamic subtitle sync rendering. Users can now stop where they would like a subtitle to start, align the time with the view and continue
• Adds api from the html player and playback manager to get the subtitles.

Issues

#4364

[jellyfin-bot]

Cloudflare Pages deployment

Latest commit	98b4af8
Status	✅ Deployed!
Preview URL	https://ca82cfe2.jellyfin-web.pages.dev
Type	🔀 Preview

View build logs

[dmitrylyzo]

1. This only works with "custom subtitle element" (doesn't work in Chrome) and obviously only for text (SRT) subtitles.
2. We probably should update the subtitles timeline on timeupdate.
3. IMO, the timeline is too close to the offset slider, which makes you think they are tied.
4. Too long for mobile.

[armanckeser]

Do you have recommendations for 1?

I am also open to some guidance on 2. I tried a bunch of things, but I feel like there is an efficient solution my lack of FE knowledge eludes me from.

[dmitrylyzo]

Do you have recommendations for 1?

When we use native API: https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/textTracks
I think for SSA/ASS and PGS we should hide the timeline.

I am also open to some guidance on 2. I tried a bunch of things, but I feel like there is an efficient solution my lack of FE knowledge eludes me from.

Ideally, we should use PlaybackSubscriber.
But something like this could work:

jellyfin-web/src/controllers/lyrics.js

Lines 162 to 189 in c9ccec8

	function bindToPlayer(player) {
	if (player === currentPlayer) {
	return;
	}
	releaseCurrentPlayer();
	currentPlayer = player;
	if (!player) {
	return;
	}
	Events.on(player, 'timeupdate', onTimeUpdate);
	Events.on(player, 'playbackstart', onPlaybackStart);
	Events.on(player, 'playbackstop', onPlaybackStop);
	}
	function releaseCurrentPlayer() {
	const player = currentPlayer;
	if (player) {
	Events.off(player, 'timeupdate', onTimeUpdate);
	Events.off(player, 'playbackstart', onPlaybackStart);
	Events.off(player, 'playbackstop', onPlaybackStop);
	currentPlayer = null;
	}
	}

[armanckeser]

Hmm, does this mean that platforms like WebOS wouldn't be able to use this implementation? I presume WebOS uses a chromiom browser.

[github-actions]

ESLint doesn't pass. Please fix all ESLint issues.

[armanckeser]

Still have to think a bit about the realtime updates, the problem is getting a smooth timeline. Updating current implementation on every timeupdate results in the timeline moving choppily every second. I need to increase the resolution for the percentage calculations or add a css transition or something, but I am not sure yet

[armanckeser]

I tried to implement a smooth scroll, but I just give up. I think this provides the value I would like it to provide. And even though the playback rendering is a bit janky, probably better than not having it at all.

[armanckeser]

@dmitrylyzo thoughts?

[Gauvino]

Absolutely incredible

[armanckeser]

@dmitrylyzo Added screenshots from Firefox, Chrome, WebOS browser. Added realtime update ability and a video from Chrome showing that the changes are accurate with the realtime rendering. Changed the styling to better match Jellyfin Web style. I believe with SSA/ASS and PGS, the subtitle offset menu does not even show up (I tried with PGS and DVD SUB, neither showed the menu), but if it did, the timeline would be hidden.

[armanckeser]

@dmitrylyzo Would love to learn what types of tests or proof I could add to make this easier to review for the team.

[dmitrylyzo]

1. As before, I think the timeline is too close to the offset slider, which makes you think they are tied.
   Close:
   
   Split:
   

2. My biggest concern right now is intermediate refactoring. I prefer to do refactoring first and then add the feature. Ideally, we should make commits atomic. If no one has any objections to the current commit sequence, so be it.

[armanckeser]

As before, I think the timeline is too close to the offset slider, which makes you think they are tied.

Ah, thanks for the clarification. I thought you meant that the current time marker itself was too close to the button (which it was), but "split" looks much better

My biggest concern right now is intermediate refactoring. I prefer to do refactoring first and then add the feature. Ideally, we should make commits atomic. If no one has any objections to the current commit sequence, so be it.

I see. Let me try a stacked approach, I am thinking a PR (or commit?) to refactor the original into OffsetController and the SubtitleSync object, and another one to add the timeline view. Does that sound good to you?

[sonarqubecloud]

Quality Gate passed

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud