Repository navigation
feat: overhaul Smart Speed engine with SponsorBlock, captions, and modular architecture (#4147) - #4355
Conversation
…dular architecture (code-charity#4147)
|
Amazing!! |
|
Fully Tested? Ready to merge in Beta or everywhere?
Complete heatmap and transcript processing might run patiently? Per video or per segment? Yet a user could still target 1.5x average speed per video to compared improvements over static speed?
depends video length i suppose.
then #4265 remains open i guess
Structure, Chapters, Speakers might be labeled too 😮💡
Shown for every Short and live Live? Sticky toggle?
Should(n't) this depend the type? (Whitelist only changes on click? Skip only per segment?)
currently a prefix of 4 chars results in average 1.38 videos among the 11.5 Million videos. The DB is 2.5 GB gzipped. A few MB can store a reference to all Channel IDs appearing in it repeatedly.
|
|
@ImprovedTube2 Thanks for the review! Testing: I've tested it by hand, but there are no automated tests yet. I plan to add unit tests later. I'd call it a stable beta: the structure is modular, but I know of a few rough edges (e.g. the Shorts toggle also affects the short-video threshold) that I'll fix. Getting users to try it in Beta is the best way to find the remaining small bugs. Processing: Heatmap and captions are processed once per video. Each tick (every 100ms) only does a cheap lookup. The target-duration solver runs only when the user clicks Apply. A target average speed (like 1.5x) would be easy to add, and it would make comparing against a static speed simple. Outro / intro: The outro is the last 3%, capped at 15s, and only used when SponsorBlock has no outro data. I haven't added an intro rule yet. A Wadsworth-style rule is a good idea, and I'd make it optional. #4265 / smoothing: It was built for the old version, so it isn't merged. Your comments there apply to the current 2.5s blend too, and I agree with most of them:
I'd also add a single 0 to 1 smoothness setting that controls transition length (0 = instant), with a default close to today's behavior, instead of exposing several curve options. Chapters and speakers: Good idea. Each can be added as a new signal without changing the engine. Shorts / Live toggles: Right now they show on every Short or live stream, and the setting is global and sticky. I agree it could be per-type or per-session, and I'll look at it. 1s UI refresh: Agreed. I'll switch to events: the whitelist button updates on click, the Live/Shorts buttons on navigation, and the skip buttons when the segment changes. k-anonymity: Your numbers made me check, so I ran the whole SponsorBlock dump (10.5M unique videos) through the same hashing our code uses. Your figures match a plain video-ID prefix (1.35 for 4 chars, 41 for 3). We send a hex SHA-256 prefix, which has fewer symbols per character:
So 4 chars already hides each request among about 160 videos, and I kept user privacy in mind while writing this. Going to 3 would only make responses about 16x bigger. The channel-ID list is still a good idea to cut requests, and I'd do it as a follow-up. Next: I'm planning a second major PR around the end of October that adds more signals so the speed adapts better to each video's context. One idea is to embed the transcript, split the video into sections, score each section's relevance to the video's main topic, and then score individual sentences within their section. |
|
While working on combining playback speed buttons A and B in connection with #4343, I noticed that the button's custom-speed setting, I'm planning to allow custom speeds up to 16× for the combined button. Since With the current settings menu, both the custom speed and Smart Speed's target multiplier are limited to 4, so their product stays within 16×. Increasing the custom-speed limit to 16× would allow the final speed to exceed 16×, depending on |
|
hi! @hkzo first we had "Speed watching: permanent Speed" (player_forced_playback_speed) Many users have this. (did _baseRate mean this? @AryaXDG ) Permanent Speed remains limited to 3.17x because i didn't want to be responsible for anything above. 16x 'fast forward' might be fun just usually not optimal, but of course shortcuts and Buttons may overwrite everything. |
|
"Speed watching: permanent Speed" (player_forced_playback_speed) is separate because it uses the value from storage.player_playback_speed. In this PR,
_baseRate() { return toNum(ImprovedTube.storage.player_custom_playback_speed, 1, 0.0625, 16); },
_setRate(rate) {
const v = this._getVideo();
if (!v || Math.abs(v.playbackRate - rate) < 0.01) return;
this.expectedRate = rate;
v.playbackRate = rate;
},In this PR, const finalSpeed = isOverrideSpeed ? targetSpeed : this._baseRate() * targetSpeed;
this._setRate(finalSpeed);Chromium(Chrome,Edge,...) throws a DOMException (NOT_SUPPORTED_ERR) when playbackRate exceeds 16. |
|
Hey @hkzo @ImprovedTube2 Just to clarify on @ImprovedTube2 question, _baseRate() uses player_custom_playback_speed (the custom speed button's setting), not Permanent Speed (player_forced_playback_speed). Permanent Speed is completely separate and stays capped at 3.17× as intended. To fix the DOMException error when speeds exceed 16×, I just opened and merged #4371. Instead of checking tick(), I updated _setRate() so it automatically clamps all incoming rates to [0.0625, 16] using toNum(rate, 1, 0.0625, 16). This acts as a global safety guard for Smart Speed so Chrome will never throw an error, while keeping internal state (this.expectedRate) in sync! |
|
hi @hkzo @AryaXDG! it is great if we connect features! |
💡 sorry. (Then 5 digits might happen😀)
Unfortuantely we dont see an amount of users per feature yet, but since Smart Speed is young and without many revisions, we may upload for everybody soon unless there is a clear disadvantage or missing legacy option of current users. |
PR: overhaul Smart Speed engine with SponsorBlock, captions, and modular architecture
Summary
This replaces the earlier proof-of-concept heatmap implementation with a full rewrite of the Smart Speed engine. It keeps the original core idea intact using YouTube's "Most Replayed" heatmap markers to slow down for high-engagement scenes and speed up for low-engagement ones but turns it into a modular, multi-signal system and fixes a number of reliability issues in the old version.
Why
Issue #4147 asks for using the heatmap's 100 intensity markers to auto-adjust playback speed, since view-density per scene tends to correlate with relevance (slow intros/outros, catch the "spikes"). The old code (partly implemented in #3628) proved the concept but was a single 400-line object with global mutable state, no cleanup between videos, and no way to combine the heatmap with other relevance signals the issue also discusses (e.g. captions/transcript, sponsor segments).
What changed
Architecture
ImprovedTube.heatmapobject:SpeechMaptimeline of speech blocks with eased lead-in/releaseSponsorBlockClientk-anonymity SponsorBlock API lookup with cachingHeatmapSourcetiered heatmap acquisition (active player -> page globals -> local cache -> watch-page fetch)CaptionSourcetranscript acquisition (TimedText API -> transcript panel scrape)Profileschannel/category/language profile resolutionSignals/computeCompositeSpeedweighted blending of multiple speed signalsOverridesabsolute (non-blended) speed rules, e.g. SponsorBlock segmentsDurationSolverbinary search for target-duration biasSessionper-video state, replacing shared global fieldsEnginestate machine driving the whole lifecycleUIall DOM work, so the engine itself never touches the DOMSessionobject created per video instead of resetting fields on a shared object avoids stale state leaking across video changes.IDLE / LOADING / ACTIVE / WHITELISTED / EXCLUDED / PAUSED / AD) instead of ad-hoc boolean flags (sessionDisabled, etc).New signals beyond the raw heatmap
computeCompositeSpeed) rather than heatmap alone, so future signals (e.g. word-difficulty/WPM from the issue) can be added toSignalswithout touching the engine.Reliability fixes
belongsToVideo()before trusting cached/global data, preventing stale data from a previous video being applied to a new one.expectedRatedrift check) and pause Smart Speed instead of fighting the user.MutationObserveron the player'sad-showingclass, forcing base rate during ads.AbortControllerso in-flight heatmap/caption/SponsorBlock fetches are cancelled when the user navigates away._blendAcrossBoundary) smooths speed transitions over 2.5s, same idea as before but generalized to work with the composite signal.Profiles
smart_speed_rule_priority(previously: channel-name lookup only, keyed by fragile display-name strings).channel:<id>) instead of display name, with automatic migration of legacy name-keyed profiles once an ID is available.Target duration
UI
UI.refreshContextualinstead of being scattered across handlers.Will be addressed next in future work