Feature Request - Waveform graph overlay as a timeline/sekbar option

[speakn2pabst]

A waveform overlay to the timeline/seekbar would be an interesting option to the current choices in uosc.

Some common use cases would be the ability for quick assessment of where to back up in the seekbar if the user missed a sentence of conversation in a movie or a how-to video.

Also this would benefit the music listening and podcast listening audience in being able to seek quickly to relevant portions or to skip past the repetitive opening jingle of a spoken word podcast.

In the same vein it may help video series viewers also identify the same opening sequence of a regular show and assist in the skipping of the repeat material (show openings, usually set to a theme song).

I asked some AI's but wasn't able to create a working example PR. But I did find it was possible to create a waveform graph with FFmpeg which as a MPV prerequisite should be available to all users.

Here's a link to a screenshot of Linux music app Quod Libet for an example waveform seekbar/timeline image.

Waveform seekbar example

[po5]

You might be interested in this project, note it requires a bunch of setup through shell scripts https://github.com/occivink/mpv-music-player

[tomasklaen]

Yeah waveform seekbar would be cool, but the available tech is not there, otherwise it'd already be included.

Issues with generating waveforms with ffmpeg:

• takes too long
• not streamed, so you see nothing until the whole thing is done, making you question if it's even working
• output is png, and overlaying images over UI is very limited and problematic in mpv

The ideal scenario would be to get a streamed data of peaks to render as a curve. But even if we somehow found a way to get that, for a waveform to be truly useful it needs to be high-ish resolution (at least one peak per 2-5 pixels), and this is than doubled since each peak has to be mirrored at the bottom to look like a waveform. That's a lot of tags to be pushing to libass. I'm pretty sure this would kill uosc rendering performance.

This is not an issue with out current youtube heatmap since that's just constant 100 points. With a waveform that would be in thousands.

For this to be viable, we'd need some sort of JavaScript like 2D canvas API to render into once and then be able to composite that image over UI with scaling and opacity control. But if we had this kind of API we wouldn't need to render uosc using ASS tags in the first place :)

[speakn2pabst]

You might be interested in this project, note it requires a bunch of setup through shell scripts https://github.com/occivink/mpv-music-player

Thanks, I may try it but was hoping for a solution that includes videos.

Yeah waveform seekbar would be cool, but the available tech is not there, otherwise it'd already be included.

... ... ... ... ...

For this to be viable, we'd need some sort of JavaScript like 2D canvas API to render into once and then be able to composite that image over UI with scaling and opacity control. But if we had this kind of API we wouldn't need to render uosc using ASS tags in the first place :)

Thanks for the reply and explainer. I'm glad it's already been considered. Reading a few other issue pages and the notion that uosc shoehorns a fair bit of functionality into an interface space that wasn't built for it sinks in, so kudos to your current results. Hopefully the current tech hurdles eventually get smoothed over. Thanks again.