AK+LibMedia+LibWeb+Tests: Implement buffered ranges for various formats#9705
Merged
Conversation
This will allow us to contain the buffered ranges within the timeline's area.
Factoring this out will let us reuse it for the buffered ranges.
The timeline will appear brightened in portions that are buffered, including sections prior to the current position.
We would ignore whether the callback/event loop had been set up for the stream. Tests appending enough data for it to start a new request after the new data would cause a crash.
This is necessary to allow the main thread to read out frames to determine the buffered ranges, without affecting the request position of an IncrementallyPopulatedStream or causing a deadlock waiting for data to come in.
This will be used to determine the buffered time ranges.
This will also advance the passed iterator so that it remains valid.
Returning early if an added chunk didn't overlap the previous one could result in overlapping chunks or 0-byte gaps if a chunk was after but within the new one. Instead, fall through to allow the merging to continue as normal. Also, make the joining condition check if the new chunk spans multiple existing chunks to merge/remove those.
This will later be used to test updating of buffered time ranges when data is removed. It will also be needed when data eviction is implemented.
These don't need to be mutable. has_cues_for_track() was also removed, as it was unused.
This was unintentionally clobbering the prior modification to the request position. Requesting data slightly earlier in the file is intended to ensure that we have data if the demuxer decides to seek backwards.
Also, update the ready state when the progress event is fired. Otherwise, we won't autoplay if 5 seconds of data aren't available in on_metadata_parsed().
The default mode is to grab the index and use it to determine the start and end of the buffered ranges. This works well for MP4. For WAV, the index is incomplete, so instead use a much simpler method, just determining the ranges based on the constant bitrate of the file. This is implemented through a new virtual ContainerNavigator class.
This function checks potentially_playing(), which means the button doesn't do anything if the player is buffering. We don't want that.
This function never worked properly anyway.
Add a new FLACNavigator class that can scan for frames both forward and backward from a byte offset to get the exact buffered ranges. The reads are done in chunks, which keeps each call pretty cheap. The ranges themselves are resolved by its ScanningContainerNavigator base class, which reconciles cached ranges against the incoming ones to avoid repeating work unnecessarily. With those combined optimizations, the calls normally take under 1 microsecond.
We could end up seeked within AVIOContext's internal buffer without it checking whether new data could be buffered without hitting EOF again, so we could get an unexpected read error.
This mainly acts as a proof of concept for manual seeking implementations, which may prove useful for other formats than WAV.
A new MP3Navigator class is added, which determines timestamps for byte positions by resyncing to a frame and then interpolating between known points on either side. The known points start out as the first frame's position in the file at timestamp 0, and EOF at the timestamp for FFmpeg's file duration estimate. New buffered ranges are interpolated between those two points, but also between the end of a prior range and the start of the next. Since MP3 can have variable bitrate without declaring it in the file header, we have to allow buffered ranges to shift forward as new data arrives to make room for underestimated durations. This is done for all ranges following the first that has been appended to, keeping the start of the current range consistent, so that subsequent seeks within that range remain consistent. Seeking is also implemented within the navigator to ensure that the byte<->timestamp mapping is consistent and the buffered ranges begin exactly where the seek landed.
This will be useful for Ogg buffered ranges.
Unfortunately, this necessarily involves parsing codec frames to get their durations. Ogg's granule positions indicate the last sample of the last complete frame of the page, so the navigator has to determine the durations of every packet in the page to offset it back to its start. Vorbis is an especially involved codec to determine that for, since it has initial data defining a mapping of indices to big or small block sizes that has to be held onto, and that mapping is preceded by a bunch of conditionally parsed bits. Also, a frame's block size calculation involves the block size of the previous frame for an overlap add. To avoid bringing that complexity directly into LibMedia, we delegate the parsing/calculation to FFmpeg's av_vorbis_parse functions.
Apparently FFmpeg probes WAV files for 32 packets before returning the stream info. With the default of 19200 bytes per packet, this could end up waiting for up to 5 seconds of data to be downloaded. Setting the max_size option only affects WAV and W64 in our build of libavformat. No other formats we care about will be affected.
Zaggy1024
force-pushed
the
buffered-ranges
branch
from
May 28, 2026 09:21
f8f6f37 to
eefc0c2
Compare
Zaggy1024
marked this pull request as ready for review
May 28, 2026 15:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Formats I've tested are:
Matroska runs a forward scan from a random access point until it reaches the end of the available bytes for a particular range to determine its start and end. Scans resume from where they left off in the last call so that work done per call is minimal.
MP4 uses FFmpeg's index to look up timestamps for the start and end of each byte range each call. The index's bytes and timestamps are in order, so this is very cheap with a binary search.
WAV simply calculates timestamps based on the byte offset from the first frame of the file and the PCM format.
MP3 is the trickiest one, since the format allows the bitrate to change between frames without warning. Timestamps are determined by interpolating between known points in the file, starting with the start and end. As buffered ranges are added, their start timestamps are calculated, and the contained frame durations are summed up. When a buffered range increases in size, all the ranges to its right are recalculated to potentially shift their timestamps to account for an increase in the prior duration of the file. By doing this only to ranges to the right, the range containing the play head will not normally shift, which keeps seeks within that range consistent, as well as all ranges to its left. Recalculating the ranges to the right, though, ensures that we never have any ranges overlap.
FLAC just involves finding a sync point forward from the first byte and backward from the last byte, and calculating the timestamp from the frame header. Probably the simplest format of them all.
Ogg is also very involved, since the container format only specifies the last timestamp contained by a page, meaning we have to subtract all its contained packets' durations. Packets are just raw codec data with no metadata, so in order to determine their durations, we need to be able to parse frames for every supported codec. We already had frame parsing for Opus and FLAC, but Vorbis is especially complex since it requires parsing a header bitstream and then tracking previous block sizes to calculate a current one.
Obligatory demo:
buffered-ranges.webm