Make timestamp indexes for Libav plugin #1
Description
When Libav reads a container file, it reads the raw timestamps that are in there. I deal with everything in Canvas in frames and samples. This makes a difficult disconnect—it's entirely possible, and even done sometimes, to build a video file with a variable frame rate, where the distance between the timestamps changes to signal a frame rate change.
I decided at the beginning I'm not interested in tackling that problem in the timeline. Someone could envision an export tool that combines differently-timed source material and produces Frankenvideos out of them, or an import tool capable of treating the different sections of a video with different properties. In the mean time, I just want the raw frames and raw samples, each and every one of them, indexed in order.
To do this properly, I need to know the timestamp of every frame and every block of samples in the file. The audio decoder does this already in an ad-hoc way. I could add that same ad-hoc method to the video decoder, but I think I'd like it to be persisted. Perhaps in a packet source that automatically indexes what's upstream of it?
Part of the issue here is the ambiguity of codec_seekFunc, which accepts int frame as its seek parameter. Perhaps that should be in dts timestamps, and I could stick a filter in to change it from timestamps to frames. Or maybe I should just build helper tools to do the indexing on behalf of the codec (because the codec, for example, will want to know where the previous keyframe was so it can seek back there).
Another issue here will be how whoever is doing the indexing will know when to update the index, which perhaps is a case for always doing the indexing in-memory. Or you could envision a hybrid system, where the codec must always do its own in-memory indexing, but can pass flags to an upstream packet source that would, say, send back packet metadata and not the data itself while building the index. Demuxers capable of this could support the scheme directly, or you could stick an indexer in between that could provide said metadata and the calling program could manage the caching issues.
Complicated. But it needs to be solved.