converting a fragmented mp4 to a progressive mp4

[nchitkara-xai]

Both #162 and #311 asked for this direction. In #162 you described what it takes, all the moofs have to be read first since the moov of the progressive file needs the metadata of every sample before anything can be written. I know mp4ff's focus has been the fragmented direction so totally fine if this stays out of scope. But I have a working implementation of that two pass approach and wanted to offer it upstream since the hard part is already written and tested.

What it does:

• first pass collects sample metadata from all the moofs (lazy mdat, payloads never get materialized). The progressive sample tables (stts/ctts/stsc/stsz/stss/stco/co64) are synthesized from the trun/tfhd/trex data, the moov is written, then the sample byte ranges are copied verbatim from the input
• moov metadata carries through untouched: movie/media timescales, tkhd (incl matrix), edit lists (no synthesis or rescaling) and stsd
• each traf becomes one chunk so the input interleaving is kept. ctts only written when some composition offset is nonzero (version 1 if any are negative), stss omitted when everything is sync, co64 instead of stco when offsets need it
• tfdt is authoritative, a forward gap vs the accumulated durations widens the previous sample duration, a backward tfdt is an error
• it fails closed instead of writing silently wrong output: kept tracks must share a decode time origin (rebased to zero), nonzero origin + edit list is rejected since the edit media times reference the un-rebased timeline, encrypted content (senc/saiz/saio) is rejected, and hostile declared counts error out before any count proportional allocation

Its been exercised against a few million real world files at this point, with differential checks against ffmpeg's remux of the same inputs.

Questions before I open a PR:

1. where should this live? a library function like Defragment(f *File, rs io.ReadSeeker, w io.Writer) error (similar to how mp4ff-crop takes the decoded file plus a ReadSeeker for the payload copy), a cmd/mp4ff-defragment CLI, or both. I'd lean library core + a thin CLI?
2. do you want track selection (keeping a subset of track ids) in the first version, or better to add later if someone needs it?
3. any opinion on the fail closed choices, especially rejecting nonzero origin + edit list instead of rewriting the elst?

[tobbee]

@nchitkara-xai Sounds like a nice addition. Some comments on your questions:

1. library core + a new thin CLI application sounds fine
2. track selection? Most fragmented files I know about have one track per fragment, so not much to select. There may be a trackID collision of course, so that should be remedied. If you can handle multi-track fragments, it may be nice, but it may get complex pretty soon. Interleaving may be tricky if the fragment durations differ a lot between different CMAF tracks. I envision that you essentially give N file patterns, one for each "track/series of files" you want to import. A track may also already be concatenated like a CMAF track.
3. The edit lists that CMAF allows are just timeline offsets for video and audio. Such edit lists should be possible to handle easily even if the timing doesn't start at 0 by displacing all timestamps, and/or keeping a time-shift edit list, so that may be nice to have. More complex edit lists should probably be disallowed anyway.

In general, I am positive to have a look at what you've done. I suppose it should be relatively easy to support the on-the-fly progressive use case I mentioned in #162, but that is certainly not a base functionality.

Let's start with what you have at the moment, since that has apparently server you well, so it should be useful and battle tested.