Codecs, codecs, codecs

[ggarra13]

Currrently, OpenRV seems severely crippled in term of codecs. While there are legal implications of course, it seems you guys went too far, particularly since you are not yet distributing OpenRV binaries.
Currently, at least on macOS, there are no codecs enough to decode some of the Netflix's open source contents, like SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov.
This prevents a proper evaluation of the package.

[ggarra13]

Never mind. I just stumbled onto ffmpeg.cmake. Congrats on your CMakeLists. They could be much more efficient, but they are really well done.

[ggarra13]

Well, I reopened this. I am trying to play the Netflix open content movie SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov (which is a heavy 4K movie to evaluate performance) and I cannot get the codecs to work. It complains about not knowing codec -147, even thou I recompiled ffmpeg with all bells and whistles. I also have problems with audio codec aac_at.
Is the ffmpeg that OpenRV uses too old for that? Or is it that you guys are protecting the API or are there other legal reasons.

[geffrak]

Hi @ggarra13, have you seen this?

OpenRV/src/lib/image/mio_ffmpeg/init.cpp

Line 22 in 41eccbe

// NOTE ON PATENT LICENSING:

[ggarra13]

Thanks for the hint. But it did not solve my issues. I commented all the codecs and I stiil get:

ERROR: MovieFFMpeg: Unsupported codec_id '147' in /Users/gga/Movies/SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov
INFO: /Users/gga/Movies/SolLevante_HDR10_r2020_ST2084_UHD_24fps_1000nit.mov

and:

ERROR: AudioToolbox init error: 1718449215
WARNING: Failed to open codec in avformat_find_stream_info
ERROR: AudioToolbox init error: 1718449215
ERROR: AudioToolbox init error: 1718449215
ERROR: AudioToolbox init error: 1718449215
WARNING: Failed to open codec in avformat_find_stream_info
ERROR: AudioToolbox init error: 1718449215
ERROR: MovieFFMpeg: Failed to open codec 'aac_at' for /Users/gga/Movies/black-widow-trailer-3_h1080p.mov
INFO: /Users/gga/Movies/black-widow-trailer-3_h1080p.mov

This is on macOS BTW.

[ggarra13]

mrViewer 1 and xStudio report the video codec for SolLevante as prores and 4CC as ap4x. Not sure why OpenRV does not know the codec_id.

[compaga]

Hi ggarra13, RV Commercial Apple ProRes support was implemented using a proprietary library provided by Apple. We had to remove it from the Open RV build. This is one of the codec we want to find the proper way to re-enabled. AFAIK, there is Apple ProRes support in ffmpeg but we haven't look at it yet.

[Cintilla]

Hi @ggarra13, have you seen this?

OpenRV/src/lib/image/mio_ffmpeg/init.cpp

Line 22 in 41eccbe

// NOTE ON PATENT LICENSING:

Hi @geffrak I´m not very experienced using github or coding, could you explain to me where should I execute this code?

[ggarra13]

As @compaga said, you cannot get access to those codecs in OpenRV as commercial RV was using.a proprietary library, and also an old version of FFmpeg.
You don't need to execute any code. You have to wait and see if Autodesk changes to use a more modern library of FFmpeg and adds the code to handle those codecs without a proprietary library. For the time being, OpenRV is crippled in terms of codec support.

[Cintilla]

Alright, thnx for the reply :)

[geffrak]

Given that you have access to the code, you can experiment with stuff like updating FFMPEG (and fixing what might break), changing the compile flags(and mio_ffmpeg's code to allow more) or using native libraries to enable more codecs if you don't want to wait.

[yeoldebrian]

Has anyone had success getting Apple Prores working?

[mcoliver]

If anyone is visiting this in the future. The command to unlock all the codecs in ffmpeg (assuming you have the license to do so) is below. Run it after you run source rvcmds.sh and before you run rvbootstrap or rvmk. It should compile with ffmpeg 8 and all the codecs including prores support.

rvcfg -DRV_FFMPEG_NON_FREE_DECODERS_TO_ENABLE="aac;aac_fixed;aac_latm;ac3;bink;binkaudio_dct;binkaudio_rdft;dnxhd;dvvideo;hevc;mpeg2video;prores;prores_ks;qtrle;svq1;svq3;vp9;vp9_cuvid;vp9_mediacodec;vp9_qsv;vp9_rkmpp;vp9_v4l2m2m" -DRV_FFMPEG_NON_FREE_ENCODERS_TO_ENABLE="aac;aac_mf;ac3;dnxhd;dvvideo;hevc;mpeg2video;prores;qtrle;svq1;svq3;vp9_qsv;vp9_vaapi" -DRV_FFMPEG="8"

There are also prores_aw and prores_lgpl options you can pass if needed.

Breakdown on the various prores codecs

prores (The Standard Encoder)

What it is: The default, native FFmpeg ProRes encoder.

Key Feature: It is designed for speed. It is generally the fastest of the bunch because it uses a simplified approach to encoding.

Downside: Because it prioritizes speed, the file size versus quality ratio isn't always as optimized as the ks version.

prores_ks (The "Kostya" Encoder)

What it is: Written by developer Kostya Shishkov, this is widely considered the "high-quality" version.

Key Feature: It supports the 4444 (Alpha) profile and provides more "pro-level" features, such as fine-tuning the quant matrix. It is generally more compliant with official Apple ProRes specifications.

Downside: It is significantly slower than the standard prores encoder.

prores_aw (The "Adobe" Encoder)

What it is: Based on a reverse-engineered implementation often associated with the "Anatoliy" (AW) code.

Key Feature: It was historically used to provide compatibility in specific workflows where the standard encoder might fail or produce artifacts.

Status: In modern FFmpeg builds, this is largely considered redundant to the standard prores or ks versions.

prores_lgpl (The LGPL-licensed Encoder)

What it is: A version of the codec specifically compiled to adhere to LGPL (Lesser General Public License) restrictions.

Why it exists: Some distributions of FFmpeg (like those bundled in commercial software) cannot use "GPL" code for legal reasons. This version ensures the codec can be distributed more freely.