Feature/FFMPEG_Core : Rewrite parts of the video playbacks and Render To File. by ExtraBinoss · Pull Request #912 · tixl3d/tixl

ExtraBinoss · 2026-01-10T00:02:27Z

This is the first steps of the FFMPEG video playback and encoding rewrite.

This will include (but not limited to) :

FFMpeg Playback [PlayVideo] [PlayVideoClip]
Render to file (with FFMPEG) supporting Mp4 (LibOpenH264), Webm, WebmAlpha (if working), HAP, HAP Alpha
Testing render using ffmpeg with Linux and MacOS devices, which should be working better, since we would be using industry standards.

Right now this feature branch I made includes the following :

FFmpeg Render in the Window --> FFmpegRender (next to Render to File)
You can render a video using seconds mode (didn't try the others. Didn't bother for the moment)
Codecs tested and working are : Webm (VP9), ProRes, HAP, HAP Alpha, MP4 with Libopenh264 works.
Image sequence work with auto increment

Roadblocks :

FFMpeg licensing is hard, the library we are using FFmpegCore
This library is using a GPL compiled version, with LibX264, LibX265 with are contaminating the whole binary, and thus cannot be used in Tixl (it would override the MIT licence).
UX is the most important, shipping it in the installation of Tixl is probably the best thing, even though it increases filesize. However if users have to download them, it's a big dealbreaker.
We need to precompile the library in the setup.exe of Tixl, with flags like :

Format / Feature	Requirement	Recommended Flags	Patent Risk
Legal: License	LGPL v3	`--disable-gpl --disable-nonfree --enable-version3`	None
Legal: Linking	Dynamic	`--enable-shared --disable-static`	None
WebM (VP9)	Encoder/Decoder	`--enable-libvpx`	Royalty Free
WebM (AV1)	Encoder/Decoder	`--enable-libaom --enable-libsvtav1`	Royalty Free
MP4 (H.264)	"Open" CPU Encoder	`--enable-libopenh264`	Paid by Cisco*
MP4 (H.264)	NVIDIA GPU	`--enable-nvenc`	Paid by NVIDIA
MP4 (H.264)	Intel GPU	`--enable-libmfx`	Paid by Intel
MP4 (H.264)	AMD GPU	`--enable-amf`	Paid by AMD
HAP / HAP Alpha	Encoder/Decoder	(Native)	Royalty Free
ProRes / Alpha	Encoder/Decoder	(Native)	Apple "Unauthorized"
MP3	Encoder/Decoder	`--enable-libmp3lame`	Expired (Free)
OGG (Vorbis)	Encoder/Decoder	`--enable-libvorbis`	Royalty Free
OGG (Opus)	Encoder/Decoder	`--enable-libopus`	Royalty Free

Ideas :
We could add av1 encoding since it's better and open-source. We will see if the library and ffmpeg supports it.

Good luck!

…retty well

…es now

…wn implem of FFMpegCore to support libopenh264), WEBM (without alpha doesn't work) (maybe we can fix in our implementation), HAP and HAP alpha works. We need to recompile into a LGPL version + shared libs, probably that the user downloads intentionally.

Copilot

Pull request overview

This PR introduces an FFMpeg-based video rendering system as an alternative to the existing render-to-file functionality. The implementation adds a new window for FFMpeg rendering with support for multiple codecs (H.264, H.265, ProRes, HAP, VP9) and includes automatic FFMpeg binary download functionality.

Key Changes:

New FFMpeg-based rendering pipeline with support for multiple video codecs and formats
Automatic FFMpeg binary download and installation functionality via FFMpegCore library
Integration of FFMpeg rendering into the existing editor UI as a separate window

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 37 comments.

Show a summary per file

File	Description
Editor/Gui/Windows/RenderExport/FFMpegRenderWindow.cs	New UI window for FFMpeg rendering with time range controls, codec selection, and render settings
Editor/Gui/Windows/RenderExport/FFMpegRenderSettings.cs	Settings class defining render parameters, codecs, and file format mappings
Editor/Gui/Windows/RenderExport/FFMpegRenderProcess.cs	Core rendering process manager handling frame updates, FFMpeg installation, and export lifecycle
Editor/Gui/Windows/RenderExport/FFMpeg/FFMpegVideoWriter.cs	Video writer implementation handling GPU texture readback and FFMpeg encoding via pipes
Editor/Gui/Windows/Layouts/WindowManager.cs	Registers the new FFMpegRenderWindow in the window management system
Editor/Gui/T3Ui.Update.cs	Integrates FFMpegRenderProcess.Update() into the main update loop
Editor/Editor.csproj	Adds FFMpegCore and FFMpegCore.Extensions.Downloader NuGet package references

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-10T00:11:26Z

Editor/Gui/Windows/RenderExport/FFMpegRenderProcess.cs

+internal static class FFMpegRenderProcess
+{
+    public static string LastHelpString { get; private set; } = string.Empty;
+    public static double Progress => _frameCount <= 1 ? 0.0 : (_frameIndex / (double)(_frameCount - 1));


The progress calculation uses _frameCount - 1 in the denominator, which will cause division by zero if _frameCount is 1. While rendering a single frame might be unusual, this should be handled gracefully. Additionally, the check should be _frameCount <= 0 rather than <= 1 to properly handle edge cases.

Suggested change

public static double Progress => _frameCount <= 1 ? 0.0 : (_frameIndex / (double)(_frameCount - 1));

public static double Progress => _frameCount <= 0

? 0.0

: _frameCount == 1

? (_frameIndex > 0 ? 1.0 : 0.0)

: (_frameIndex / (double)(_frameCount - 1));

Copilot · 2026-01-10T00:11:26Z

Editor/Gui/Windows/RenderExport/FFMpegRenderProcess.cs

+        if (_renderSettings.Bitrate > 0 && true) // Check auto-increment?
+        {
+             // Maybe implement increment if settings have it
+        }
+


The condition check "true" is a hardcoded boolean that makes the entire conditional statement pointless. Either remove the condition or implement the actual auto-increment logic that was intended here.

Suggested change

if (_renderSettings.Bitrate > 0 && true) // Check auto-increment?

{

// Maybe implement increment if settings have it

}

Copilot · 2026-01-10T00:11:27Z

Editor/Gui/Windows/RenderExport/FFMpegRenderProcess.cs

+        DownloadProgress = 0f;
+
+        try 
+        {
+            if (!Directory.Exists(folder))
+                Directory.CreateDirectory(folder);
+
+            Log.Info($"Downloading FFMpeg to {folder}...");
+
+            // Progress not supported in this version of FFMpegCore.Extensions.Downloader
+            await FFMpegDownloader.DownloadBinaries(FFMpegCore.Extensions.Downloader.Enums.FFMpegVersions.LatestAvailable, 
+                                                    options: new FFOptions { BinaryFolder = folder });
+
+            DownloadProgress = 1.0f;


The DownloadProgress field is updated from an async context (InstallFFMpeg) and read from the UI thread without synchronization. This could cause visibility issues across threads. Consider using volatile keyword or proper synchronization mechanisms.

Copilot · 2026-01-10T00:11:27Z

Editor/Gui/Windows/RenderExport/FFMpegRenderWindow.cs

+        FormInputs.AddVerticalSpace();
+
+        FormInputs.AddFloat("FPS", ref FFMpegRenderSettings.Fps, 0);
+        if (FFMpegRenderSettings.Fps < 0) FFMpegRenderSettings.Fps = -FFMpegRenderSettings.Fps;


The FPS value validation on line 185 uses absolute value to convert negative FPS to positive, but negative FPS could indicate a logic error elsewhere that should be caught and reported rather than silently fixed. Consider logging a warning or throwing an exception for invalid FPS values.

Suggested change

if (FFMpegRenderSettings.Fps < 0) FFMpegRenderSettings.Fps = -FFMpegRenderSettings.Fps;

if (FFMpegRenderSettings.Fps < 0)

{

System.Diagnostics.Debug.WriteLine($"[FFMpegRenderWindow] Invalid negative FPS value ({FFMpegRenderSettings.Fps}) encountered. Converting to a positive value.");

FFMpegRenderSettings.Fps = -FFMpegRenderSettings.Fps;

}

Copilot · 2026-01-10T00:11:27Z

Editor/Gui/Windows/RenderExport/FFMpegRenderProcess.cs

+    private static int _frameIndex;
+    private static int _frameCount;
+
+    private static RenderSettings _renderSettings = null!; // Using standard settings for timing helpers


The field _renderSettings is marked with null! but is only initialized when TryStart is called. This creates a potential null reference issue if Update() is called before TryStart() and uses _renderSettings in RenderTiming methods. Consider initializing with a default value or adding null checks.

Suggested change

private static RenderSettings _renderSettings = null!; // Using standard settings for timing helpers

private static RenderSettings _renderSettings = new RenderSettings(); // Using standard settings for timing helpers

Copilot · 2026-01-10T00:11:33Z

Editor/Gui/Windows/RenderExport/FFMpegRenderWindow.cs

+
+        if (FFMpegRenderProcess.State == FFMpegRenderProcess.States.NoValidOutputType)
+        {
+            _lastHelpString = FFMpegRenderProcess.MainOutputType == null


Write to static field from instance method, property, or constructor.

Copilot · 2026-01-10T00:11:33Z

Editor/Gui/Windows/RenderExport/FFMpegRenderWindow.cs

+            return;
+        }
+
+        _lastHelpString = "Ready to render.";


Write to static field from instance method, property, or constructor.

Copilot · 2026-01-10T00:11:34Z

Editor/Gui/Windows/RenderExport/FFMpeg/FFMpegVideoWriter.cs

+        var height = cpuAccessTexture.Description.Height;
+        var rowStride = SharpDX.WIC.PixelFormat.GetStride(SharpDX.WIC.PixelFormat.Format32bppRGBA, width);
+
+        var dataBox = ResourceManager.Device.ImmediateContext.MapSubresource(cpuAccessTexture, 0, 0, MapMode.Read, MapFlags.None, out var inputStream);


This variable is manually disposed in a finally block - consider a C# using statement as a preferable resource management technique.

Copilot · 2026-01-10T00:11:34Z

Editor/Gui/Windows/RenderExport/FFMpegRenderSettings.cs

+    // Actually, let's keep the structure similar.
+
+    public  TimeRanges TimeRange = TimeRanges.Custom;
+    public  float ResolutionFactor = 1f;


Field 'ResolutionFactor' can be 'readonly'.

Suggested change

public float ResolutionFactor = 1f;

public readonly float ResolutionFactor = 1f;

Copilot · 2026-01-10T00:11:34Z

Editor/Gui/Windows/RenderExport/FFMpegRenderSettings.cs

+    public int OverrideMotionBlurSamples;   // forwarded for operators that might read it
+
+
+    public  RenderModes RenderMode = RenderModes.Video;


Field 'RenderMode' can be 'readonly'.

Suggested change

public RenderModes RenderMode = RenderModes.Video;

public readonly RenderModes RenderMode = RenderModes.Video;

… into feature/ffmpeg_core

…ing and breathes finally. mostly for ffmeg render right now.

…write, auto increment etc

… into feature/ffmpeg_core

del: bad annotations

…mpeg_core

… part.

enderprism · 2026-02-19T08:00:14Z

the FFMpegRenderWindow is missing some new stuff from RenderWindow

ExtraBinoss added 4 commits January 9, 2026 19:59

fix: ffmepg POC: render to file using mp4 and seems to work already p…

65873c1

…retty well

fix: ffmpeg now works with the correct seconds mark

8568ca8

fix: HapAlpha works, Vp9 works now. settings have advanced capabiliti…

513ea65

…es now

Copilot AI review requested due to automatic review settings January 10, 2026 00:02

Copilot started reviewing on behalf of ExtraBinoss January 10, 2026 00:03 View session

Copilot AI reviewed Jan 10, 2026

View reviewed changes

enderprism and others added 21 commits January 10, 2026 18:28

fix: remove non-LGPL or unsupported codecs

a3e067b

chore: unused imports

00d6cfc

update: ffmpegRender to have audio from the timeline

be86575

Merge branch 'feature/ffmpeg_core' of github.com:ExtraBinoss/tixl-dev…

aad0c8b

… into feature/ffmpeg_core

add: progress bar and export summary and output directory button

237857d

update: good UX now. with the export summary

37bf0be

update: formInputs with a better enum dropdown that has way more padd…

6511d61

…ing and breathes finally. mostly for ffmeg render right now.

update: ImageSequence addition with almost the right things like over…

b347e38

…write, auto increment etc

update: image sequence with auto incrementation

42dd3ce

feat: respect project volume when exporting video

ddc5d39

add: webp rendering and also the good auto increment of the videos

cf5b95f

Merge branch 'feature/ffmpeg_core' of github.com:ExtraBinoss/tixl-dev…

725e93c

… into feature/ffmpeg_core

chore: fix documentation comment

65f33c2

Merge branch 'feature/ffmpeg_core' of github.com:ExtraBinoss/tixl-dev…

258718d

… into feature/ffmpeg_core

fix: webp encoding with image sequence

e8b6678

fix: webp rendering with compression level to 0

8b77aa8

fix: compression level to 0 and regex for the output auto increment

0d57eb2

fix: UX with bitrate, auto incrementing is okay

3b9c1b0

update: better classes inside the ffmpeg render window

8232715

del: bad annotations

fix: open folder and the output summary aligned better

1399cc1

change h264 codec back to libopenh264; chore: format code

836a38b

ExtraBinoss marked this pull request as draft January 17, 2026 15:56

Merge branch 'main' of https://github.com/tixl3d/tixl into feature/ff…

22b09e2

…mpeg_core

ExtraBinoss and others added 4 commits January 20, 2026 21:54

fix merge conflicts it was simply some bad interface management on my…

259d3f0

… part.

ffmpeg resolution now.

5771d97

Merge branch 'main' into feature/ffmpeg_core

35f7d14

fix: branch doesn't build; fix audio exporting

6d4e0d6

Merge branch 'main' into feature/ffmpeg_core

27ffc4b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/FFMPEG_Core : Rewrite parts of the video playbacks and Render To File.#912

Feature/FFMPEG_Core : Rewrite parts of the video playbacks and Render To File.#912
ExtraBinoss wants to merge 31 commits intotixl3d:mainfrom
ExtraBinoss:feature/ffmpeg_core

ExtraBinoss commented Jan 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

enderprism commented Feb 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-    public static double Progress => _frameCount <= 1 ? 0.0 : (_frameIndex / (double)(_frameCount - 1));
+    public static double Progress => _frameCount <= 0
+                                         ? 0.0
+                                         : _frameCount == 1
+                                               ? (_frameIndex > 0 ? 1.0 : 0.0)
+                                               : (_frameIndex / (double)(_frameCount - 1));

	private static RenderSettings _renderSettings = null!; // Using standard settings for timing helpers
	private static RenderSettings _renderSettings = new RenderSettings(); // Using standard settings for timing helpers

	public float ResolutionFactor = 1f;
	public readonly float ResolutionFactor = 1f;

		public int OverrideMotionBlurSamples; // forwarded for operators that might read it


		public RenderModes RenderMode = RenderModes.Video;

	public RenderModes RenderMode = RenderModes.Video;
	public readonly RenderModes RenderMode = RenderModes.Video;

Conversation

ExtraBinoss commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

enderprism commented Feb 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ExtraBinoss commented Jan 10, 2026 •

edited

Loading