win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA #11993

thirumalai-qcom · 2025-03-24T05:36:08Z

Description

This PR re-introduces the win-mf plugin to OBS Studio, enabling Media Foundation-based encoding for H264, HEVC, and AAC. This plugin was previously removed in PR #7372 but is now being restored to support hardware-accelerated encoding on Windows on ARM (WoA) devices, specifically those utilizing Qualcomm hardware encoders. The Key changes include:

win-mf: Add base files for MFT encoders support on WoA
win-mf: Add Media Foundation H264 encoder for WoA
win-mf: Add Media Foundation HEVC encoder for WoA
win-mf: Add Media Foundation ACC encoder for WoA
cmake: Add win-mf plugin to OBS build for WoA

Motivation and Context

The primary motivation for these changes is to enable hardware-accelerated encoding on Windows on ARM (WoA) devices by reintroducing the win-mf plugin in OBS Studio. With the growing adoption of WoA devices, ensuring OBS Studio can leverage hardware encoders on this platform improves performance and efficiency, reducing CPU load while maintaining high-quality encoding.

By integrating the win-mf plugin back into the build system, we enable native support for Media Foundation-based encoding on WoA, ensuring users on this platform can leverage hardware acceleration for streaming and recording. This change ensures better utilization of Qualcomm hardware encoders, providing a more efficient and optimized encoding experience.

How Has This Been Tested?

The changes have been tested by building OBS Studio on a Windows on ARM (WoA) device with Qualcomm hardware. The build completed successfully, and basic functionality tests confirmed that OBS Studio runs as expected with the reintroduced win-mf plugin. Media Foundation-based encoding for H264, HEVC, and AAC was verified to function correctly, ensuring proper hardware acceleration on WoA devices utilizing Qualcomm hardware.

Types of changes

New feature (non-breaking change which adds functionality)

Checklist:

My code has been run through clang-format.
I have read the contributing document.
My code is not on the master branch.
The code has been tested.
All commit messages are properly formatted and commits squashed where appropriate.
I have included updates to all appropriate documentation.

PatTheMav

I'm wary about just reintroducing the old code as-is, particularly as we have progressed to C++17 and expect modern C++ coding practices, including but not limited to:

Use of anonymous namespaces instead of static functions is deprecated in C++ (static functions are only used for class methods in modern C++).
Use of constexpr as it's preferable over macros in almost all situations as they enable the compiler to optimise code better and sometimes resolve entire code paths at compile time
Abuse of macros instead of functions marked as constexpr (see above) which makes the code harder to debug

Just merging this would mean re-importing the code smell as well and I just don't see someone else volunteering to "refactor" it after it's been merged again.

PatTheMav · 2025-03-25T13:56:26Z

plugins/win-mf/CMakeLists.txt

This file should be adapted to current CMake guidelines instead of copied-over from the legacy variant.

@PatTheMav
I have Updated the CMakeLists.txt to meet with the current guidelines. Could you please review the changes and share any suggestions you might have?

The file still requires some revisions, follow the general skeleton of:

cmake_minimum_required(VERSION 3.28...3.30) find_package(<dependencies) # If any 3rd party dependencies add_library(<module> MODULE) add_library(OBS::<module> ALIAS <module>) target_sources( <module> PRIVATE <source and header files, sorted alphabetically> PUBLIC # If any public headers <public source and header files, sorted alphabetically> ) target_link_libraries( <module> PRIVATE <dependencies> PUBLIC <dependencies>. # If any public dependencies ) set_target_properties_obs(<module> PROPERTIES FOLDER plugins PREFIX "")

plugins/CMakeLists.txt

plugins/win-mf/data/locale/en-US.ini

derrod · 2025-03-25T15:28:14Z

I tested this on my Snapdragon X laptop and it does work as expected.

But Ideally this plugin should be either rewritten from the ground up, or replaced wholesale with just using the FFmpeg implemenation. At the very least, the AAC encoder could probably be removed. Of course since this is ancient it also doesn't support AV1 (which the X Elite chips do support).

Aside from what Pat mentioned, the naming also conflicts with #10471 right now.

plugins/win-mf/mf-aac.cpp

dd-han · 2025-04-13T16:22:39Z

Will this work with my older Surface Pro X (SQ2)?

If it works I can help test it.

thirumalai-qcom · 2025-05-20T12:48:47Z

Will this work with my older Surface Pro X (SQ2)?
If it works I can help test it.

@dd-han Thanks for offering to help.
I believe it should work with your Surface Pro X [SQ2] which is based on the Snapdragon 8cx, If you get a change to try it out, please let me know how it goes.

SA-Root · 2025-05-31T13:27:03Z

Hi, I tried using version 31.1.0 beta 1 on my pc with 8cx Gen 3, and I got 2 problems with MF based encoding:

When using h264_mf encoder, it seems to be using CPU to encode the video.
When I tried to use hevc_mf encoder, OBS immediately crashed when I click the Start Recording button. The corresponding log files are attached below.

2025-05-31 21-12-18.txt
Crash 2025-05-31 21-14-41.txt

RytoEX · 2025-05-31T16:02:51Z

Hi, I tried using version 31.1.0 beta 1 on my pc with 8cx Gen 3, and I got 2 problems with MF based encoding:

When using h264_mf encoder, it seems to be using CPU to encode the video.

You are using the Media Foundation encoder via FFmpeg. That is not what is in this PR, and this PR has not been merged, so it is not in OBS Studio 31.1.0 Beta 1. Please use our forums or Discord for assistance.

driver1998 · 2025-06-13T10:00:07Z

@SA-Root When using h264_mf encoder, it seems to be using CPU to encode the video.

You need to pass additional flags to make hardware encoding work with the ffmpeg backend.
See https://driver1998.github.io/posts/ffmpeg-hardware-video-encoding-on-qualcomm-woa/ (Chinese only)

This commit re-introduces the base files required for Media Foundation Transform (MFT) support in the win-mf plugin, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It includes core implementation files, language configuration files for localization, and the initial CMakeLists.txt for integration.

This commit introduces H264 encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-h264.cpp and mf-h264-encoder.cpp/.hpp to implement the encoding logic. Additionally, CMakeLists.txt is updated to include the new source and header files for proper integration into the win-mf plugin.

This commit introduces HEVC encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-hevc.cpp and mf-hevc-encoder.cpp/.hpp to implement the encoding logic. Additionally, CMakeLists.txt is updated to include the new source and header files for proper integration into the win-mf plugin.

This commit introduces AAC encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-aac.cpp and mf-aac-encoder.cpp/.hpp to implement the encoding logic. Additionally, CMakeLists.txt is updated to include the new source and header files for proper integration into the win-mf plugin

This commit updates the CMakeLists.txt file in the plugins directory to incorporate the win-mf plugin into the OBS build system. This change ensures that Media Foundation-based encoding is available for Windows on ARM (WoA) devices, utilizing Qualcomm hardware for efficient video Encoding.

PatTheMav

Left a first round of reviews, will still have to look at the HEVC and H264 encoder implementations.

Except for the missing newline issues I haven't commented on every repeat issue, so things like functions marked inline in compilation units, opaque variable names, "magic" global variables, should be fixed throughout and not just were I first encountered them in review.

PatTheMav · 2025-07-10T14:27:46Z

plugins/win-mf/CMakeLists.txt

+
+project(win-mf)
+
+set(


Do not use separate _SOURCES and _HEADERS variables, call target_sources with them directly (see also the other CMake build scripts in the project).

PatTheMav · 2025-07-10T14:28:30Z

plugins/win-mf/CMakeLists.txt

@@ -0,0 +1,43 @@
+cmake_minimum_required(VERSION 3.28...3.30)
+
+project(win-mf)


Do not specify a separate project for a plugin.

PatTheMav · 2025-07-10T14:31:36Z

plugins/win-mf/CMakeLists.txt

The file still requires some revisions, follow the general skeleton of:

cmake_minimum_required(VERSION 3.28...3.30) find_package(<dependencies) # If any 3rd party dependencies add_library(<module> MODULE) add_library(OBS::<module> ALIAS <module>) target_sources( <module> PRIVATE <source and header files, sorted alphabetically> PUBLIC # If any public headers <public source and header files, sorted alphabetically> ) target_link_libraries( <module> PRIVATE <dependencies> PUBLIC <dependencies>. # If any public dependencies ) set_target_properties_obs(<module> PROPERTIES FOLDER plugins PREFIX "")

PatTheMav · 2025-07-10T14:32:43Z

plugins/win-mf/mf-aac-encoder.cpp

+	*extraData_ = extraData;
+	*extraDataLength = sizeof(extraData);
+	return true;
+}


Missing newline at the end of file.

PatTheMav · 2025-07-10T14:35:34Z

plugins/win-mf/mf-aac-encoder.cpp

+
+#include <array>
+
+inline void LogAAC(const obs_encoder_t *encoder, int level, const char *format, ...)


It looks like this function (and the next) are not supposed to have external linkage and are only used within this compilation unit, so they should instead both be wrapped in an anonymous namespace and could even be marked as constexpr:

namespace { constexpr void LogCOMError(const obs_encoder_t *encoder, const char *operation, HRESULT hr) { _com_error err(hr); LogAAC(encoder, LOG_ERROR, "%s failed, %S (0x%08lx)", operation, err.ErrorMessage(), hr); } }

The inline keyword in most cases should only be used to make it explicit that a function or variable is defined "in-line" with its signature in header files, to allow what would otherwise be a violation of ODR.

Using constexpr instead allows the compiler to resolve it at compile time (similar to how a macro would be expanded before compilation) if the associated conditions are fulfilled.

PatTheMav · 2025-07-10T16:42:29Z

plugins/win-mf/mf-encoder-descriptor.cpp

+	const char *id;
+	EncoderType type;
+
+} guidNameMap[] = {


I'd like to think that it would be better for readability and understanding to split the type definition and the initialisation of a new variable with that type into separate statements.

PatTheMav · 2025-07-10T16:48:47Z

plugins/win-mf/mf-encoder-descriptor.cpp

+
+	ComHeapPtr<WCHAR> guidW;
+	StringFromIID(guid, &guidW);
+	std::string guidString = MBSToString(guidW);


Correct me if I'm wrong, but C++ has no built-in conversions from wstring to string? IIRC they even deprecated such functions in more recent C++ versions.

PatTheMav · 2025-07-10T16:49:18Z

plugins/win-mf/mf-encoder-descriptor.hpp

+	bool isHardware;
+	EncoderType type;
+};
+}; // namespace MF


Missing newline at the end of file.

PatTheMav · 2025-07-10T16:51:21Z

plugins/win-mf/mf-hevc-encoder.cpp

+	pendingRequests--;
+
+	return true;
+}


Missing newline at the end of file.

PatTheMav · 2025-07-10T16:51:27Z

plugins/win-mf/mf-hevc-encoder.hpp

+	std::atomic<UINT32> outputRequests{0};
+	std::atomic<UINT32> pendingRequests{0};
+};
+} // namespace MF


Missing newline at the end of file.

derrod · 2025-07-10T19:25:27Z

Couple more notes:

You should remove the AAC encoder, it was removed due to glitches and we don't really have a reason to reintroduce it
- If you insist on keeping it, add the .codecs property so it is correctly recognised by the encoder selectors, currently it's not selectable due to missing codec information
MediaFoundation does support DirectX textures as encoder input since Windows 8.1 and that should be preferred over copying data to system RAM

WizardCM added Seeking Testers Build artifacts on CI New Feature New feature or plugin labels Mar 24, 2025

thirumalai-qcom force-pushed the mft-support branch 2 times, most recently from 55aa652 to ea69f55 Compare March 25, 2025 11:32

thirumalai-qcom marked this pull request as ready for review March 25, 2025 11:55

PatTheMav reviewed Mar 25, 2025

View reviewed changes

derrod reviewed Mar 30, 2025

View reviewed changes

plugins/win-mf/mf-aac.cpp Outdated Show resolved Hide resolved

thirumalai-qcom force-pushed the mft-support branch from ea69f55 to 109de95 Compare April 3, 2025 13:35

thirumalai-qcom force-pushed the mft-support branch 4 times, most recently from 13c8aac to 2aa95e6 Compare May 20, 2025 11:10

thirumalai-qcom requested review from PatTheMav and RytoEX May 20, 2025 12:49

rbqvq mentioned this pull request Jun 2, 2025

build(windows): add arm64 initial support LizardByte/Sunshine#3905

Draft

15 tasks

thirumalai-qcom force-pushed the mft-support branch from 2aa95e6 to e066af7 Compare June 13, 2025 10:52

thirumalai-qcom added 5 commits July 10, 2025 13:53

thirumalai-qcom force-pushed the mft-support branch from e066af7 to f7fd3aa Compare July 10, 2025 08:23

PatTheMav requested changes Jul 10, 2025

View reviewed changes

		@@ -0,0 +1,43 @@
		cmake_minimum_required(VERSION 3.28...3.30)

		project(win-mf)


		#include <array>

		inline void LogAAC(const obs_encoder_t encoder, int level, const char format, ...)


		project(win-mf)

		set(

Uh oh!

win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA #11993

Are you sure you want to change the base?

win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA #11993

Conversation

thirumalai-qcom commented Mar 24, 2025

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

PatTheMav left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

derrod commented Mar 25, 2025

Uh oh!

Uh oh!

dd-han commented Apr 13, 2025

Uh oh!

thirumalai-qcom commented May 20, 2025

Uh oh!

SA-Root commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RytoEX commented May 31, 2025

Uh oh!

driver1998 commented Jun 13, 2025

Uh oh!

PatTheMav left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

derrod commented Jul 10, 2025

Uh oh!

Uh oh!

SA-Root commented May 31, 2025 •

edited

Loading