Capture Refactor #2360
Merged
Capture Refactor #2360
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces Mode class to represent specific capture configurations with: - Resolution (width/height) - Frame rate - Codec (Uncompressed, JPEG, H264, HEVC) - Pixel format (RGB/YUV variants) Adds Device::getModes() to query supported capture modes and new Capture::create(device, mode) factory method for explicit mode selection. Implements comparison operators and stream output for Mode class.
Adds msw::ComPtr template class for managing COM interface lifetimes without ATL dependency. Provides automatic AddRef/Release management with copy semantics suitable for traditional COM reference counting. Updates Windows build configuration and fixes .clang-format for compatibility with vc2019-included clang-format version (SortIncludes: Never -> false).
Implements Device::getModes() to enumerate available capture formats with resolution, frame rate, and pixel format information. Adds mode-based initialization through initWithDevice:mode: for explicit format control. Adds session error notifications and device disconnection detection. The isCapturing() method now checks session state to detect unplugged devices, preventing crashes from attempting to access disconnected hardware.
Implements CaptureImplGStreamer with full Mode support for Linux video capture. Uses GStreamer's v4l2src for camera access with automatic format negotiation and conversion pipelines. Supports both compressed (MJPEG, H264, HEVC) and uncompressed (YUV/RGB) formats with automatic decompression when needed. Enumerates device capabilities through V4L2 and provides Mode-based initialization for explicit format control. Handles device disconnection gracefully and provides detailed error reporting. Updates build system with GStreamer dependencies and adds GStreamer to the dependencies documentation table.
Replaces videoInput library with direct DirectShow implementation using native COM interfaces. Provides full Mode support with automatic format enumeration and negotiation.
Adds ImGui interface for runtime device and mode selection. Users can choose capture devices from a dropdown and select specific modes (resolution, framerate, format) or use automatic mode selection. Displays current capture details including resolution, connection status, frame rate, codec, pixel format, and color model. Demonstrates the new Mode-based capture API with practical usage examples.
Enhances test application with automated stress testing that randomly switches between devices and modes. Includes device refresh capability to detect hot-plugged devices without restarting the app. Adds vc2019 project files and CMake configuration. Updates to use ImGui for runtime control and monitoring of stress test parameters.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a GStreamer-based Linux
Captureimplementation, as well as explicit video capture mode enumeration and selection, enabling applications to specify exact resolution, pixel format, and codec requirements. Implementation includesModesupport across all platforms (AVFoundation, DirectShow, GStreamer) with enhanced device enumeration and hot-plug detection.Key Changes
Capture::ModeAPI - Explicit enumeration and selection of resolution, internal pixel format (RGB24, YUV420P, NV12, etc.), and internal codec (Uncompressed, JPEG, H264, HEVC)A note on GStreamer vs. v4linux: This implementation uses GStreamer rather than v4l2 directly, diverging from #2061 and the master...whg:CaptureLinux. While v4l2 is part of the Linux kernel and requires minimal dependencies, I believe GStreamer provides some advantages for video capture:
Movieplayback on Linux, making capture implementation architecturally consistentThe tradeoff is an additional system dependency, but GStreamer is widely available on Linux distributions and already required for video playback functionality.
Tested with macOS, Windows 10, Windows 11, Ubuntu 24.04, and Ubuntu 25.10 beta using a Zed2i, an ELP 4K USB camera, an Insta360 2C camera, the Apple Studio Display camera (macOS only), an iPhone remote camera (macOS only), and an ASUS laptop camera (Win11 only).