Add vhost-device-media backend for virtio-media#944
Add vhost-device-media backend for virtio-media#944aesteve-rh wants to merge 7 commits intorust-vmm:mainfrom
Conversation
aesteve-rh
requested review from
dorindabassey,
epilys,
stefano-garzarella,
stsquad and
vireshk
as code owners
This commit introduces vhost-user-media device. The virtio-media device first appeared in virtio-spec v1.4 specification[1]. It standardizes how guest applications communicate with media devices on the host, such as encoders, decoders, and cameras. The protocol relies on the V4L2 (Video4Linux2) specification to format the communication between the guest and host, exposing the host's media devices to the virtualized environment. The low-level implementation of this backend for the virtio-media protocol is provided by the device crate from the virtio-media repository[2]. [1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022 [2] https://github.com/chromeos/virtio-media Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit introduces a memory allocator for managing the shared memory regions used by the virtio-media device. The allocator is responsible for finding and managing free regions within the shared memory, ensuring that buffers are allocated without conflicts. The new `MediaAllocator` tracks available memory pools and provides a mechanism for allocating and releasing memory blocks of a requested size, while also handling alignment requirements. This replaces the previous, more error-prone approach of managing address ranges in a simple Vec. Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit introduces a new backend that uses FFmpeg for software-based video decoding. This provides a hardware-independent way to test and use the virtio-media device for video decoding tasks. The new `FfmpegDecoder` backend leverages the `virtio-media-ffmpeg-decoder` crate from the virtio-media repository [1]. It is exposed as a new backend option and can decode common video formats such as H.264, VP8, and VP9 without requiring any specific host hardware. This addition is useful for development and testing and also serves as a reference implementation for a virtio-media decoder device. [1] https://github.com/chromeos/virtio-media/tree/main/extras/ffmpeg-decoder Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit refactors the vhost-device-media crate to make each backend an optional feature. This allows users to compile only the backends they need, reducing the final binary size and dependency footprint. The following features have been introduced: - `simple-capture`: Enables the simple V4L2 capture device. - `v4l2-proxy`: Enables the V4L2 proxy device. - `ffmpeg`: Enables the FFmpeg software decoder. The `v4l2-proxy` feature is enabled by default. The `v4l2r` and `virtio-media-ffmpeg-decoder` dependencies are now optional and are only included when their corresponding features are enabled. The source code has been updated with `#[cfg]` attributes to support conditional compilation of the backends. Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add unit tests to existing code for vhost-device-media backend. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Albert Esteve <aesteve@redhat.com>
This change reorganizes the vhost-device-media crate to follow a similar pattern used by other vhost-device crates, including the template, splitting the codebase into a library component (lib.rs) and a minimal binary entry point (main.rs). A new src/lib.rs was added containing the core backend logic, including backend creation functions (create_*_device_config), and serve functions for each backend type. The main.rs file was simplified to focus solely on CLI argument parsing and delegating to the library. Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add V4l2DeviceType enum class that reads the device path received in the command line and resolves the device type according to the VFL_TYPE_* kernel constants. Added tests to verify path resolution is correct. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Albert Esteve <aesteve@redhat.com>
aesteve-rh
force-pushed
the
virtio-media
branch
from
c3a8789 to
4a6c56d
Compare
| virtio-media = { git = "https://github.com/chromeos/virtio-media", rev = "c6cc93a"} | ||
| virtio-media-ffmpeg-decoder = { git = "https://github.com/chromeos/virtio-media/", package = "virtio-media-ffmpeg-decoder", rev = "c6cc93a", optional = true} | ||
| v4l2r = { git = "https://github.com/Gnurou/v4l2r", rev = "7b44138", optional = true } |
There was a problem hiding this comment.
v4l2r and virtio-media are on crates.io, are they behind this revision? Maybe we can request new releases (and also publishing of virtio-media-ffmpeg-decoder)
There was a problem hiding this comment.
The problem with virtio-media from crates.io is that ffmpeg-decoder has the dependency set with a relative path (https://github.com/chromeos/virtio-media/blob/main/extras/ffmpeg-decoder/Cargo.toml#L13).
Cargo treats these as different crates, so VideoDecoder expects VideoDecoderBackend from crates.io 0.0.7, but FfmpegDecoder implements VideoDecoderBackend from the git version. Rust sees these as different traits, causing E0277: the trait bound FfmpegDecoder: VideoDecoderBackend is not satisfied.
I did send this PR some time ago: chromeos/virtio-media#19
But I must confess I did not follow up. And maybe I did not explain the issue good enough.
Maybe I can use the crates.io version and patch it in the root Cargo.toml, as a workaround.
Asking them to publish virtio-media-ffmpeg-decoder sounds reasonable.
For v4l2r I tested and it works now with the latest released version, so that much is good. I will change it.
There was a problem hiding this comment.
We could try to do that after this PR is merged then :) It's not that we need crates.io publishing anyway.
|
Thank you for this PR @aesteve-rh |
|
@dorindabassey virtio-video is still under development (see virtio-comment list for v10). It's not replaced by virtio-media. |
3 participants
Summary of the PR
This PR introduces vhost-user-media device.
The virtio-media device first appeared in virtio-spec v1.4 specification[1].
It standardizes how guest applications communicate with media devices on
the host, such as encoders, decoders, and cameras.
The protocol relies on the V4L2 (Video4Linux2) specification to
format the communication between the guest and host, exposing the
host's media devices to the virtualized environment.
The low-level implementation of this backend for the virtio-media
protocol is provided by the device crate from the virtio-media
repository[2].
Implementation is still WiP as it is waiting for vhost release and SHMEM_MAP/UNMAP patch to be merged in Qemu (so that the specs align with current implementation).
[1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022
[2] https://github.com/chromeos/virtio-media
Requirements
Before submitting your PR, please make sure you addressed the following
requirements:
git commit -s), and the commit message has max 60 characters for thesummary and max 75 characters for each description line.
test.
Release" section of CHANGELOG.md (if no such section exists, please create one).
unsafecode is properly documented.