src: Get video formats using Device Monitor by joaoantoniocardoso · Pull Request #281 · mavlink/mavlink-camera-manager

joaoantoniocardoso · 2023-09-04T12:28:25Z

Merge after #607

update 13/05/26:

This PR is the first step towards supporting libcamera. In this step, we move from the v4l crate to give us the stream formats to get them from the GStreamer DeviceMonitor.

The further steps are already planned, and libcamera will be functional in the next PR.

With these changes, the v4l crate is only used for the controls API (which is fine).

update 09/29/25:

This seems very promising -- so far, I have already replaced the v4l formats and the v4l device discovery.

I'm thinking we should refactor the source/pipeline part of the application to fully make use of this Device Monitor:

I still need to reason about the architecture, but I think we can safely say that at least we should split our "Pipelines" (v4l2_pipeline.rs, etc) into a source (pipeline: source -> capsfilter -> parser -> proxysink) + transcoding pipeline (pipeline: proxysrc -> video_tee -> rtp_payloader -> capsfilter -> rtp_tee, as well as the future optional decoder+encoder), which will then connect to our many sink (pipeline: proxysrc -> sink).
Then we create a Device enum (w/ VideoSource and AudioSource variants) and a DeviceProvider trait, implemented by both this GSTDeviceMonitor and by the OnvifDiscovery, to give us a common interface.
Each "gst::Device" is also an element factory provider, which allows you to create its source element with no effort, which will help us to more easily reach libcamera and audio support.

Helps #372, #273, #139, #79

patrickelectric · 2026-05-13T19:48:11Z

@joaoantoniocardoso can you rebase this PR ?

…ded formats

joaoantoniocardoso · 2026-05-13T20:34:46Z

@patrickelectric oops, accidentally clicked to merge before review; Can you review it and I make a following PR to adjust any discrepancies?

patrickelectric

I had no time to finish the review today. Will do the final commit tomorrow.

patrickelectric · 2026-05-13T21:41:06Z

+    info!("GST Device Providers: {providers:#?}");
+
+    Ok(())
+}


just for curiosity, is there a reason for not having this in the default call of Manager ?

patrickelectric · 2026-05-13T21:49:22Z

+            };
+
+            device.properties().iter().any(|s| {
+                let Ok(api) = s.get::<String>("device.api") else {


It would be nice to have a comment for this string:

# device.api is an inner string defined by gst for which api this device is accessed with: Ex: alsa, v4l..

It would be even better to have such strings as "device.path" to be defined as a global private property that explains that is a GST property string for this field

patrickelectric · 2026-05-13T21:53:15Z

+                let device = device_weak.upgrade()?;
+                let name = device.display_name().to_string();
+                let properties = device.properties()?;
+                let device_path = properties.get::<String>("device.path").ok()?;
+                let bus = properties.get::<String>("v4l2.device.bus_info").ok()?;


isn't it better to have abstraction for it under gst_device_monitor as the function device_caps ?

patrickelectric · 2026-05-13T21:54:11Z

    Yuyv,
 }

-#[derive(Apiv2Schema, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)]


This commit is unnecessary moving this structure.

I'm slowly moving things to get a top-to-bottom declaration, but if I added another commit, we'd have multiple commits touching the same lines.

patrickelectric · 2026-05-13T22:00:03Z

+        for candidate in candidates.iter() {
+            let VideoSourceType::Local(camera) = candidate else {
+                continue;
+            };
+            if camera.name != source.name {
+                continue;
+            }
+            formats.insert(
+                candidate.inner().source_string().to_string(),
+                candidate.formats().await,
+            );
+        }


Suggested change

for candidate in candidates.iter() {

let VideoSourceType::Local(camera) = candidate else {

continue;

};

if camera.name != source.name {

continue;

}

formats.insert(

candidate.inner().source_string().to_string(),

candidate.formats().await,

);

}

for candidate in candidates.iter().filter(|candidate| {

matches!(candidate, VideoSourceType::Local(camera) if camera.name == source.name)

}) {

formats.insert(

candidate.inner().source_string().to_string(),

candidate.formats().await,

);

}

patrickelectric · 2026-05-13T22:03:39Z

+
+fn get_device_formats_using_gstreamer(
+    device_path: &str,
+    _typ: &VideoSourceLocalType,


why is this necessary ?

The main idea is to use the DeviceMonitor's Device to interface the "local" kind of sources, and the previous approach to get formats was specific to v4l, so it wouldn't work for libcamerasrc or other sources.

joaoantoniocardoso requested a review from patrickelectric September 4, 2023 12:28

joaoantoniocardoso mentioned this pull request Sep 4, 2023

Google Coral camera not detected on /dev/video3 #273

Open

joaoantoniocardoso commented Sep 4, 2023

View reviewed changes

Comment thread src/video/video_source_local.rs Outdated

joaoantoniocardoso force-pushed the use_gst_device_monitor branch 2 times, most recently from 3639f6f to e25d761 Compare September 29, 2025 04:02

joaoantoniocardoso force-pushed the use_gst_device_monitor branch 2 times, most recently from ab854c7 to 7bb361a Compare May 13, 2026 14:01

joaoantoniocardoso marked this pull request as ready for review May 13, 2026 17:15

joaoantoniocardoso added 5 commits May 13, 2026 17:07

src: lib: video: gst_device_monitor: Add module

54a2aba

src: lib: video: local: Use gst::Device for cameras_available

f7a12bc

src: lib: video: types: Derive Hash and accept YUY2 alias

3be55f8

src: lib: video: local: Replace VIDEO_FORMATS cache with caller-provi…

20b0bdb

…ded formats

src: lib: video: local: Use gst::Device for formats discovery

f85a476

joaoantoniocardoso force-pushed the use_gst_device_monitor branch from 7bb361a to f85a476 Compare May 13, 2026 20:07

joaoantoniocardoso merged commit 797996f into mavlink:master May 13, 2026
6 checks passed

patrickelectric reviewed May 13, 2026

View reviewed changes

joaoantoniocardoso mentioned this pull request May 20, 2026

GST Device Monitor - Part II #609

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

src: Get video formats using Device Monitor#281

src: Get video formats using Device Monitor#281
joaoantoniocardoso merged 5 commits into
mavlink:masterfrom
joaoantoniocardoso:use_gst_device_monitor

joaoantoniocardoso commented Sep 4, 2023 •

edited

Loading

Uh oh!

Uh oh!

patrickelectric commented May 13, 2026

Uh oh!

Uh oh!

joaoantoniocardoso commented May 13, 2026 •

edited

Loading

Uh oh!

patrickelectric left a comment

Uh oh!

patrickelectric May 13, 2026

Uh oh!

patrickelectric May 13, 2026

Uh oh!

patrickelectric May 13, 2026

Uh oh!

patrickelectric May 13, 2026

Uh oh!

joaoantoniocardoso May 13, 2026

Uh oh!

patrickelectric May 13, 2026

Uh oh!

patrickelectric May 13, 2026

Uh oh!

joaoantoniocardoso May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

joaoantoniocardoso commented Sep 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

update 13/05/26:

update 09/29/25:

Uh oh!

Uh oh!

patrickelectric commented May 13, 2026

Uh oh!

Uh oh!

joaoantoniocardoso commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

patrickelectric left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joaoantoniocardoso commented Sep 4, 2023 •

edited

Loading

joaoantoniocardoso commented May 13, 2026 •

edited

Loading