New UI Overlay with FPV for better LR monitoring

[ValkyrieDomi]

Hello,

Coming again with some idea for better overlay in MWP

Key Ideas:

• This Optional Overlay is user selectable
• Includes FPV Feed
• Telemetry widgets bar below FPV Feed with better spacing/scaling and positions
• Side map

This is just "concept art", we have only 7 widgets and after trying more layout styles, this was the best one to utilize the space and available widgets.

Ideally some better "formatting" of widgets would be needed. Some of them has empty space around, or are not centered or scaled properly. So some polishing and scaling would be nice.

On an actual device all perfectly visible.

[b14ckyy]

I really like that idea. I wonder how well that will work with the Video player being its own process? @stronnag

[stronnag]

Why would be video player be its own process, rather than a device node / URI?

[b14ckyy]

Oh I thought it is an extra process right now with that video player module of GTK4

[stronnag]

Regardless, I'm looking forward to the PR ...

[b14ckyy]

I think that was more a feature request 😅

[ValkyrieDomi]

It was very more a feature request.
I can only do very primitive Arduino programming and CNC professionally. MWP won't accept my G-codes.

[stronnag]

It was very more a feature request. I can only do very primitive Arduino programming and CNC professionally. MWP won't accept my G-codes.

Perfectly qualified then.
I guessed was a feature request.

[stronnag]

In theory this is achievable:

• The "container" for the main window objects is flexible, having alternate layouts is not hard.
• Having a horizontal panel split between the map and embedded video is quite trivial
• Having the right hand pane further split vertically with an embedded video and "something else" is trivial

Then we get to the "something else": a new layout of the already flaky panel. I'm really not sure I have the energy to revisit this disaster area, at leas not in any kind of configurable way. The following is real code generated, not a cut and paste screen shot.

[b14ckyy]

I can make a poll what others think about that.
Or link this thread on FB.

[ValkyrieDomi]

Lovely!

I´d agree, if we don´t have tons of widgets, there is nothing (no need) to customize.

AHI good size, anything else does not need to be any smaller or any bigger. Does not make sense to move with it.

From Real Flight perspective, u have your video feed with OSD providing more detailed info, and when feed is lost, you still have at least widget on your radio with more tele data in such a situation.

The main point is tho to split the screen and accommodate alternative fixed widget bar, but mainly the video feed for LR monitoring purposes alongside with the map and live position of the airplane.

[b14ckyy]

Thats a perfect example why I would not mind to get rid of the panel size customization.

[stronnag]

It seems like the whole panel will be getting some more attention. One simple and immediate thing would be to set wide-handles on the panes to make the dividers more obvious

[ValkyrieDomi]

Here we go, much better. I usually prefer to have AHI on top for full size.
Can you put or make an option for another row for the widgets? Example below shows too much space after consolidation.

[stronnag]

OK, I'm almost sold on the idea.

While I have your attention; I had an idea about an option to show altitude as AGL where
(a) the user chooses to display altitude as AGL; and
(b) has the relevant DEMs available

We know the AMSL at launch, we know the AMSL at the vehicles location and the altitude difference between home and the vehicle; so we can simply compute the AGL at the vehicle's location.

We also now have freely / easily available and reasonably accurate global DEMs ; the only downside being that the best option (Mapzen, which mwp uses) provides bathymetric depths over the sea (but not lakes). Which might be confusing in some, albeit less common, use cases.

[b14ckyy]

I like the idea of AGL. Do you think of a visual or numerical representation? Graphical would probably be quite some work.

This could be even extended with warning when approaching terrain.

Regarding altitude data. I did not know that, so the elevation data does not take the water surface as "surface" but actually the sea bed over the ocean. That's interesting. Maybe for the terrain follow data you can just filter out any terrain data below 0m AMSL and consider that as ground?

It could cause issues in very very rare areas of the world where you have land below sea level though.

[ValkyrieDomi]

For me AGL is OK. I am usually referring to AGL instead of AMSL when flying RC planes.

[stronnag]

For the moment, I'll leave it "as is"

• For lakes (e.g. your Terrain-follow.mission inland water is reported as the adjacent land, so it makes some sense (for your example, 89-93m).
• For boats and submarines, on / under the sea it might useful to avoid running aground.
• There are indeed places where people fly below AMSL (Netherlands for example; I have a 'wilco' log I must checkout).

[stronnag]

Setting alt-prefer-agl = false

Setting alt-prefer-agl = true

[stronnag]

So let's redesign / implement the whole panel thing such that:

• We use the same container for each orientation (so switching is easy)
• The user can configure (by some magic means TBD) the space in terms of:
  • rows and columns (arbitrary number, you own it).
  • content of each cell
  • whether the cell spans a number of rows or columns
  • whether the cell shows in both orientation
• When the orientation changes, the the meaning of rows and columns flips, so one definition covers both cases.
• The size of each cell is determined by the content so the user cannot "lose" the cell.
• If a row or column is empty it is not displayed, doesn't affect anything else.

So in the example below, (for demo/test hard-coded) the cells are set by:

			//          name     x  y   w    h   span
			load_widget("ahi",   0, 0, 200, 200, 2);
			load_widget("rssi",  1, 0, -1,  -1, 1);
			load_widget("vario", 1, 1, -1,   -1, 1);
			if (is_vertical) {
				load_widget("wind",  2, 0, -1,   -1, 1);
				load_widget("dirn",  2, 1, -1,   -1, 1);
			}
			load_widget("flight", 3, 0, -1,   -1, 2);
			load_widget("volts", 4, 0, -1,   -1, 2);

• Hope is obvious enough that a Arduino / CNC programmer can follow it
• That two widgets only appear in the vertical layout doesn't matter for the horizontal case
• It is necessary to specify a size for AHI, because "reasons".

[ValkyrieDomi]

Hope is obvious enough that a Arduino / CNC programmer can follow it

No worries there. Such a things we already did in kindergarten 👍

Anyway, lovely work as always! Already looks more....... polished..., what have you done? Waxing?

[stronnag]

This is NOT "photoshopped"

[b14ckyy]

Is there even Photoshop for Linux? ;)

But that looks good. So for now the container are fixed in size right?

[stronnag]

This is NOT Gimped

Note: There is no video widget wired in yet ...

[stronnag]

You should be able to use mwp's atstart setting to run it silently as startup.

gsettings set org.stronnag.mwp atstart 'c:\\pathto\\mediamtx.exe [any parameters if needed]'

• You may (or not) need the full path
• You may (or not, I suspect you do) need to escape any back-slashes if using gsettings.

Or set it via "Edit/All Settings" (in which it will handle back-slash escapes automagically for you).

If it is possible in Windows to kill a process by name (e.g. equivalent of Linux pkill -f mediamtx), you can also stop it via the atend setting when mwp exits.

[stronnag]

Possibly

taskkill -f -im "mediamtx.exe"

Thus

gsettings set org.stronnag.mwp atend 'taskkill -f -im "mediamtx.exe"'

[b14ckyy]

yep I did that with atstart and the mediamtx process is running but that's it. It does not do anything so it does not seem to load its config file as it does when manually started.

Might just make a .cmd file to launch it and see if that works. After dinner.

[stronnag]

yep I did that with atstart and the mediamtx process is running but that's it. It does not do anything so it does not seem to load its config file as it does when manually started.

Might just make a .cmd file to launch it and see if that works. After dinner.

Did you try passing the path of the config as well?

gsettings set org.stronnag.mwp atstart 'c:\\pathto\\mediamtx.exe c:\\mediamtxstuff\\mediamtx.conf'

[b14ckyy]

No I just launched the exe as I would manually, there it works. but I can give it a shot.

[stronnag]

Some random video posted on Discord recently (this is not Gimped either):

[ValkyrieDomi]

Wooooow! Lovely! ❤

No worries, we believe you it is not GIMPED. U just took a little inspiration from ArduPilot, don´t you?

Don´t mind my jokes, nor take them offensive. They call me Domi The Little Joker. 👼

[stronnag]

Never used Ardupilot (or even seen it, other than videos of evil TU95s getting a make over).

[b14ckyy]

Next step: Virtual OSD Like ardupilot :P

And no I am just joking. :D

[stronnag]

PRs are welcome ... as usual (and I'm just joking, as usual, sort of).

[b14ckyy]

Its pointless anyway. All the information are in the widgets already.
so there is no point in that except some "cool-factor"

I thought it would be fun to have the Video panel as a massive AHI but then I realized, if video is off we have a full screen map so... also pointless :D

[stronnag]

In the unlikely event that anyone else wants to play with this, it's in the new_panel branch (which one would need to checkout and build).

In order to get the FPV view, it is necessary to set the MWP_USE_FPVMODE environment variable (to anything), for example:

$ MWP_USE_FPVMODE=1 mwp

[stronnag]

flightlog2kml-linux-aarch64-1.0.33.tar.gz

[b14ckyy]

Ehm I would need the risc64 for the Pi :)

[stronnag]

Ehm I would need the risc64 for the Pi :)

I think not; Rpi is arm64 (aka aarch64).

[b14ckyy]

Iiiiinteresting because till now I used risc64 all the time. I thought thats for Arch Linux :O lemme try.

[stronnag]

OK, I can see where the confusing arises. You find something of performance improvement with native (vice emulated) code.

[stronnag]

Just a simple little proof of concept for the switcher ... next step, move this into mwp ... may be less straightforward.

Wayfarer_2025-08-24_103930.mkv.mp4

(Unfortunately) it was necessary to remove the frames around each item (for now).

[b14ckyy]

If it risks to break too much things, I guess we can also live without live switching unless you like the challenge :)

[stronnag]

The frames are now back.
We are having live switching ... famous last words.

[ValkyrieDomi]

Superb! 👀

[stronnag]

mwp mode switch

• In the video below, the replay and embedded videos are entirely disconnected

• As the embedded video is playing from a file, it will restart from the beginning each time (I'll probably fix that).

• The mode view mode is memorised for next time mwp is started

• The last embedded video source is memorised and will be started the next time mwp is started (if the source is available).

• Restarting USB webcam sources is fixed.

• The "opinionated" widget set for each mode is not (easily) configurable ... yet; but see below

• Merged into GH development branch (you can discard new_panel)

mwp-preview.mp4

Widget configuration

• Opinionated default set at first run
• Stored in ~/.config/mwp/.panel.conf (other OS eqivalents etc.)

$ cat ~/.config/mwp/.panel.conf 
# mwp panel v2
# name, row, col, width, height, span, mode
ahi,0,0,200,200,2,3
rssi,1,0,-1,-1,1,3
vario,1,1,-1,-1,1,3
wind,2,0,-1,-1,1,1
dirn,2,1,-1,-1,1,1
flight,3,0,-1,-1,2,3
volts,4,0,-1,-1,2,3

• Size of -1 means ignore
• col(umn), row relate to a vertical panel, they are flipped for horizontal
• span is number of rows/columns to span (so above ahi, flight and volts span two row / columns (orientation dependent))
• mode 1=vertical, 2=horizontal, 3=both.
• End user is not really expected to edit this ....

[b14ckyy]

stderr.txt

I launched 2 times. First time I selected the webcam again while already running > Video disappears
Second time I switched FPV mode off and back on and then tried to select the webcam again

[stronnag]

I'll do a windows .exe when I think things are stable enough to make it worthwhile. Later today, tomorrow maybe.

[b14ckyy]

No idea. 'mwp' just asked the OS what cameras there are and offers a selector. The default maybe ?

So seems like you call the video device with v4l2-ctl but I could not find info if it takes defaults from the connected hardware, its own defaults or if it locks in with the first best settings that return a stream. But it does not return the highest framerate automatically as it seems. in 1080p that webcam can do 60fps and I just tested that with Guvcview

[b14ckyy]

Update. It seems the defaults are actually adjustable but they are not persistent as they are usually set by the application.
https://askubuntu.com/questions/295637/v4l2-change-default-setting

So it would need a startup script to set new default resolution or framerates. Thats a bit easier on windows as here the settings can be changed on driver level.

never mind that's just for image parameters and cropping controls. Stream resolution and framerate needs to be set on call anyway.

PS: this would be the windows equivalent with ffmpeg to directly capture from USB cameras
https://trac.ffmpeg.org/wiki/DirectShow

[stronnag]

If you really want to go to the trouble, you can do the same sort of thing with gstreamer; one of many benefits of the Gtk4 migration was that I can just use Gtk.Video / Gtk.MediaStream and avoid the pain.

[stronnag]

I tried my one "detachable" USB camera (a 2015? Mobius) on the RPi4. No problems switching modes, albeit with the later mwp commit that is a bit more aggressive about closing any extant stream. FWIW, log attached, gzipped because GH doesn't like this text file.

• Arch vice Debian
• RPI4 vice RPI5
• Much older camera
• X11 vice Wayland etc.

rpi4-mwp-stderr.txt.gz

But no obvous reason why one works flawlessly and the other doesn't.

[stronnag]

mwp doesn't do anything special dependent on resolution or interface type. Let me add some more diagnostics afer "Embedded video" to see how far it gets.
I'll update here when it's pushed.

[stronnag]

Not sure how much we will get out of d231691 (last commit)

• More rigorous attempt to close extant video on switch
• Error handler on embedded video, but I'll be surprised if shows anything

Can you also see if you video device works in the "standard" windowed mwp video player? That might show something more.

[b14ckyy]

So the behavior is the same between video player and FPV view.

stderr.txt

Additionally I used the 4K capture card on USB3 again to trigger the crash at >1080p res.

18:53:29.659274 VideoDevs </dev/video0> <USB3 Video: USB3 Video>
18:53:29.659293 VideoDevs </dev/video23> <pispbe>
18:53:29.659299 VideoDevs </dev/video24> <pispbe>
18:53:29.659305 VideoDevs </dev/video25> <pispbe>
18:53:29.659310 VideoDevs </dev/video26> <pispbe>
18:53:29.659316 VideoDevs </dev/video31> <pispbe>
18:53:29.659321 VideoDevs </dev/video32> <pispbe>
18:53:29.659327 VideoDevs </dev/video33> <pispbe>
18:53:29.659332 VideoDevs </dev/video34> <pispbe>
18:53:29.758949 Main window activated true
18:53:30.046098 {id:1, addr:3C:38:F4:56:E5:56, name:3C-38-F4-56-E5-56, type:BLUEZ_DEV_DEVICE_TYPE_OTHER, rssi:-63, connected:false}
18:53:31.621366 {id:2, addr:00:BB:C1:3E:92:C2, name:TS6100 seri, type:BLUEZ_DEV_DEVICE_TYPE_OTHER, rssi:-67, connected:false}
18:53:33.490733 :DBG: embeded video player close (is null=true)
18:53:40.029752 :DBG: Video v4l2:///dev/video0
18:53:40.393886 :DBG: Embedded video 1152x2048 

[b14ckyy]

P.S. don't get confused why the Dell Webcam shows 2 times now. The second one is the IR camera part since this cam is Windows Hello capable So it shows up as 2 cams when connected to USB 3.0. Had it on 2.0 before.

[b14ckyy]

So I did some more testing now.
On Ubuntu I see the same issue. Cam does not turn off most of the time when closing the video window or switching off FPV mode.
On the raspberry Pi4 I even get the issue with the Microsoft cam that works fine on the Pi5 most of the time. The Pi4 I tested with RaspberryOS Trixie. Ubuntu was still on the Pi5.

[stronnag]

Here's some experimental Windows files.

• Open the archive
• Copy the folders over the last installed folders (25.08.16), replacing all the enclosed files.

Known issues

• Map pane doesn't fully resize on switch

trivially tested with a RTSP source.

mwptools-win64-2025-08-24.zip

[b14ckyy]

I just checked myself. The settings is set to "true" but the path does not disconnect. Its the same behavior when false or true.

[stronnag]

Interesting. So Windows/Gtk must not actually report it as a touch screen, the new behaviour is initially dependent on the device being identified as touch, and then the setting is checked.

That explains a lot.

That's a real pain. On my Linux tablet, if I drag on the screen, the icons disconnect and don't jump; if I then move an icon via the touchpad or mouse, standard / no disconnect behaviour. No jumps in either case.

So probably the answer is

• revert the touch-drag-disconnected setting to default to off
• On windows, just depend on touch-drag-disconnected with the side effect that you also will have disconnected behaviour when using the mouse / touchpad as well as touch screen when it's set.

Looks like I'm building a new Windows binary tomorrow.

[b14ckyy]

maybe that also explains why Pinch-zooming adds a new WP every time where the first finger touches but then it zooms.

Could it be that the first finger contact is detected as a click and the information that its actually a touch input comes delayed?

[stronnag]

Another platform issue, mwp should not have to know about such things. Not reproducible on my touchscreen. So not fixable.

[ValkyrieDomi]

I tried go2rtc. Seems it has some control over resolution and framerate as well as mjpeg, h264, h265.

So I could get the video working in MWP for testing, Tho even when set 1080p it is pixelated af xD Latency very good.
Obv. with h264 or h265 it is fine.

For now, Good night.

[stronnag]

Windows Touch Screen Test

mwptools-win64-2025-08-25-touch.zip

• Just contains mwptools\bin\mwp.exe

• Requires one of 2025-08-24 or 2025-08-25 to have been installed over 25-08-16

• Uses setting touch-drag-disconnected unconditionally (i.e. regardless of Gtk test for touch gesture).

• Setting touch-drag-disconnected if now false (off) by default (does not change existing 08-24/08-25 versions)

• There will be a line in the log

  HH:MM:SS.ssssss: :TOUCH: screen is touch: true, touch-drag-disconnected: true
  

• which for the first boolean, state either true or false. I'd be interested to know what it states with a Windows touchscreen (i.e. does Gtk fail to recognise the screen as touch capable or does it just fail to recognise touch gestures).

[ValkyrieDomi]

I´ve been trying some custom code for video, and even tho I had full control over resolution, framerate and bitrate, the "best" latency was 5 seconds. Slowly getting out of ideas, therefore I did not test the newest MWP build because busy with video attempts.
Luckily we are not talking about nuclear fusion here, so it´s doable, somehow, I guess, I hope.

But my knowledge for actually solving it, in an elegant way, is too short.

[stronnag]

re: Latency

The video player subsystem (Linux, FreeBSD) has been migrated from Gtk.Video to a custom implementation. This allows control of latency of the rtsp connection as well as more relxed video sync. The default GStreamer rtsp latency is 2000ms; mwp reduces this to c. 0.

So for Linux / FreeBSD, there is essentially zero latency using either direct connection (/dev/video0) or rtsp. In addition, at least for my ancient USB camera via go2rtc, transcoding to h264 in go2rtc also seems to help latency (rather than streaming mjpeg directly).

Proprietary OS (MacOS, Windows) remain on Gtk.Video (and rtsp vice direct USB access), so there will still be a minimum 2000ms latency.

github latest: d231691

[stronnag]

The player latency is now removed in Windows / MacOS as well. Not yet pushed.

[ValkyrieDomi]

What a hero! Perfection

[b14ckyy]

OMG awesome!
In my head I was already mentally preparing to make a beggars post on FB and Github to find someone who could help with the windows side 😂

Best news to start the weekend!

[stronnag]

What would be more interesting would be to see if it is possible to stream a Windows (USB) camera. This doesn't work in a VM, so it would be useful if one of you could do some experiments for me:

• Extract the attached archive so the two files therein end up in Program Files\mwptools\bin ; then:
• cd Program Files\mwptools\bin
• Run ./wcamv.exe which should provide a ":" separated list of detected cameras.

This actually works on a VM:

# No camera connected
> ./wcamv.exe
Found (NULL)

# Single camera
> ./wcamv.exe
Found Mobius

If you have devices detected, it would be (even more) useful to see if GStreamer can stream it. For the above example:

• By name (use your own detected name !!!, probably not "Mobius")

> gst-launch-1.0 ksvideosrc device-name="Mobius" ! decodebin ! videoconvert ! autovideosink

• By id (zero based)

> gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! autovideosink

It is also possible that one of decodebin or videoconvert is not needed (note there is a single ! between each pipeline element).

Either way, I'd like to see the console output.
wcamv.exe.zip

[b14ckyy]

Nice, will give it a try tomorrow. Should I test with the last Win build you shared here? Not sure if that was the one with generally broken RTSP?

[stronnag]

The tests are independent of mwp functions, they just need the GStreamer DLLs and Exe supplied with mwp (that's why it is necessary to update mwptools/bin and cd there).
Using the last mwp will be the best as it has the latest dll/exe.
None of above depends on RTSP, this is to test direct USB camera access, primarily for the case where the user has a UVC device that can receive FPV video.

[ValkyrieDomi]

Good Morning ❤️

Rear Integrated Webcam
DroidCam Video
USB2.0 PC CAMERA

I tested will all cams and no problem to run it with Gstreamer.

[b14ckyy]

And maybe Resolution/Framerate selection 🥹

[stronnag]

FFS, I give up.

[b14ckyy]

What happened? :O
that res framerate thing is not a must. Just if it does not add much work.

[stronnag]

It's really not that simple; cameras have a large range of settings, some of which need to be set in concert, for example(s). Note this is not a proposed UI, it's just a simple way to show the options:

Default

Full set of Options

Quite complex with embedded dependencies, e.g YUV is only available at certain sizes / framerates

Selected option

This is why a different (new) UI is would be needed: this won't work for the longer options (YUV etc).

As an aside; I dispair at cheapo webcams. WTF can't it be lablled as "Rear Camera" rather than "USB2.0 camera". For maximum UX stupidity, the front camera is "USB2.0 Camera" ... one bit of difference. UX big fail.

[b14ckyy]

Yeah I see it makes things complicated. Although YUY2 or anything else than MJPEG could be filtered out. I have not seen a camera yet that would not do MJPEG and YUY2 or other uncompressed formats would usually be limited by the USB bandwidth and won't be good at framerate and resolution anyway.

[MiguelitoBeard]

Just my luck I am an Ubantu user :) did not realize was an old OS. What is a modern Linux distro I should try? Happy to move and learn so I have a modern OS. Funny video in worked before when I tried…

[b14ckyy]

You can still use Ubuntu Version 25.04 or later. I think even 24.10 is fine but not sure.
Debian needs Version 13 Trixie that was just released but works perfectly fine for me.

[MiguelitoBeard]

That is good news! Will be able to try in a couple of days.

[b14ckyy]

A bit of an issue with the release. On 2 of my machines with different cameras, I only get a freeze frame when I start the video view. Desktop witht he DELL Webcam and tablet with front and rear camera the same. @ValkyrieDomi has the same issue.

When I open the Video view in the classic popout window, MWP crashes entirely on both systems.

it worked fine with the test a few days ago but in the snapshot it does not.
A buddy of mine who runs it on Windows 10 and not 11 posted a video where it seems to work. Not sure if its W11 related.

I can provide a log in a little bit.

Edit:
mwp_stderr_2025-09-02.txt

I launched FPV mode first, then closed it, then launched the classic video to trigger the crash.

[ValkyrieDomi]

I tried on W10 and W11

On both I did reinstall of MWP.

[stronnag]

Please do the following:

• cd Program Files\mwptools\bin (or add to path)
• gst-device-monitor-1.0 > somefile.txt
• review somefile.txt (it's harmless) and let me have it
• log with --debug-flags 128 (if not included above).

[stronnag]

I have no idea what this means Gstreamer error), but it doesn't look good to me. Some sort of Windows error thing I think.

10:12:19.083898 Video error: failed to share contexts through wglShareLists 0xaa (3)

OTOH, I was just adding catching Gstreamer errors into a warning dialog.

[b14ckyy]

Log from me above has debug 128 enabled already.
Here the device monitor output:

gst-dm-out.txt

[b14ckyy]

with the test command that we did last week, it still works fine. If that matters.

[stronnag]

There are some very old gstreamer issues on the topic. mainly relating to Windows GL drivers, you could try setting each of GTK graphices modes in turn (Linux notation, I don't know how to do this in Windows):

# baseline
GSK_RENDERER=cairo mwp
# old GL
GSK_RENDERER=gl mwp
# new GL
GSK_RENDERER=ngl mwp
# Vulkan 
GSK_RENDERER=vulkan mwp

Note this is an environment variable , not a command option: no --

[stronnag]

OK. No surprises there.
Not fixable in mwp then (can't even investigate from my side, as it works just fine in the VM).
Either video is Linux/FreeBSB/MacOS only or I add the option to use Gtk.Video on Windows (and older Ubuntu). Which probably also mean no USB devices on WIndows.

[b14ckyy]

let me try a few more things when I am back at home.

One thing I don't really understand is, why does it work with gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! autovideosink but not when within MWP?

I know I am just an idiot but I would really like to understand how this all works together.
And why it works on Shawn's system with Windows 10 no problem while not working on Valkyrie's PC with windows 10.

[stronnag]

let me try a few more things when I am back at home.

One thing I don't really understand is, why does it work with gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! autovideosink but not when within MWP?

Because the end of the pipeline for gst-launch-1.0 is autovideosink which will be the bespoke windows viewer (D3D11, DirectX11) most likely. Gtk does not know about DirectX, it uses cross-platform GL either legacy OpenGL or Vulkan, hence the pipieline in the mwp log ends gtk4paintablesink. You may even be able to get gst-launch-1.0 to crash with

gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! gtk4paintablesink sync=false

Or you may not; from the GST issue, it seems Windows OpenGL drivers are sensitive things, particularly to "thread context" (whatever that might be, but not anything I can influence).

If gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! gtk4paintablesink sync=false does crash, please try

gst-launch-1.0 ksvideosrc device-index=0 ! decodebin ! videoconvert ! gtk4paintablesink 

I know I am just an idiot but I would really like to understand how this all works together. And why it works on Shawn's system with Windows 10 no problem while not working on Valkyrie's PC with windows 10.

Most likely different hardware / drivers. Without more info (like video h/w and gst-device-monitor-1.0 Video/Source > somefile.txt) just a guess. Might still be a guess even with the info.

[stronnag]

fyi: autovideosink will use d3d11videosink on Windows.

[b14ckyy]

btw I have some new info:
For Shawn, when he wants to use video, he has to have FPV mode off first. Then he opens the Video player window with his webcam and presses play to see the video. Then he switches to FPV view and he has no functioning video there.

If he opens FPV view first and then directly picks the webcam, he also gets the freeze frame.
But I can't test that since when I try to use the classiv video window, my MWP crashes.

[stronnag]

And (sadly) I need to strip CR from values read from cmdopts on Windows.

C:\Program Files\mwptools\bin\mwp.exe --voice-command voice.exe -f -n "Microsoft Zira Desktop"^M --rings=10,100 ^M --GSK_RENDERER=vulkan^M --debug-flags 5 C:\Program Files\mwptools\bin\mwp.exe --voice-command voice.exe -f -n "Microsoft Zira Desktop"^M --rings=10,100 ^M --GSK_RENDERER=vulkan^M --debug-flags 128^M --raw-log^M --radar-device COM3^M

note also that GSK_RENDERER is an environment variable no hyphens.

[b14ckyy]

No idea when I added that vulkan thing. I drag that CMDOPTS with me since the initial windows port so I totally forgot about it :D

[stronnag]

Development branch 2025-09-04 877d58a

• Adds "Fallback" video widget (Gtk.Video)
• Automatically detects missing gtk4paintablesink on Ubuntu 24.04 et al and uses the fallback player.
• On Windows (or to force the fallback widget anyway), requires that the new setting use-fallback-video is set to true.

Consequences of using the fallback video player:

• On Windows, no direct USB camera access
• Higher latency (e.g. 2000ms on RTSP feeds)
• Fewer video codecs available.

[b14ckyy]

For the Ubuntu topic.
After installing the complete gstreamer packet as stated here it worked fine on 25.04.
Will try to figure out what plugin is ACTUALLY needed so you can add it to the docs if that helps. The whole set is over 1GB and a bit excessive.

[stronnag]

So let's have a "Settings" button. Requests parameters from the camera. If the user selects an item, the settings are used verbatim in the GSteamer "playbin"

21:19:57.565879 Playbin: v4l2src device=/dev/video2 ! video/x-raw, format=YUY2, interlace-mode=progressive, width=640, height=480, framerate=30/1 ! decodebin ! autovideoconvert ! gtk4paintablesink sync=false

vice normal (no setting selected, let the device decide)

Playbin: v4l2src device=/dev/video2 ! decodebin ! autovideoconvert ! gtk4paintablesink sync=false

[stronnag]

Even maintains a reasonable frame rate (the average for RTSP is a bit off, as it includes the initial not rendered frames while it's working out latency_.

[stronnag]

Note : video stats for RTSP and other "non-direct webcams" needs a newer GH commit (2025-09-06).

[b14ckyy]

Windows mwp.exe and settings schema. Usual procedure, replace path/file in c:\Program Files\mwptools

mwp.exe-caps-list-fbc07573.zip

With that version i get this error unfortunately.
Also with the go2rtc

Installed latest snapshot with the changes you posted above.

[stronnag]

If that's the fallback player, then limited codecs is a documented restriction.

[stronnag]

OTOH, this is my "unadulterated by any dev tools" cloned VM. Just last night's image dropped over the last snapshot. Fallback viewer (see video title bar). Rendering RTSP.

Either of the go2rtc mjpeg rtsp or h264 rtsp from the Linux box work.

streams:
  usbcam1: v4l2:device?video=/dev/video0&input_format=mjpeg&video_size=1280x720&framerate=30
  usbcam2: ffmpeg:/dev/video0#video=h264#audio=aac#scale=1280:720

[stronnag]

And there's no real reason why switching should not be both ways ....

vswitch.mkv.mp4

I have a little plan for this ....

[stronnag]

And the little plan is, of course, to link log replay with video replay. At the moment the UI is sub-optimal / experimental.

• If a video is loaded when log replay is invoked, the replay UI gains a few more controls:

Extra controls, from right to left:

• The text entry defines a time offset between the video and the log (-ve, video starts first, +ve log starts first)
• The => arrow button sets the video position to the log timestamp.
• The switch drives the log from the video.

Starting position ...

• Fast-forward the log to 57 seconds
• Use the "arrow" to move the video to the log position
• Use the switch to start the video (from 57s) and drive the log replay.

The video can be switched between windowed / FPV views, and the video controls used to position / play / pause both log and video.

Work in progress, the offset is not fully wired in.

[fatbaldmen]

Brilliant folks!!! Working very well.

https://youtu.be/VbeWkjqfOBI?si=tkZAZ7GVVBbh12MQ

[ValkyrieDomi]

Oh boy! That's amazing 👌 Really awesome work! Even direct 60fps FPV feed.

VID_20250929_164053.mp4