Fix mouse motion in VRR

[Szwagi]

I am unfamiliar with the codebase, so this PR most definitely has plenty of issues I haven't found.
If the actual solution is more complicated than this, I'm unlikely to be of any help, but would love to see this fixed and working as well as this is for me.

Fixes #12032 and this sounds related too #12013.

I will take CS2 on a 360Hz display as an example. When I say FPS measurements I mean what the monitor reports.

Before (the issues):

• no_break_fs_vrr 0: The sensitivity is consistent. Moving the mouse even while the cursor is locked spikes the FPS to 357 (sends duplicate frames, making VRR useless).
• no_break_fs_vrr 1: The sensitivity of the mouse changes depending on the FPS of the game.
• The game feels very unsmooth, even when it runs above 300FPS, likely to do with weird delayed mouse motion.

After:

• no_break_fs_vrr 0: The sensitivity is consistent. Moving the mouse spikes the FPS only when the cursor is unlocked. During gameplay (locked cursor), the FPS stays at the FPS cap I set.
• no_break_fs_vrr 1: The sensitivity is consistent.
• The game feels very smooth, basically the same as other compositors.

My idea of what is happening is that

1. Hyprland is resetting the stored mouse movement before it gets sent to the game.
2. I do not understand why Hyprland even has to accumulate the movement instead of sending it straight to the game. I guess it could help performance on 8000Hz mice, but in practice it seems like it just adds a bunch of input lag and possibly holds on to it so long it misses frames (hence the unsmoothness).

I have tested in CS2 with native Wayland and XWayland. I have also tested in a Wine game running through Gamescope.

[github-actions]

Hello and thank you for making a PR to Hyprland!

Please check the PR Guidelines and make sure your PR follows them.
It will make the entire review process faster. :)

If your code can be tested, please always add tests. See more here.

beep boop, I'm just a bot. A real human will review your PR soon.

[UjinT34]

If this works for all locked/unlocked hw/sw wayland/xwayland combinations with different games then I'm happy that all the pointer skipping and movement accumulation gets nuked. The accumulation was needed because sending the movement with locked cursor to an xwayland client straight away caused and immediate frame event in return which was treated as a valid new frame from the game and broke VRR.
Looks like this also nukes m_pointerEvents.frame.emit() and its handling. Maybe the original issue was cause by using this event to call g_pSeatManager->sendPointerFrame(); instead of calling it right after PROTO::relativePointer->sendRelativeMotion. No idea why the event was used, it was this way before the hack.

[Szwagi]

I just checked and this definitely does not work correctly with software cursors.
In CS2, the sensitivity is correct, but VRR is broken in a weird way...
It sends new frames on cursor move when the cursor is locked and does not send frames when the cursor is unlocked.
Could this be as simple as an inverted if statement? I looked for it and found a bunch of weird cases like >0 on a bool, but nothing that fixed it.

[UjinT34]

WAYLAND_DEBUG=1 and some HL logging might help. The hack exists because for whatever reason there is an extra unwanted wl surface frame event right after a pointer frame with locked cursor. Nothing says that it should be any different from unlocked cursor or that there should/shouldn't be such surface frame event. The code just accumulates mouse movement to send the result and a pointer frame right after the wanted surface frame and discard the next one surface frame. Without this hack there is no way to distinguish between wanted and unwanted surface frames. The proper fix should somehow avoid getting those extra surface frames but it's probably outside of HL's control.

[Szwagi]

Two reasons for mouse motion scheduling frames when it shouldn't:

1. Scheduling new frames even though the damage box was empty. This scheduled frames for all mouse motion.
2. Using monitor->shouldSkipScheduleFrameOnMouseEvent() with g_pHyprRenderer->damageBox(...)... When the cursor is on the edge of the screen, box.overlaps(...) passes, secondary monitor does not want to skip the schedule, then g_pHyprRenderer->damageBox will schedule a frame for the primary monitor.

no_break_fs_vrr=0 with software cursors is still broken, and I'm unsure if there's any hope for it.
In CS2, the cursor is not visible, but the cursor box is still 30x30, I'll investigate if I can check if any cursor is visible at all.

Admittedly that code is in need of some cleanup now, and I'm unsure if g_pCompositor->m_unsafeState is needed there (I think the 'if' checks shouldn't pass?).

Also, maybe the better way is to do this in g_pHyprRenderer->damageBox?

- if (!skipFrameSchedule)
+ if (!skipFrameSchedule && (!isMouseEvent || !monitor->shouldSkipScheduleFrameOnMouseEvent()))

[Szwagi]

After more testing, the current status is (for things I tested):

• native, no_hardware_cursors = 0, no_break_fs_vrr = 0 - Works
• native, no_hardware_cursors = 0, no_break_fs_vrr = 1 - Unlocked cursors schedule extra frames
• native, no_hardware_cursors = 1, no_break_fs_vrr = 0 - Works
• native, no_hardware_cursors = 1, no_break_fs_vrr = 1 - Works
• xwayland, no_hardware_cursors = 0, no_break_fs_vrr = 0 - Works
• xwayland, no_hardware_cursors = 0, no_break_fs_vrr = 1 - Unlocked cursors schedule extra frames
• xwayland, no_hardware_cursors = 1, no_break_fs_vrr = 0 - Works when it works, for 50% of game launches the locked cursors get warped back (?)
• xwayland, no_hardware_cursors = 1, no_break_fs_vrr = 1 - Works when it works, for 50% of game launches the locked cursors get warped back (?)

[Nosamdaman]

First of all, thank you @Szwagi for digging deep into this, this is quite literally the only major show-stopper preventing me from gaming on Hyprland. Secondly, I'm not quite able to replicate your above behavior matrix. For me, no matter what, on the latest version of your branch, I'm seeing stuttery camera movement on CS2 with a frame limiter set, though perhaps there's something I'm missing. That being said, It looks like Half-Life 2 works for me with no_break_fs_vrr enabled and an FPS limit below my refresh rate regardless of whether or not hardware cursors are enabled.

This seems like a real nightmare of an issue for you guys lol, if you want me to validate any further findings on my end let me know.

[njdom24]

Dunno if there's any relation, but last I checked, software cursors disable direct scanout regardless of if they're visible or not. Could direct scanout be related to the difference here?

[Nosamdaman]

Dunno if there's any relation, but last I checked, software cursors disable direct scanout regardless of if they're visible or not. Could direct scanout be related to the difference here?

So I just re-tested this. With both direct scanout and hardware cursors enabled, playing Wayland native CS2 with an FPS limit of 140 on 144 VRR display, I'm still getting the occasional spike to 144 when moving the mouse, causing visible hitching.

[Szwagi]

So I just re-tested this. With both direct scanout and hardware cursors enabled, playing Wayland native CS2 with an FPS limit of 140 on 144 VRR display, I'm still getting the occasional spike to 144 when moving the mouse, causing visible hitching.

Can't reproduce on NVIDIA, could you share your config?
For me the only time hardware cursors do this is when the cursor is unlocked (which is fine with no_break_fs_vrr = 1 but broken for no_break_fs_vrr = 0).

Worth noting that CS2's fps_max is unreliable, engine_low_latency_sleep_after_client_tick was meant to fix it but I don't think it works. Try mangohud's fps limiter.

Edit: I'm able to reproduce it in MPV when playing back 24FPS footage... it spikes to 28FPS. I'll look into it
Never mind, it was min_refresh_rate... The default is 24FPS and the way it works isn't great so it spikes above that.

[Nosamdaman]

Worth noting that CS2's fps_max is unreliable, engine_low_latency_sleep_after_client_tick was meant to fix it but I don't think it works. Try mangohud's fps limiter.

Ah, I was unaware of that, switching to ManguHud's limiter did fix the issue on wayland. I think I'm still seeing the issue with CS2 in xwayland mode, though it is far less frequent. That being said, it feels amazing in Wayland.

[Szwagi]

I am unsure if e4013b8 is entirely correct, but it should function the same or better than the frame schedule removed in ee471d1, and without introducing unnecessary frame schedules on mouse motion. From my testing it's only necessary for Firefox which seems to be broken (based on the amount of Firefox hacks)? It also fixes an issue with frames not being scheduled when the surface x/y is not on any monitor.

I didn't look hard for the remaining hardware cursor issue yet, but I suspect it might be a bug in Aquamarine.

Status to my eye:

• native, no_hardware_cursors = 0, no_break_fs_vrr = 0 - Works
• native, no_hardware_cursors = 0, no_break_fs_vrr = 1 - Unlocked cursors schedule extra frames
• native, no_hardware_cursors = 1, no_break_fs_vrr = 0 - Works
• native, no_hardware_cursors = 1, no_break_fs_vrr = 1 - Works
• xwayland, no_hardware_cursors = 0, no_break_fs_vrr = 0 - Works
• xwayland, no_hardware_cursors = 0, no_break_fs_vrr = 1 - Unlocked cursors schedule extra frames
• xwayland, no_hardware_cursors = 1, no_break_fs_vrr = 0 - Works
• xwayland, no_hardware_cursors = 1, no_break_fs_vrr = 1 - Works

[Nosamdaman]

I just pulled and did a test of my own with the 8 combinations when no_break_vs_vrr = 1 and they all seem to be working perfectly to my eyes, as in the games I've tested, the cursor is only unlocked in menus, thus making the extra frame-scheduling not noticeable.

This even fixed the mild VRR flicker I was getting. Good job again on this one, you have no idea how much sanity I was losing testing things before I saw this PR.

[Szwagi]

e60c987 fixes framerate spiking when changing window focus caused by animations scheduling many thousands of frames per second for animations that aren't visible.

I did more research and I don't believe Hyprland is responsible for breaking VRR with hardware cursors and no_break_fs_vrr=1. With an unlocked cursor, Hyprland schedules exactly 125FPS on a 125FPS lock, but my monitor reports 250FPS or an unstable 330FPS coming in depending on the game.

It's possible there's still an extra frame scheduled on cursor visibility change. My reports spikes by 1-2FPS, but I couldn't see any extra frame schedules except for the unrelated 37ae610, perhaps monitor getting confused by a lag spike in the game when opening the menu.

@vaxerski I think this is all for the issues I found. I will now daily this branch to see if something comes up, but would like to know what you think of this and what the glaring issues are :D

[vaxerski]

in general looks ok just needs some testing yea

[vaxerski]

why is this

[Szwagi]

Some programs like Firefox get empty damageBox here even though they redrew the entire window, ignoring makes it freeze and update only on mouse motion. Old code used getMonitorFromVector(x, y) and scheduled a frame on that monitor, but that breaks when the x, y is not on any monitor, making it freeze