Validate SteamOS / Steam Deck support

[RaymondYouTube]

Does this work on Steam Deck, or anything with SteamOS? Since I'm having trouble logging in to Sunshine on the browser, I figured if Apollo would be any better. SteamOS is based on Arch Linux.

[MrOz59]

Hi! There's been real progress on this recently.

Hermes now detects when it's running under SteamOS Game Mode (a standalone Gamescope session) and adapts: instead of trying to create a virtual display (which doesn't apply when Gamescope is already the compositor), it captures the Gamescope output directly. On the desktop side (KDE/GNOME), the normal virtual-display path is used. There's also a Steam Deck native EDID mode (1280×800) and a configurable Gamescope backend.

Since SteamOS is Arch-based, the Arch package path applies. A couple of honest caveats: this is freshly implemented and not yet validated on real Deck/SteamOS hardware (I don't have a Deck to test on right now), so if you try it, feedback would be hugely valuable. The Hermes-KMS virtual-display driver also needs the kernel module installed.

If you give it a shot, let me know how it goes — especially what the diagnostics report for your session environment.

[MrOz59]

Status update: Hermes now has explicit handling for standalone Gamescope sessions such as SteamOS Game Mode. It detects the session, captures the existing Gamescope output directly instead of trying to create another virtual display, includes a native 1280×800 Steam Deck mode, and has automated tests for the environment detection.

However, this path has not yet been validated on real Steam Deck or SteamOS hardware. I am keeping this issue open as a public validation tracker. Reports from anyone able to test the latest nightly on actual hardware would be very helpful, especially the session diagnostics and logs around stream startup.

[blondejefe]

Hermes worked great on CachyOS in my Game Mode (gamescope-session) setup until roughly two weeks ago. Since then, I haven't been able to get it working automatically in Game Mode despite a frankly embarrassing amount of tinkering with systemd overrides, startup timing, environment imports, etc. KDE Plasma continues to work, my problem is specific to booting directly into Game Mode where my system lives 99% of the time. I did manage to get a clean before/after reproduction from the same Gamescope session below.

Fresh boot directly into gamescope-session:

hermes.service is inactive (dead)
None of the expected ports (47984, 47989, 47990, 48010) are listening
Artemis cannot discover the host

Without leaving Game Mode, I then run only:

systemctl --user restart hermes

Hermes immediately becomes active (running), all four ports begin listening, Avahi registers the host, and Artemis connects successfully. Nothing else about the Gamescope session changes. One potentially relevant detail is that even during the successful manual start, the service reports that DISPLAY, WAYLAND_DISPLAY, XAUTHORITY, PULSE_SERVER, and PIPEWIRE_RUNTIME_DIR are not set. Hermes itself reports the following:

Environment variable WAYLAND_DISPLAY has not been defined Session: an unrecognised compositor (no window system)

It also reports that no window system is attached to the process. Despite that, Hermes-KMS initializes, KMS capture finds the display/connector, VAAPI H.264/HEVC/AV1 initialize successfully, the service opens its ports, and Artemis works normally. This makes me suspect an autostart/session lifecycle issue with gamescope-session, rather than Hermes-KMS or encoding itself. Hermes works once manually started after Game Mode is already running; it simply isn't coming up automatically with the session anymore. Sadly, everything worked wonderfully just a few weeks ago. I've officially driven myself up the wall troubleshooting it and thrown in the towel for now. I really hope SteamOS/Gamescope support can be sorted out because I miss Hermes tremendously.

My HTPC is deliberately set up like a console: power on, boot directly into Game Mode, grab a controller, and leave the keyboard stored away. Having to open a terminal and manually restart Hermes after every boot unfortunately defeats the purpose of the setup. I'm happy to test fixes or provide whatever would be useful! I have the before/after Hermes service status, before/after listening-port captures, Hermes journal, and the full systemd user-session boot journal from the successful reproduction.

[MrOz59]

Thanks for the detailed report! The clean before/after reproduction is especially useful, and yes, I'd definitely like to take a look at those logs.

I don't currently have a SteamOS machine available to reproduce your exact setup, but Gamescope itself is something I can install and test here, so I should at least be able to reproduce part of the environment. The SteamOS/Game Mode session lifecycle may still behave differently, though, so I'll probably need your help with some of the troubleshooting.

If you're willing to test a few things and send me logs/results along the way, I'm more than happy to work through this with you and try to get automatic startup working properly again.

From what you've described, I also suspect this is much more likely related to the systemd user-session / gamescope-session lifecycle or startup ordering than Hermes-KMS or the encoder stack itself. The fact that a simple:

systemctl --user restart hermes

makes everything immediately work, without changing anything else in the Gamescope session, is a very useful clue.

Please send me the Hermes journal, the full user-session boot journal, and the before/after service status and port captures you mentioned. That should give me a pretty good starting point.

I'm currently working on a fairly substantial update to both Hermes and the Hermes-KMS driver, so I may be a little slower than usual getting through this one, but this issue is definitely on my list.

One of the things taking most of my development time lately is the client-isolation work. Long term, I want Hermes to be usable almost like a small self-hosted GeForce NOW-style server, where multiple clients can have properly isolated sessions/displays instead of everything being tied to a single desktop session. That feature has turned into a lot of back-and-forth between implementing things, finding edge cases, fixing bugs, and testing again.

Game Mode / Gamescope support is still important to me though, especially for exactly the kind of console-like HTPC setup you're describing. Having to pull out a keyboard and manually restart Hermes every time obviously defeats the whole point.

If you'd like, you can also add me on Discord. For an issue like this it might make things much easier, since we can exchange logs, try experimental changes, and iterate on tests much faster there.

[blondejefe]

Absolutely, thank you for the kind reply.

I've got the clean reproduction logs saved now. I would like to note that the Hermes journal from before the manual restart is empty because hermes.service is simply inactive (dead) on the initial Game Mode boot, so there are no Hermes entries for that phase. I'm also absolutely willing to test experimental changes/builds. This HTPC boots directly into CachyOS gamescope-session, so I'm happy to be your guinea pig for the SteamOS/Game Mode side of this.

Discord works for me too if that's easier for rapid testing. I tried looking you up by "MrOz59" to no avail, but I am available at "blondejefe" if you can reach out. I look forward to squashing this together for the good of all. I would also like to mention that your roadmap sounds absolutely amazing! Wow, good stuff, man. I had a feeling Hermes-KMS wasn't just built as a clever way to shave capture latency.

For now, below are the logs I currently have.

hermes-status-before.log — https://paste.cachyos.org/p/7ba8498.txt
hermes-ports-before.log — https://paste.cachyos.org/p/db4db52.txt
hermes-after-restart.log — https://paste.cachyos.org/p/c3f1ae4.txt
hermes-status-after.log — https://paste.cachyos.org/p/939f32f.txt
hermes-ports-after.log — https://paste.cachyos.org/p/f0ce2a8.txt
gamescope-hermes-boot.log — https://paste.cachyos.org/p/4ce9dc1.txt