Skip to content

Commit 1736c84

Browse files
committed
fix(stream): gate every vhost and refuse CRLF in broker headers
A full security review of the broker, the s6 init scripts, the sudoers rule and the Dockerfile found two exploitable faults. Both are fixed here. The third finding, an unset BROKER_SECRET accepting every request, stays as it is and is now written up in the README as a known debug-only hole rather than treated as a bug. The nginx gate covered one vhost. The awk that injects it anchored on ssl_certificate_key, which appears only in the 3001 SSL server block, but the base image ships a second, identical plain-HTTP vhost on 3000: same /websocket proxy to selkies, same /files alias of /config/Desktop. Anyone reaching port 3000 got the full interactive desktop with the ROM library mounted and no stream token asked for, and publishing 3000 is exactly what the base image's own docs tell you to do. Confirmed by reading /etc/nginx/sites-available/default out of the running container. The anchor is now every `server {` block. /50x.html is exempted because it is the error_page target, and a gated error page turns a broker outage into an internal-redirect loop. The injected port comes from BROKER_PORT instead of a hardcoded 8000. The script counts the vhosts it gated, warns when that count is under the two the base image ships, and reports an ERROR when it matched nothing rather than claiming success. stream_sid is a Secure cookie, so reaching 3000 directly over plain HTTP now admits the query token but cannot persist the cookie: 3000 works behind a TLS-terminating proxy, and unproxied plain HTTP fails closed instead of serving the desktop. Two response headers echoed filenames straight to the wire. X-State-Filename and the save-file Content-Disposition both carry names taken off the filesystem, where CR and LF are legal bytes, and http.server writes header values through without validating them. A state file named with a CRLF could split the response and inject headers of the attacker's choosing into a browser that fetched it. Both emit sites now run through a shared _header_token, and PUT /state-file refuses a non-printable-ASCII filename outright rather than storing a name it would have to mangle on the way back out. Hardening applied alongside. The shared secret is compared as bytes, because hmac.compare_digest refuses a str carrying non-ASCII characters, so a UTF-8 BROKER_SECRET would have raised inside every check and answered 500 to every request. The gated vhosts also send Referrer-Policy: same-origin, so the stream token in the iframe URL does not leak through a Referer. Twelve tests cover the new behavior: the header filter, the PUT refusal, the GET sanitizing a CRLF name already on disk, and the secret comparison across the UTF-8, ASCII, missing-header and unset-secret cases. Deploying this needs a recreate, not a restart. The idempotency check greps the site config for the marker it already wrote, and /etc/nginx lives in the container's writable layer, so a restarted container keeps its single-vhost gate.
1 parent 420a2f0 commit 1736c84

4 files changed

Lines changed: 234 additions & 28 deletions

File tree

README.md

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ Save states don't care about any of this. The requirement is only for memory-car
108108

109109
| Variable | Default | What it does |
110110
|---|---|---|
111-
| `BROKER_SECRET` | *(none)* | Shared secret, sent as `X-Broker-Secret`. Unset means every request is accepted, which is not safe on a shared network. |
111+
| `BROKER_SECRET` | *(none)* | Shared secret, sent as `X-Broker-Secret`. Unset means every request is accepted. Debug-only, see [Security](#security). |
112112
| `BROKER_PORT` | `8000` | Port the broker listens on. |
113113
| `ROM_ROOT` | `/romm/library` | Where ROMs are mounted. A `rom_path` outside this is rejected. |
114114
| `SAVE_SLOT` | `10` | Slot `/save-and-exit` uses when none is given. 10 as autosave leaves 1-9 for the player. |
@@ -208,6 +208,18 @@ State files are named `{SERIAL} ({CRC}).{slot:02d}.p2s`, which is PCSX2's own sc
208208

209209
It is refused while a game is running or a launch is in flight, because PCSX2 holds the card open and swapping it mid-game corrupts it. Hydrate before launch or after exit. `POST /launch` is refused for the duration too, so a launch can't slip in mid-replace. Dashboard crash recovery is *not* blocked, since the gameless dashboard never opens a card and the swap is atomic either way.
210210

211+
## Security
212+
213+
Two credentials, guarding two different things.
214+
215+
**`BROKER_SECRET` guards the API on port 8000.** Sent as `X-Broker-Secret`, compared in constant time. `GET /health` and `GET /verify` are deliberately exempt: `/health` so it works as a container healthcheck, `/verify` because nginx's `auth_request` cannot forward the secret and the stream token is itself the credential there.
216+
217+
**The stream token guards the desktop on 3000/3001.** It is minted per session by `POST /launch`, 256 bits from `secrets.token_urlsafe`, and enforced by an nginx `auth_request` that the mod injects into *every* `server` block in the site config. That matters: the base image ships two identical vhosts, plain HTTP on 3000 and TLS on 3001, both proxying the same selkies stream and both serving `/config/Desktop` at `/files`. Gating only the TLS one left a complete bypass a port number away. The `stream_sid` cookie is `Secure`, so 3000 is usable only behind a TLS-terminating proxy — direct plain-HTTP browsing to it now fails closed.
218+
219+
**Leaving `BROKER_SECRET` unset is a known hole, not a supported mode.** The broker runs as root inside the container, so the open API means root-privileged reads and writes under `/config`, plus arbitrary launches within `ROM_ROOT`. Worse, the two credentials stop being independent: `POST /launch` *returns* a stream token, so an unauthenticated broker hands out the credential that opens the desktop. Use it for local debugging on a trusted host and nothing else. The broker logs a warning at startup when it is unset.
220+
221+
Port 8000 is plain HTTP, so the secret crosses the network in the clear. Keep the broker on an internal network, or put TLS in front of it.
222+
211223
## Troubleshooting
212224

213225
**Reading the logs.** Every request the broker refuses or fails is logged to stdout, not only returned as JSON, so `docker logs pcsx2` is enough to see what went wrong without turning on `DEBUG`. The shape is `HTTP <code> <method> <path> from <caller>: <reason>`. A `WARNING` is a caller-side rejection (bad slot, no game running, a claim already held); an `ERROR` is a broker-side fault (pactl failed, a state file that could not be written) and always deserves attention. An unhandled crash inside a handler logs a full traceback and still answers `500 internal broker error` rather than dropping the connection. Stream tokens are redacted from logged URLs, so log output is safe to paste into a bug report.
@@ -238,7 +250,9 @@ The usual cause is the hardware encoder: Selkies routes `x264enc` through VAAPI
238250
239251
Confirm it took with `docker logs pcsx2 | grep -E "VAAPI|CPU Software"`. If it still cycles, drop to `SELKIES_ENCODER=jpeg` to skip H.264 entirely, at a real cost in bandwidth and sharpness.
240252

241-
**"An error occurred, restarting stream", over and over.** The stream client says this whenever its WebSocket drops, so the useful signal is on the broker side: `docker logs pcsx2 | grep "/verify"`. A `403` there names the reason (`no stream token in the request`, `stream token expired`, `stream token superseded by a newer launch`), and the token is redacted so the line is safe to share. No `/verify` lines *at all* means the gate is not running: either the mod's init never patched nginx (`docker logs pcsx2 | grep broker-mod`), or your reverse proxy is reaching the ungated plain-HTTP vhost on port 3000 instead of the SSL vhost on 3001. Point it at 3001 and do not publish 3000: the gate is the only thing standing between the ROM library and anyone who finds the address.
253+
**"An error occurred, restarting stream", over and over.** The stream client says this whenever its WebSocket drops, so the useful signal is on the broker side: `docker logs pcsx2 | grep "/verify"`. A `403` there names the reason (`no stream token in the request`, `stream token expired`, `stream token superseded by a newer launch`), and the token is redacted so the line is safe to share. No `/verify` lines *at all* means the gate is not running: the mod's init never patched nginx. Check `docker logs pcsx2 | grep broker-mod` for the "Applied nginx stream gate to N vhost(s)" line, which should say 2.
254+
255+
**A stream that 403s on every asset over plain HTTP.** The `stream_sid` cookie is `Secure`, so a browser will not store it on an unencrypted origin. The query token admits the first request and everything after it is refused. Reach the container over TLS: either port 3001 directly, or port 3000 behind a reverse proxy that terminates TLS. Plain-HTTP 3000 straight to a browser is not a supported shape and now fails closed rather than serving an ungated desktop.
242256

243257
## Development
244258

@@ -250,7 +264,7 @@ python3 -m unittest discover -s tests -v
250264

251265
CI runs the same tests under pytest, plus `ruff check .` against the shared [ruff.toml](ruff.toml). `tests/` is excluded from the image by `.dockerignore` and never ships.
252266

253-
The suite covers what can be tested without a running emulator: save-file and memory-card archive handling (round trips, the last-write-wins mtime guard, path-traversal and subtree rejection, File-card refusal), `PCSX2.ini` patching, ROM path resolution, and the session state machine (crash-loop limiter, launch and card-op claims, deferred `load_slot` generation checks). Anything needing a real X display, PINE socket, or pcsx2-qt process is out of scope on purpose, because it is only provable on a live container.
267+
The suite covers what can be tested without a running emulator: save-file and memory-card archive handling (round trips, the last-write-wins mtime guard, path-traversal and subtree rejection, File-card refusal), `PCSX2.ini` patching, ROM path resolution, the session state machine (crash-loop limiter, launch and card-op claims, deferred `load_slot` generation checks), the stream-token gate decision, and the header/secret handling that keeps a filename off the response line and a UTF-8 secret out of a 500. Anything needing a real X display, PINE socket, or pcsx2-qt process is out of scope on purpose, because it is only provable on a live container.
254268

255269
**Init ordering matters more than it looks.** The mod ships two s6 oneshots. `init-pcsx2-config` rewrites files that base-image services read exactly once at startup (selkies' `input_handler.py`, the nginx site config, labwc's autostart), so `init-services` is made to depend on it and the whole service stack waits. Drop that edge and the patches still land on disk while changing nothing about the processes already running, which fails silently: the log says "Applied nginx stream gate" and the live nginx has no gate. Keep that script fast and offline. `init-pcsx2-deps` holds the slow networked work (apt, `patches.zip`) and only `svc-broker` waits for it, so a GitHub timeout cannot stall the stream.
256270

root/etc/s6-overlay/s6-rc.d/init-pcsx2-config/init.sh

Lines changed: 44 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -110,47 +110,72 @@ else
110110
echo "[broker-mod] ERROR: selkies input_handler.py not found under /lsiopy, /usr/lib, /usr/local/lib, /opt or /config/.local, so neither the gamepad EOF fix nor the log-level patch was applied"
111111
fi
112112

113-
# Gate the browser-facing stream with nginx auth_request. The 3001 SSL vhost is
114-
# the host RomM loads in the iframe; without this, anyone who learns the address
115-
# gets an interactive desktop with the ROM library mounted, since RomM's auth
116-
# never sits on this socket. auth_request sends every 3001 request to the
117-
# broker's /verify, which checks the session-bound stream token RomM appends to
118-
# the iframe URL and, on the first (query-token) hit, hands back a stream_sid
119-
# cookie that carries every later asset and the WebSocket upgrade. Anchored on
120-
# ssl_certificate_key, which appears only in the 3001 server block, so the plain
121-
# 3000 vhost is untouched. The broker exempts /verify from its shared secret
122-
# because nginx cannot forward that secret and the stream token is the credential.
113+
# Gate the browser-facing stream with nginx auth_request. Without this, anyone
114+
# who learns the address gets an interactive desktop with the ROM library
115+
# mounted, since RomM's auth never sits on this socket. auth_request sends every
116+
# request to the broker's /verify, which checks the session-bound stream token
117+
# RomM appends to the iframe URL and, on the first (query-token) hit, hands back
118+
# a stream_sid cookie that carries every later asset and the WebSocket upgrade.
119+
# The broker exempts /verify from its shared secret because nginx cannot forward
120+
# that secret and the stream token is the credential.
121+
#
122+
# EVERY server block is gated, not just the 3001 SSL vhost RomM points at. The
123+
# base image ships a second, identical plain-HTTP vhost on 3000 (same /websocket
124+
# proxy to selkies, same /files alias of /config/Desktop), so anchoring this on
125+
# ssl_certificate_key left a complete bypass of the gate one port number away,
126+
# and publishing 3000 is exactly what the base image's own docs tell you to do.
127+
# Note that stream_sid is a Secure cookie: reaching 3000 directly over plain
128+
# HTTP admits the query token but the cookie will not stick, so 3000 only works
129+
# behind a proxy that terminates TLS. That is the supported shape; unproxied
130+
# plain-HTTP 3000 is not, and now fails closed instead of serving the desktop.
131+
#
132+
# /50x.html is exempted because it is the error_page target: left gated, a
133+
# broker outage makes each request bounce between the 500 and its own gated
134+
# error page until nginx's internal-redirect limit trips.
123135
NGINX_SITE="/etc/nginx/sites-available/default"
124136
if [ -f "$NGINX_SITE" ]; then
125137
if grep -q "RomM stream gate" "$NGINX_SITE"; then
126138
echo "[broker-mod] nginx stream gate already applied."
127139
else
128-
awk '
129-
{ print }
130-
/ssl_certificate_key/ && !injected {
140+
awk -v bport="${BROKER_PORT:-8000}" '
141+
/^[[:space:]]*server[[:space:]]*\{/ {
142+
print
131143
print " # ── RomM stream gate (pcsx2-broker-mod) ──"
132144
print " auth_request /_stream_auth;"
133145
print " auth_request_set $stream_set_cookie $upstream_http_set_cookie;"
134146
print " add_header Set-Cookie $stream_set_cookie;"
147+
print " add_header Referrer-Policy \"same-origin\" always;"
135148
print " location = /_stream_auth {"
136149
print " internal;"
137150
print " auth_request off;"
138-
print " proxy_pass http://127.0.0.1:8000/verify;"
151+
print " proxy_pass http://127.0.0.1:" bport "/verify;"
139152
print " proxy_pass_request_body off;"
140153
print " proxy_set_header Content-Length \"\";"
141154
print " proxy_set_header X-Original-URI $request_uri;"
142155
print " }"
143-
injected = 1
156+
next
157+
}
158+
/^[[:space:]]*location[[:space:]]*=[[:space:]]*\/50x\.html[[:space:]]*\{/ {
159+
print
160+
print " auth_request off;"
161+
next
144162
}
145-
' "$NGINX_SITE" > "$NGINX_SITE.tmp" && mv "$NGINX_SITE.tmp" "$NGINX_SITE"
146-
if grep -q "RomM stream gate" "$NGINX_SITE"; then
147-
echo "[broker-mod] Applied nginx stream gate to 3001 vhost."
163+
{ print }
164+
' "$NGINX_SITE" > "$NGINX_SITE.tmp"
165+
gated=$(grep -c "RomM stream gate" "$NGINX_SITE.tmp" 2>/dev/null)
166+
[ -n "$gated" ] || gated=0
167+
if [ "$gated" -gt 0 ]; then
168+
mv "$NGINX_SITE.tmp" "$NGINX_SITE"
169+
echo "[broker-mod] Applied nginx stream gate to $gated vhost(s)."
170+
if [ "$gated" -lt 2 ]; then
171+
echo "[broker-mod] WARNING: only $gated vhost gated; the base image ships two (3000 and 3001). Any other listener is serving the desktop ungated."
172+
fi
148173
if ! nginx -t 2>/dev/null; then
149174
echo "[broker-mod] WARNING: nginx -t failed now (ssl cert may not exist yet at init); nginx revalidates at start."
150175
fi
151176
else
152177
rm -f "$NGINX_SITE.tmp"
153-
echo "[broker-mod] ERROR: nginx stream gate not applied (ssl_certificate_key anchor missing; base image may have changed)."
178+
echo "[broker-mod] ERROR: nginx stream gate not applied (no 'server {' block matched; base image may have changed)."
154179
fi
155180
fi
156181
else

root/root/broker.py

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@
2929

3030
PORT = int(os.environ.get("BROKER_PORT", "8000"))
3131
SECRET = os.environ.get("BROKER_SECRET", "")
32+
# The secret is compared as bytes, never as str: hmac.compare_digest refuses a
33+
# str with non-ASCII characters, so a UTF-8 BROKER_SECRET would raise inside
34+
# every _check_secret and answer 500 to every request. Encoding once here keeps
35+
# the comparison total for any secret an operator can set.
36+
_SECRET_BYTES = SECRET.encode("utf-8")
3237
ROM_ROOT = Path(os.environ.get("ROM_ROOT", "/romm/library")).resolve()
3338

3439
# JSON request bodies are tiny (a rom_path, a slot number); anything larger
@@ -1812,6 +1817,18 @@ def _redact_uri(uri: str) -> str:
18121817
return re.sub(r"(stream_token=)[^&]*", r"\1REDACTED", uri)
18131818

18141819

1820+
def _header_token(value: str, fallback: str) -> str:
1821+
"""Reduce `value` to something safe to send as an HTTP header value.
1822+
1823+
Header values are latin-1 and http.server writes them through without
1824+
validating, so a CR or LF reaching send_header splits the response. Both
1825+
callers pass names taken off the filesystem, where those bytes are legal
1826+
in a filename, so the filter belongs here rather than at each call site.
1827+
"""
1828+
safe = "".join(c for c in value if c.isascii() and c.isprintable()).strip()
1829+
return safe or fallback
1830+
1831+
18151832
class BrokerHandler(BaseHTTPRequestHandler):
18161833

18171834
def log_message(self, fmt, *args):
@@ -1820,9 +1837,12 @@ def log_message(self, fmt, *args):
18201837
def _check_secret(self) -> bool:
18211838
if not SECRET:
18221839
return True
1840+
# latin-1 is how http.server decoded the header, so encoding it back
1841+
# recovers exactly the bytes that arrived on the wire, which is what
1842+
# _SECRET_BYTES is comparable against.
18231843
return hmac.compare_digest(
1824-
self.headers.get("X-Broker-Secret", ""),
1825-
SECRET,
1844+
self.headers.get("X-Broker-Secret", "").encode("latin-1", "replace"),
1845+
_SECRET_BYTES,
18261846
)
18271847

18281848
def _verify_stream(self) -> None:
@@ -1967,7 +1987,9 @@ def _get_state_file(self):
19671987
self.send_response(200)
19681988
self.send_header("Content-Type", "application/octet-stream")
19691989
self.send_header("Content-Length", str(len(content)))
1970-
self.send_header("X-State-Filename", state_path.name)
1990+
self.send_header(
1991+
"X-State-Filename", _header_token(state_path.name, "state.p2s")
1992+
)
19711993
self.end_headers()
19721994
self.wfile.write(content)
19731995
log.info("state-file: served %s (%d bytes)", state_path.name, len(content))
@@ -1996,9 +2018,7 @@ def _get_save_file(self):
19962018
)
19972019
return
19982020
# Header values must be latin-1; ROM stems can be anything.
1999-
safe_name = "".join(
2000-
c for c in (rom_name or "pcsx2") if c.isascii() and c.isprintable()
2001-
).strip() or "pcsx2"
2021+
safe_name = _header_token(rom_name or "pcsx2", "pcsx2")
20022022
self.send_response(200)
20032023
self.send_header("Content-Type", "application/zip")
20042024
self.send_header("Content-Length", str(len(archive)))
@@ -2155,6 +2175,13 @@ def _handle_PUT(self):
21552175
if not filename or filename.startswith(".") or not filename.endswith(".p2s"):
21562176
self._send_json(400, {"error": "filename must be a .p2s basename"})
21572177
return
2178+
# CR and LF are legal in a Linux filename, so without this a stored name
2179+
# carrying them would split the response when GET /state-file echoes it
2180+
# back as X-State-Filename. _header_token sanitises that emit too; this
2181+
# refuses the name outright rather than silently storing a mangled one.
2182+
if filename != _header_token(filename, ""):
2183+
self._send_json(400, {"error": "filename must be printable ASCII"})
2184+
return
21582185

21592186
try:
21602187
length = int(self.headers.get("Content-Length", 0))

0 commit comments

Comments
 (0)