You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
host.on('layout', (a) => { if (!a.ok) report(a.issues); }); // codes only
126
+
host.on('scale', (s) =>report(s.deviceScale)); // remote px per device px
127
+
host.on('health', (h) =>report(h.code, h.detail)); // this embed is breaking the keyboard
128
+
```
129
+
130
+
`auditLayout()` runs itself on hello and on first paint, warns to the console with
131
+
the offending codes, emits `.on('layout')`, and posts the finding down to the
132
+
viewer's structural session log — so a "the stream is blurry" report is
133
+
attributable from the pod side without asking anyone to open devtools on a phone.
134
+
`.on('scale')` carries the four numbers that separate the causes: `fbWidth/fbHeight`
135
+
(remote px sent), `cssWidth/cssHeight` (the box on the device), `scale` (fb/CSS)
136
+
and `deviceScale` (fb per DEVICE px — the one that predicts what the user sees).
137
+
138
+
Reproduce the failure on purpose with `host/test-host.html?badlayout=1`, which
139
+
leaves the iframe inside a flex + `overflow: auto` + transformed wrapper. Combine
140
+
with `&nest=1` for the full three-level chain.
141
+
142
+
### Geometry: the failure that looks like a broken keyboard
143
+
144
+
An embedder that posts geometry it cannot measure is worse than one that posts
145
+
nothing. A middle frame whose `PopcornHost` falls back to measuring *itself* is a
146
+
cross-origin iframe whose `visualViewport` never shrinks, so it reports
147
+
`occludedBottom: 0` forever — and host geometry deliberately **suppresses** the
148
+
viewer's own detectors (two detectors driving the lift with different heights is what
149
+
causes keyboard-open jitter). The result is no lift, no pan budget to reach the
150
+
field, and the local-echo pill positioned behind the keyboard, so the one mechanism
151
+
that masks per-keystroke round-trip latency becomes invisible and typing appears
152
+
dead until the remote's pixels arrive.
153
+
154
+
Three defences, so a misconfigured embedder degrades to "no help" rather than
155
+
"actively broken":
156
+
157
+
- an embedded fallback measurer that sees no occlusion stays **silent**;
158
+
- the viewer only lets a host silence its detectors once that host has reported a
159
+
real occlusion at least once;
160
+
- the legacy `{type:'parent-viewport', innerHeight, viewportHeight}` message — what
161
+
the deployed portal sends — is **translated** into `POPCORN_HOST_GEOMETRY`, so that
162
+
portal works unmodified. Opt out with `attach(frame, { legacyGeometry: false })`.
163
+
164
+
`host/test-host.html?legacybridge=1` exercises the translation; add `&legacyxlate=0`
165
+
to reproduce the original break.
166
+
167
+
### `.on('health')` — the viewer's verdict on your integration
168
+
169
+
Every failure in this chain that has cost real time degraded *silently*: the viewer
170
+
knew something was wrong and the only place it could say so was a console inside a
171
+
cross-origin iframe on somebody's phone. The integrator saw a working page, the
172
+
user saw a broken keyboard, and nobody had both halves at once.
173
+
174
+
So the viewer reports its own health up the bridge, in codes. Alert on them, or log
175
+
them beside your own session id — they are structural (short strings plus rounded
176
+
numbers, never anything derived from page content), so they are safe to forward
177
+
into your own logging.
178
+
179
+
| code | what it means |
180
+
| --- | --- |
181
+
|`host-geometry-blind`| you are feeding geometry but have never seen an occlusion, while the viewer's own detectors say the keyboard is up — you are measuring the wrong window |
182
+
|`host-geometry-stale`| your feed stopped while the keyboard was up; the viewer has fallen back to local detection |
183
+
|`host-geometry-disagrees`| both sides see a keyboard, with materially different occlusion — usually an iframe that is not full-viewport, so the lift is wrong by the difference |
184
+
|`focus-stolen`| something in your page took the focus while the keyboard was open |
185
+
|`no-virtual-keyboard`| embedded without `allow="virtual-keyboard"`, so your geometry is load-bearing |
186
+
|`remote-unconfirmed`| keystrokes were sent that the remote field never reported holding — a real lost-input signal, as opposed to a slow repaint |
187
+
188
+
Each code is reported at most once per 30s, and every message carries the
189
+
cumulative `codes` list, so a listener that mounts late still learns what went
190
+
wrong. `host/test-host.html` logs them in its debug panel.
191
+
192
+
### Sharpness on a phone: the supersampled framebuffer
193
+
194
+
Even with the layout contract satisfied, the framebuffer is sized in the phone's CSS
195
+
pixels (`deviceScaleFactor: 1`), so a 411px viewport streams 411 remote pixels onto
196
+
~1080 device pixels — `dev=0.38` in the scale line, i.e. every remote pixel is
197
+
upscaled ~2.6x by the phone. No encoder setting can put that detail back.
198
+
199
+
`?fbscale=` raises CDP `deviceScaleFactor`**and** grows the framebuffer with it, so
200
+
the page still lays out as a 411px mobile viewport (same reflow, no reload — and
201
+
`injected.js` pins `devicePixelRatio` to 1, so the site sees no change) while the
202
+
raster carries k times the detail per axis: `dev` 0.38 → 0.76 at k=2.
203
+
204
+
| value | behaviour |
205
+
| --- | --- |
206
+
|`auto`| Opt-in adaptive mode: 2x once the link is measured healthy — magnify + touch + DPR≥2 + not in desktop-fit + RTT<400ms sustained 3s + no saveData/2g/3g. **Cold start is always 1x**, and it drops back to 1x if the link degrades. |
207
+
|`1` (default) | Off — today's behaviour byte-for-byte. Use this default until device A/B data proves supersampling improves input-to-paint latency as well as sharpness. |
208
+
|`2`, `3`| pinned, ignoring link health. For a device A/B. |
209
+
210
+
**It costs k² pixels per frame** — ~4x encode CPU on the pod and ~4x bytes on the
211
+
wire at k=2. That trades against paint latency, which is why `auto` never spends it
212
+
on a link it has not measured. The CPU side is the one to watch: this image runs
213
+
TigerVNC 1.12, whose Tight/JPEG encoding is single-threaded per client (no equivalent
214
+
of KasmVNC's `-RecThreads`), so 4x the pixels is 4x the work on ONE core. KasmVNC
215
+
ships the same idea on by default — its Medium/High presets auto-scale the remote
216
+
resolution to the client and explicitly scale upward on mobile — but its encoder fans
217
+
out across cores. The mechanism is proven; the cost profile is not the same. Pages that hit desktop-fit are excluded because they
218
+
are already supersampled (980 remote px into a 411px viewport ≈ `dev=0.91`, which is
219
+
why desktop-fallback pages look sharp while responsive ones look soft).
220
+
221
+
### Diagnostics
222
+
223
+
All opt-in, all structural — no typed text, no field values, no coordinates, no
224
+
URLs. Append to the viewer URL (they survive every embedding hop; see
225
+
`PopcornHost.VIEWER_PARAMS`):
226
+
227
+
| param | what it adds |
228
+
| --- | --- |
229
+
|`diag=1`| ship the structural keyboard/input log to the proxy's `/klog`|
230
+
|`kbddebug=1`| the same, plus an on-screen overlay and console mirror |
231
+
|`fbscale=1`| disable the supersampled framebuffer (see above) — first thing to try if sharpness improved but latency got worse |
232
+
|`e2e=1`| input→paint traces: `e2e tap g#7 sent=+3ms written=+58ms paint=+412ms total=473ms`. Needs `diag=1`. Reads a 5×5 grid of 12×12 pixel patches to detect localized paints, reduced to a checksum and discarded, so it costs a GPU readback per poll — bounded to 8 traces per load, one in flight, 2.5s each. `e2e=N` for N traces. |
233
+
234
+
`scale` lines (`fb=… css=… dpr=… sc=… dev=…`) and `host layout …` lines land in
235
+
the same log, which is where a blur report should be read from first.
236
+
83
237
noVNC HTTP/WebSocket is served on `6080`. Restricted CDP is served on `9222`
84
238
and full CDP is served on `9226` for trusted internal routing. Raw VNC listens
85
239
on `127.0.0.1:5900` inside the container, and Chromium's raw DevTools endpoint
0 commit comments