Feature/core device orientation#1727
Merged
Merged
Conversation
Decoded from a Xcode-mirror sniff of four back-to-back rotate-left clicks.
Each request is an OrientationRequest envelope on the new
'com.apple.coredevice.devicecontrol' RemoteXPC service:
{featureIdentifier: 'com.apple.coredevice.feature.remote.devicecontrol.orientation',
messageType: 'OrientationRequest',
payload: {rotate: {_0: 'left'}}}
The reply carries currentDeviceOrientation + currentDeviceNonFlatOrientation
+ currentDeviceOrientationLocked. Four 'left' requests cycle the device
through portrait -> landscapeLeft -> portraitUpsideDown -> landscapeRight.
Exposed as 'pymobiledevice3 developer core-device rotate [left|right]'.
Adds Rotate ↺ / Rotate ↻ buttons in the viewer's util-tray and a POST /rotate endpoint on the server that forwards to OrientationService.rotate(direction). The rotation is applied INSIDE the canvas (ctx.translate + ctx.rotate + drawImage on each frame), not via CSS transform on the canvas element, so the canvas's own CSS box matches the rotated footprint and the surrounding flex layout + cosmetic bezel wrap it naturally — no overflow, no separate bezel rotation. drawPending watches the buffer's aspect ratio; if it flips between portrait and landscape (i.e. iOS is re-rendering for the new orientation) the in-canvas rotation auto-resets to 0 so we don't double-rotate already-oriented content. Touch coordinates project through the inverse rotation back into the device buffer's frame before normalising to HID 0..65535.
New util-tray button that pauses keydown/keyup forwarding so the browser receives keys normally (Cmd-L, Cmd-W, devtools shortcuts, etc). The toggle is purely client-side — no device-side release flush on toggle-off, since the device only knows about the keys we've sent. Preference persisted in localStorage.
Modal lists the Ctrl-hotkeys and a brief explainer of how the keyboard-capture toggle and rotation buttons interact. Opens via the '?' key or the new util-tray '?' button; closes on Esc, click outside the card, or the X. While open all keystrokes are gated from the device so typing into the page (e.g. searching for help) doesn't leak through.
The virtual keyboard surface is host-registered against the live media stream via UniversalHIDServiceService.create_keyboard_service. When the stall watchdog restarts the stream, dtuhidd publishes fresh HID surfaces and the previous _ServiceID becomes stale — every report posted to it is silently dropped by backboardd. _stop_hid already drops the UHS/Indigo channel handles; also drop _kb_service_id so the next /key triggers _ensure_keyboard to register a fresh surface against the new stream context.
Moves the utility buttons (Sound / Style / Frame / Restart / Help / Rotate / Keyboard) into a left-side panel that participates in the document flow instead of position:fixed, so a wide / rotated canvas can't slide underneath them. Each tray has a chevron toggle to collapse to a thin strip and reclaim canvas room; the open/closed state persists in localStorage. Also lands: - 'Screenshot' button + Ctrl+P hotkey -> canvas.toBlob saved as PNG (uses the canvas backing store as-is so the file matches what the user sees, including in-canvas rotation). - 'Reload' button -> location.reload() for a full client reset distinct from /restart (which only restarts the device stream). - Offline overlay over the canvas when frameCount hasn't advanced in ~3 s while subscribed -- gives the user a clear 'stream is stuck, here's why your input isn't doing anything' signal. fitCanvasToViewport now MEASURES the trays' actual widths instead of the previous hard-coded 160 px reserve, so collapse/uncollapse immediately reflows the canvas to the new available space.
Right-side collapsible tray that mirrors 'pymobiledevice3 developer accessibility settings show' -- each AXAuditDeviceSetting renders as a checkbox (bool) or 0..1 slider (float, e.g. DYNAMIC_TYPE), and edits round-trip through AccessibilityAudit.set_setting on the device. 'Reset all' calls reset_settings. Lockdown / DTX comes up lazily on the first /accessibility request, so a serve-web run that never opens the panel doesn't pay the usbmuxd handshake cost.
The accessibility sidebar leaked one fresh AccessibilityAudit per request -- each held an open DTX channel whose reader loop got cancelled at process exit, producing a stack of 'Channel reader loop cancelled' ERROR-level tracebacks in the log. Cache the audit handle on the server, reuse it across panel interactions, and close it in the shutdown sequence so the DTX channel exits via the normal close() path (channel._closed=True suppresses the cancellation traceback). serve-web shutdown is now clean: one 'shutting down…' line, 'hid worker cancelled' acknowledgement, 'shutdown complete'.
Rotation used to snap instantly -- the canvas dim swap + content redraw all happened in one frame. Add a 300 ms eased CSS transform on #device-frame so the visual rotation animates while the canvas-internal rotation still snaps to the final orientation underneath. Mechanism: on each setVisualRotation() the canvas dims + content update immediately (so the surrounding layout reflows now), then #device-frame gets transform=rotate(-delta) with transition=none so its visual position matches where it was a moment ago; a double-RAF then enables a cubic-bezier transition and animates the transform back to identity. The result is the canvas content appearing to rotate smoothly into place over 300 ms while the flex layout reflows in step.
Hitting /accessibility right before SIGINT produced a 'coroutine ignored GeneratorExit' RuntimeError on Python 3.14: the request handler was mid-bplist decode (synchronous plistlib code, not at an await point) when shutdown closed the audit out from under it, and the loop's coroutine cleanup couldn't unwind through the non-awaiting frame. _stop_accessibility now takes _accessibility_lock first, so any in-flight handler completes its decode (or hits the outer 3s shutdown bound) before we tear the DTX channel down. Verified across 5 stress trials: 0 channel-reader tracebacks, 0 GeneratorExit warnings, same clean three-line shutdown log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.