Implement remoteFetchSymbols service client and ios fetchsymbols command (#700)

[danielpaulus]

Problem

Resolving symbol addresses of a remote iOS process currently requires parsing Mach-O headers over GDB RSP memory reads. Only Xcode can download the dyld shared cache ("Preparing debugger support"), so go-ios users have no way to get device libraries locally for fast symbol resolution, crash symbolication or offline analysis (#700).

Design

New RemoteXPC client for com.apple.dt.remoteFetchSymbols, mirroring pymobiledevice3's RemoteFetchSymbolsService and the file-transfer stream handling in its RemoteXPCConnection:

1. Send {"XPCDictionary_sideChannel": <uuid>, "DSCFilePaths": []} (wanting-reply flag) → the reply carries the file count.
2. Receive one metadata message per file on the reply channel: filePath plus an XPC file-transfer object whose s property is the expected byte length (go-ios already decodes this as xpc.FileTransfer).
3. For file i, open HTTP2 stream (i+1)*2 with an empty XPC wrapper flagged FILE_TX_STREAM_RESPONSE (0x00200000); the device streams the raw file bytes as DATA frames on that stream, terminated by END_STREAM.
4. Store files under <dir>/<ProductVersion>_<BuildVersion>/<on-device path> so the multi-GB download is cached per iOS version/build and reused across devices.

This is RemoteXPC only — no dtx/nskeyedarchiver code is touched.

Implementation

• ios/http: file-transfer side streams for the existing HttpConnection — NewFileStreamReadWriter registers an extra stream id, DATA frames for it are buffered (interleaved stream 1/3 frames keep working), END_STREAM surfaces as io.EOF, and received bytes are granted back to the connection + stream flow-control windows (batched at 256 KiB, force-flushed at stream end) so multi-gigabyte transfers don't stall on the 1 MiB initial window.
• ios/xpc: new FileTransferStreamResponseFlag const; FileTransfer encode support (inverse of the existing decoder) for codec symmetry and fixture tests.
• ios/fetchsymbols: New/ListFiles/DownloadFile/DownloadToCache client plus caching helpers: traversal-safe CachePath (device-controlled paths cannot escape the cache dir), IsCached (size match), atomic temp-file + rename writes so interrupted downloads never look cached, and a progress callback.
• CLI: ios fetchsymbols [--path=<dir>] (default ./ios-symbols), gated on a running tunnel, prints an explicit multi-gigabyte-download warning with file count/total size before transferring, logs progress every 512 MiB, skips already-cached files, JSON summary output. Registered in the command registry, needsAutomaticTunnelInfo, help catalog and usage doc.

Options considered

1. Reuse ios/fileservice's raw data connection — rejected: remoteFetchSymbols does not use the com.apple.coredevice.fileservice.data wire protocol; its content arrives on extra HTTP2 streams of the same XPC connection.
2. Give xpc.Connection a built-in file-transfer API — rejected for now: xpc stays transport-agnostic; the side-stream mechanics live in ios/http (which owns the HTTP2 framing) and the service package composes them. Smallest surface change to shared code.
3. Concurrent downloads (pymobiledevice3 uses 4 workers) — rejected: HttpConnection is intentionally single-threaded like all other go-ios XPC services; files download sequentially. Simpler, and throughput is tunnel-bound anyway. (Preferred: option 2/3 as implemented.)
4. Prompt y/N before downloading — rejected in favor of explicit opt-in messaging: the command itself is the opt-in, and a prompt would break automation; a clear warning with total size is printed before the transfer starts.

Test plan

• go build ./... and go test ./... pass; gofmt -l clean on changed files.
• Unit tests added:
  • ios/fetchsymbols: DSCFilePaths request envelope encode/decode roundtrip (side-channel UUID included), file-count and per-file metadata fixture decode + parsing (incl. the fileTransfer expected length), chunk reassembly (copyFileChunks) incl. short-stream and error propagation, cache-path traversal safety, cached-size checks, atomic write semantics, progress reporting.
  • ios/http: fake in-memory HTTP2 server test that streams a 360 KiB payload in 16 KB DATA frames on a file stream — verifies reassembly, EOF on END_STREAM, buffering of interleaved reply-channel data, full flow-control window replenishment, and rejection of unknown/duplicate/reserved stream ids.
• Follow-up: the real download path gets validated via e2e on an iOS 17+ farm device (needs tunnel + several GB of transfer, so it is not part of the device-free unit suite).

Fixes #700

🤖 Generated with Claude Code

https://claude.ai/code/session_01J8eMENxJ1nec9CeHp4tjWk

[danielpaulus]

/test-devices

[github-actions]

🧪 Running real-device tests on PR #814 — run.

[github-actions]

❌ Real-device tests failed — see run.

[danielpaulus]

Adversarial review (Claude)

Reviewed the full diff with focus on the shared HTTP2 layer, XPC FileTransfer codec, chunk reassembly, and path handling. One correctness defect fixed; the rest of the PR holds up well.

Fixed

• Flow-control window under-credit on padded DATA frames (ios/http/http.go). replenishReceiveWindow was called with len(d.Data()), which excludes the Pad Length byte and padding. RFC 7540 §6.9.1 requires the entire DATA frame payload (padding included) to be accounted for in flow control, so padded frames would leak window on both the connection and the file stream and could eventually stall a multi-GB dyld cache download. Now credits d.Length (frame header length). Added TestFileStreamPaddedFramesCreditFullLength, which sends exactly-sized padded frames and asserts the full frame length is credited back (fails on the old code). (commit 6cec6f3)

Verified correct (no change needed)

• Frame routing / no regression to XPC control streams: streams 0/1/3 keep their existing special-casing; only genuinely-unknown stream ids still error. File streams are registered explicitly before use.
• Window batching threshold: 256 KiB batch with flush-on-END_STREAM; no off-by-one at the boundary and well under the 1 MiB initial window, so no stall. Both connection (stream 0) and per-stream windows are replenished — correct RFC 7540 dual accounting for the transfer.
• Buffer growth is bounded: readFileStream pulls one frame at a time only when its buffer is empty, so the consumer pull rate bounds fileStream.buf; not unbounded.
• Chunk reassembly: copyFileChunks copies exactly Size bytes via LimitReader; short stream / mid-file EOF surfaces as an error, read errors propagate. Covered by tests.
• FileTransfer encode/decode roundtrip: encodeFileTransfer is the exact inverse of the pre-existing decoder (dict key "s", matching the wire format other services already use); 0x00200000 matches pymobiledevice3's file-transfer stream-response flag. Roundtrip test passes.
• Path traversal: CachePath neutralizes ../ and backslash traversal lexically and rejects empty/root/dots-only paths; tested.
• Atomic write / crash safety: download goes to *.download and is renamed only on success; IsCached checks regular-file + exact size, so an interrupted download never looks complete. Tested.

Considered and dismissed (out of scope / not regressions)

• Connection window not replenished for streams 1/3 — pre-existing behavior; those streams only carry small XPC control messages, and the large file data (on streams 2/4/6) is credited to the connection window. Not made worse by this PR.
• Symlink redirect under the cache dir — requires a local attacker to pre-plant a symlink in the user's own output directory; the device (the untrusted party here) only controls the lexical path, which is confined. Same posture as existing file pull / fsync pull; not specific to this PR.
• No per-operation timeout on a stalled device — consistent with every other go-ios XPC service (Ctrl-C is the escape); not a regression.
• Device could under-report Size — the size is device-authoritative with no independent manifest to cross-check (same as pymobiledevice3); not fixable here.

Test status

go build ./..., go test ./..., and go test -race ./ios/http/ ./ios/fetchsymbols/ all pass; gofmt clean.

Not merging — leaving that to a maintainer.