Proposal: reuse Apple's CoreDevice tunnel on macOS instead of bringing up our own

[aluedeke]

Context

On macOS Tahoe (26+), reading the pair record at /var/db/lockdown/RemotePairing/ is now TCC-walled and requires Apple-signed code. This breaks the current "bring up our own QUIC tunnel" path for any non-Apple-signed binary on the host. Tracked as #710.

Meanwhile, on every macOS host that has Xcode installed, Apple's remotepairingd is already maintaining a trusted device tunnel for every paired iPhone. Today go-ios fights this — the README recommends pkill -SIGSTOP remoted to keep Apple's daemon out of the way. That recommendation gets harder to live with on Tahoe.

Proposal: on macOS, borrow the tunnel that Apple's daemons have already established, instead of bringing up our own.

Approach

Talk to the user-domain XPC mach service com.apple.CoreDevice.CoreDeviceService directly. It is reachable from any process — devicectl itself ships with no CoreDevice-private entitlements. The daemon maintains its own pair record and tunnel; we consume it.

Apple's API model is connection-by-name, not "give me a tunnel address." The flow:

1. Open XPC mach connection to com.apple.CoreDevice.CoreDeviceService via xpc_connection_create_mach_service.
2. Send DeviceManagerCheckInRequest (Mercury "Codable-over-XPC" envelope: {mangledTypeName, value} with the request UUID inside value). Receive ServiceEvent → DeviceManagerCheckInCompleteEvent → initialDeviceSnapshots[] asynchronously on the connection's event handler. Each snapshot's deviceInfo sub-dict contains udid, pairingState, tunnelIPAddressString, tunnelTransportProtocol, etc.
3. Send CreateServiceConnection*Request (the framework exposes CreateServiceConnectionProtocol and CreateServiceConnectionCapability) with (deviceIdentifier, serviceName). Daemon returns an FD/sub-connection that already routes through the tunnel.
4. Wrap the returned FD as net.Conn and feed it to existing service-specific code (appservice, dtx_codec, fileservice, deviceinfo, …).

No private entitlements required. The dynamic RSD port (the thing that's been awkward to discover from third-party code) is never needed — Apple does the lookup server-side.

Architectural change

The current public shape — ConnectToServiceTunnelIface(device, serviceName) — is already the right abstraction. The implementation just leaks the tunnel internally. The refactor turns that leak into a backend choice inside the daemon, with a single CLI-side connector that always speaks HTTP.

// New, in ios/services
type ServiceConnector interface {
    Dial(ctx context.Context, serviceName string) (ios.DeviceConnectionInterface, error)
}

// XPC framing is a free helper, not on the interface — keeps the
// transport contract minimal and avoids dragging the xpc package into
// the connector's dependency graph.
func DialXPC(ctx context.Context, c ServiceConnector, serviceName string) (*xpc.Connection, error)

func NewServiceConnector(ctx context.Context, device ios.DeviceEntry, opts Opts) (ServiceConnector, error)

The connector itself is stateless (just an HTTP-client wrapper to the daemon); each Dial returns a net.Conn the caller owns and closes when done. No connector-level Close() because XPC mach connections, tunnel state, etc. all live inside the daemon.

CLI-side: HTTP GET 127.0.0.1:28100/connect/{udid}/{service} returns {127.0.0.1, fwdPort}, the connector dials localhost, returns net.Conn. Same code path on every platform; no XPC, no SCM_RIGHTS, no platform-specific Go. The daemon is always in the data path — it binds an ephemeral forward listener per service connection and proxies bytes between it and the upstream.

Two backends, both proxy via the same forward-port helper. Only the upstream source differs:

Backend	Where the upstream comes from	Notes
tunnelBackend (any platform)	TCP socket to [tunnel-IPv6]:rsd-port, dialed by the daemon after RSD lookup	Underlying transport (kernel utun vs. userspace gvisor) is daemon-internal; today's --userspace flag stays as an explicit override
appleBackend (macOS only)	FD returned by com.apple.CoreDevice.CoreDeviceService via XPC; unwrapped via xpc_fd_dup → os.NewFile → net.FileConn	Pure Go with CGo against libxpc; one darwin-only file, build-tagged

Trade-off vs. today's behaviour: today's kernel-utun mode has the CLI dialing the device's tunnel IPv6, with the OS routing those packets through utun where the daemon picks them up and wraps them in QUIC. The new design swaps that end-to-end TCP over utun for a userspace TCP bridge inside the daemon — CLI dials 127.0.0.1:fwdPort, daemon owns a separate TCP/FD upstream, daemon io.Copys between the two. The proxy must explicitly propagate half-close / RST / deadlines (easy, but code that has to exist). Throughput cost on localhost is small.

Who actually feels the change: today on macOS, kernel-utun mode requires sudo ios tunnel start; in practice virtually all macOS users run --userspace to avoid that, which already does the localhost proxy hop. Linux users running the daemon as root and Windows users on the wintun direct path are the only ones who'll see a regression. They can override the daemon's internal transport via the existing --userspace flag if needed. The architectural simplification (one code path, no device.Address/device.Rsd leak to CLI consumers, daemon owns every flow for shutdown/observability) is judged worth the cost.

appleBackend is implemented with CGo against libxpc. The daemon opens one long-lived XPC mach connection to com.apple.CoreDevice.CoreDeviceService at startup, sends CreateServiceConnection*Request envelopes (Mercury "Codable-over-XPC" wire format: {mangledTypeName, value} dictionaries), unwraps the returned xpc_fd_t, wraps as net.Conn. CGo cost is contained to one darwin-only file (ios/tunnel/apple_backend_darwin.go); Linux/Windows builds keep CGO_ENABLED=0.

The existing HTTP API on 127.0.0.1:28100 keeps everything it does today (GET /tunnels, GET /tunnel/{udid}) and gains the one new endpoint above. Nothing is deprecated.

I count 13 call sites total (ConnectToService, ConnectToShimService, ConnectToServiceTunnelIface, ConnectToXpcServiceTunnelIface and their consumers in appservice, deviceinfo, dtx_codec, fileservice). All keep their current signatures; only the internals change.

Phasing — independent PRs

Phase	What	Behavior change
1	Extract ServiceConnector interface; add GET /connect/{udid}/{service} to the daemon's HTTP API; move RSD lookup into the daemon; switch all 13 call sites to the new connector	new HTTP endpoint; CLI consumer code stops reading device.Address / device.Rsd
2	appleBackend inside the daemon (CGo against libxpc) + per-device backend factory; reuses the existing userspace-mode forward-port proxy helper for the FD-to-localhost handoff	new opt-in GO_IOS_BACKEND=apple daemon env var
3	Auto-detect on macOS (daemon picks appleBackend per-device when CoreDeviceService knows the device)	macOS daemon default flips to Apple
4	UX polish (ios tunnel ls shows backend per device, daemon prints hint when delegating to Apple, optional CLI auto-launches daemon)	cosmetic

Phase 1 is worth landing on its own even if Phase 2+ is rejected — the cleaner abstraction (daemon-owned RSD lookup, no consumer-visible port knowledge) stands on its own.

The CoreDeviceService-specific code lives entirely in Phase 2 inside one darwin-only file (ios/tunnel/apple_backend_darwin.go), so the rest of the codebase doesn't carry that knowledge.

Design decisions for review

1. Single CLI-side path; backends live in the daemon. Considered putting the Apple integration in the CLI process (per-invocation Swift helper). Rejected: forced two control flows in the CLI forever and paid XPC setup cost (~tens of ms) per CLI invocation. The daemon-owned approach keeps the CLI uniform across platforms, amortises the XPC connection across all CLI invocations, and centralises all backend logic.
2. Daemon always in the data path; one proxy helper for both backends. Considered keeping today's kernel-utun direct-dial path as an optimisation; rejected for the architectural simplification. With always-proxy, there's one code path in the daemon, no device.Address leak to CLI consumers, and the daemon owns every flow for clean shutdown / observability / future features. Cost is a localhost proxy hop on every connection — long-accepted for --userspace users; the new baseline is "userspace experience for everyone."
3. CGo against libxpc for appleBackend, no Swift bridge. Considered a separate Swift child process passing FDs back via SCM_RIGHTS; rejected because the daemon is already long-lived and CGo against libxpc accomplishes the same thing without a second binary, subprocess supervision, or SCM_RIGHTS plumbing. CGo cost is contained to one darwin-only file behind a build tag — Linux/Windows builds keep CGO_ENABLED=0. Cross-compilation to macOS from non-macOS hosts gets harder (rare workflow).
4. Per-device backend selection, not per-host. Mixed setups (one iPhone via Apple, one Apple TV via Wi-Fi-direct) work transparently — the daemon picks per device.

Open questions

• Does the Apple-supplied FD support half-close semantics that some go-ios callers rely on? (The daemon's forward-port proxy needs to propagate this in both directions.)
• Will Apple's tunnel survive while only our daemon is checked in (vs. when Xcode is also running)?
• Forward-port proxy throughput for high-throughput services (instruments, screen recording, file transfer) — acceptable as the new universal default, or is the regression vs. today's kernel-utun direct-dial a deal-breaker? Today's --userspace mode has lived with this trade-off since it shipped — the new baseline matches that.
• Any concerns about adding CGo against libxpc to a single darwin-only file? (Cross-compilation to macOS from Linux/Windows hosts gets harder.) Alternative is a small Swift helper binary spawned by the daemon — extra artifact and SCM_RIGHTS plumbing but stays pure Go in the main process.
• Is the project's policy on Apple-private-XPC reverse-engineering acceptable for Phase 2+, or should the macOS-Apple path live in a fork?
• For the Phase 1 endpoint: minimal GET /connect/{udid}/{service} → {address, port} — anything you'd want richer (long-poll for tunnel-up, batched lookup)?

Happy to start with Phase 1 as a standalone refactor PR if there's interest — it stands on its own merits (cleaner abstraction, daemon-owned RSD lookup, no breaking changes) even if Phase 2+ doesn't land.

[aluedeke]

@gmegidish @danielpaulus @dmissmann curious to hear your opinion, breaking the current tunnel interface might have some negative impact, reusing the existing tunnel on osx would be a great feature addition. Technically we could implement the cli to connect to the osx device tunnel directly which would improve usability as no extra daemon would be needed.

[nanoscopic]

I suggested this a bunch of times and I've implemented this myself for my own C implementation of RSD...

Yes. You can do it. 🤷

As for reverse engineering and private calls, this entire project is based on RE of private Apple APIs. Albeit initially only the protocols were analyzed and reimplemented much of the later stuff was based on introspection and decomp. That's all water under the bridge from my point of view.

You don't need to do all the fancy calling of XPC stuff though; there are more straightforward ways to hold open a tunnel... but the way you have laid out sounds pretty interesting.