feat(ios): support gateway WDA base URLs (#3187)

* feat(ios): support gateway WDA base URLs

* fix(ios): reject conflicting WDA connection options

* feat(ios): support independent gateway MJPEG URLs

* fix(studio): preserve iOS gateway target identity and replay config

* fix(studio): retain legacy iOS recording history matches

* test(playground): keep MJPEG mock cleanup void

* test(ios): exercise path-prefixed WDA on simulator CI

* test(ios): exercise explicit legacy WDA host and port

Author: quanru

apps/site/docs/en/automate-with-scripts-in-yaml.mdx

@@ -367,6 +367,17 @@ ios:
   # WebDriverAgent host address, optional, defaults to localhost.
   wdaHost: <host>
 
+  # For gateways with a path prefix, use wdaBaseUrl instead of wdaHost/wdaPort.
+  # Setting wdaBaseUrl together with either wdaHost or wdaPort causes an error.
+  # wdaBaseUrl: <url>
+
+  # Optional independent MJPEG stream URL, including a gateway path if needed.
+  # Cannot be combined with wdaMjpegPort.
+  # wdaMjpegUrl: <url>
+
+  # Local MJPEG stream port, optional. Use this or wdaMjpegUrl.
+  # wdaMjpegPort: <port>
+
   # Whether to auto dismiss keyboard, optional, defaults to false.
   autoDismissKeyboard: <boolean>
 
@@ -383,6 +394,16 @@ ios:
   # See the IOSDevice constructor documentation for the complete list
 ```
 
+For a gateway with a path prefix, set `wdaBaseUrl` and leave `wdaHost` and `wdaPort` unset. The MJPEG stream can use a separate URL:
+
+```yaml
+ios:
+  wdaBaseUrl: ${WDA_BASE_URL}
+  wdaMjpegUrl: ${WDA_MJPEG_URL}
+```
+
+Set both environment variables before running the script. Studio Recorder exports use these references so gateway paths and access tokens stay out of the YAML file.
+
 :::info View Complete iOS Configuration Options
 
 YAML scripts now support all configuration options from the `IOSDevice` constructor. For the complete list of options, see [`IOSDevice`](./reference/#iosdevice) in the iOS API reference.

apps/site/docs/en/platforms/ios.mdx

@@ -233,6 +233,27 @@ For remote devices, you also need to set up port forwarding accordingly:
 iproxy 8100 8100 YOUR_DEVICE_ID
 ```
 
+If a gateway exposes WDA under a path prefix, set the full API base URL instead:
+
+```typescript
+const agent = await agentFromWebDriverAgent({
+  wdaBaseUrl: 'https://gateway.example/device/wda',
+});
+```
+
+Midscene appends `/status`, `/session`, and session commands to this base URL. Set either `wdaBaseUrl` or `wdaHost`/`wdaPort`; combining them throws an error.
+
+`wdaBaseUrl` only configures the WDA API. By default, the native MJPEG stream still connects to `http://localhost:9100` when using a gateway. If the gateway also exposes a stream, set its complete URL separately:
+
+```typescript
+const agent = await agentFromWebDriverAgent({
+  wdaBaseUrl: 'https://gateway.example/device/wda',
+  wdaMjpegUrl: 'https://gateway.example/device/mjpeg',
+});
+```
+
+`wdaMjpegUrl` accepts an HTTP(S) stream URL with its own host, port, path, and query. Credentials and fragments in the URL are rejected. It cannot be combined with `wdaMjpegPort`. For a local port forward, leave `wdaMjpegUrl` unset and use `wdaMjpegPort` (default `9100`). Playground falls back to screenshot polling if the native stream is unavailable.
+
 ### How to get smoother live screen preview in Playground?
 
 Playground's screen preview supports two modes:

apps/site/docs/en/reference/index.mdx

@@ -2499,9 +2499,11 @@ const device = new IOSDevice({
 
 - `wdaPort?: number` — WebDriverAgent port. Default: `8100`.
 - `wdaHost?: string` — WebDriverAgent host. Default: `'localhost'`.
+- `wdaBaseUrl?: string` — Full HTTP(S) WDA API base URL, including a gateway path prefix. Cannot be combined with `wdaHost` or `wdaPort`.
 - `iOSDeviceClassOverride?: string` — Optional npm module path that replaces the default `IOSDevice` when using `agentFromWebDriverAgent()` or iOS Playground. The module must export an `IOSDevice` class or a default class.
 - `sessionId?: string` — Existing WebDriverAgent session ID to reuse. When provided, Midscene skips creating a new WDA session. During cleanup, Midscene detaches from the externally supplied WebDriver session instead of deleting it.
 - `wdaMjpegPort?: number` — WDA MJPEG server port for real-time screen streaming. Default: `9100`.
+- `wdaMjpegUrl?: string` — Full HTTP(S) MJPEG stream URL, including any gateway path or query. Cannot be combined with `wdaMjpegPort`.
 - `wdaMjpegFrameSource?: { enabled?: boolean }` — Use WDA's MJPEG stream as the continuous frame source for `agent.startObserving()`. Disabled by default; when disabled, observers fall back to sequential `screenshotBase64()` capture.
 - `autoDismissKeyboard?: boolean` — Whether to hide the on-screen keyboard after text input. Default: `true`.
 - `keyboardTypeDelay?: number` — Finite non-negative delay in milliseconds between keystrokes. A positive value makes legacy input enter one Unicode code point at a time through WDA's `/wda/keys` endpoint. Use this option when an input field drops characters during fast input.
@@ -2512,6 +2514,8 @@ const device = new IOSDevice({
 
 - Ensure Developer Mode is enabled and WDA can reach the device; use `iproxy` when forwarding ports from a real device.
 - Use `wdaHost`/`wdaPort` to target remote devices or custom WDA deployments.
+- Use `wdaBaseUrl` when a gateway routes WDA through a path prefix; all WDA API requests and readiness checks use that prefix.
+- Set `wdaMjpegUrl` separately when the native MJPEG stream is available through a gateway. The API base URL does not determine the stream URL.
 - For multi-device concurrency, use distinct `wdaPort` and `wdaMjpegPort` values for each device so WDA commands and MJPEG streams do not conflict.
 - For shared interaction methods, see [Shared Agent APIs](#interaction-methods).
 

apps/site/docs/zh/automate-with-scripts-in-yaml.mdx

@@ -370,6 +370,16 @@ ios:
   # WebDriverAgent 主机地址，可选，默认 localhost
   wdaHost: <host>
 
+  # 网关带路径前缀时，用 wdaBaseUrl 替代 wdaHost/wdaPort。
+  # 同时设置 wdaBaseUrl 与 wdaHost 或 wdaPort 会报错。
+  # wdaBaseUrl: <url>
+
+  # 可单独设置 MJPEG 流地址，包括网关路径；不能与 wdaMjpegPort 同时设置。
+  # wdaMjpegUrl: <url>
+
+  # 本地 MJPEG 流端口，可选。与 wdaMjpegUrl 二选一。
+  # wdaMjpegPort: <port>
+
   # 是否自动关闭键盘，可选，默认 false
   autoDismissKeyboard: <boolean>
 
@@ -386,6 +396,16 @@ ios:
   # 完整配置项请参考 IOSDevice 的构造函数文档
 ```
 
+连接带路径前缀的网关时，只设置 `wdaBaseUrl`。MJPEG 流可以使用单独的地址：
+
+```yaml
+ios:
+  wdaBaseUrl: ${WDA_BASE_URL}
+  wdaMjpegUrl: ${WDA_MJPEG_URL}
+```
+
+运行脚本前，请设置这两个环境变量。Studio Recorder 导出的 YAML 也引用环境变量。这样，网关路径和访问令牌不会写入文件。
+
 :::info 查看完整的 iOS 配置项
 
 YAML 脚本现在支持 `IOSDevice` 构造函数的所有配置选项。完整的配置项列表请参考 [iOS API 参考中的 IOSDevice](./reference/#iosdevice)。

apps/site/docs/zh/platforms/ios.mdx

@@ -233,6 +233,29 @@ const agent = await agentFromWebDriverAgent({
 iproxy 8100 8100 YOUR_DEVICE_ID
 ```
 
+如果网关把 WDA 放在带路径前缀的地址下，可以直接配置完整的 API 基地址：
+
+```typescript
+const agent = await agentFromWebDriverAgent({
+  wdaBaseUrl: 'https://gateway.example/device/wda',
+});
+```
+
+Midscene 会在该地址后追加 `/status`、`/session` 和会话接口路径。连接网关时，只设置 `wdaBaseUrl`。如果使用主机和端口，则设置 `wdaHost` 和 `wdaPort`。两种方式混用会报错。
+
+`wdaBaseUrl` 只配置 WDA API。MJPEG 流默认连接 `http://localhost:9100`。如果网关也提供画面流，请单独设置完整地址：
+
+```typescript
+const agent = await agentFromWebDriverAgent({
+  wdaBaseUrl: 'https://gateway.example/device/wda',
+  wdaMjpegUrl: 'https://gateway.example/device/mjpeg',
+});
+```
+
+`wdaMjpegUrl` 可以使用独立的协议、主机、端口、路径和查询参数。协议须为 HTTP(S)，地址中不能包含用户名、密码或片段。`wdaMjpegUrl` 与 `wdaMjpegPort` 不能同时设置。
+
+如果使用本地端口转发，可以设置 `wdaMjpegPort`，默认值为 `9100`。画面流不可用时，Playground 会退回到截图轮询。
+
 ### 如何在 Playground 中获得更流畅的实时画面？
 
 Playground 的画面预览支持两种模式：

apps/site/docs/zh/reference/index.mdx

@@ -2457,9 +2457,11 @@ const device = new IOSDevice({
 
 - `wdaPort?: number` —— WebDriverAgent 端口，默认 `8100`。
 - `wdaHost?: string` —— WebDriverAgent host，默认 `'localhost'`。
+- `wdaBaseUrl?: string` —— 完整的 HTTP(S) WDA API 基地址，可包含网关路径前缀。不能与 `wdaHost` 或 `wdaPort` 同时设置。
 - `iOSDeviceClassOverride?: string` —— 使用 `agentFromWebDriverAgent()` 或 iOS Playground 时替换默认 `IOSDevice` 的 npm module path。目标模块必须导出 `IOSDevice` class 或 default class。
 - `sessionId?: string` —— 复用已有的 WebDriverAgent session ID。传入后，Midscene 不再创建新的 WDA session；清理时只会从这个外部 WebDriver session 分离，不会删除它。
 - `wdaMjpegPort?: number` —— WDA MJPEG 服务端口，用于实时画面流，默认 `9100`。
+- `wdaMjpegUrl?: string` —— 完整的 HTTP(S) MJPEG 画面流地址，可包含网关路径或查询参数。不能与 `wdaMjpegPort` 同时设置。
 - `wdaMjpegFrameSource?: { enabled?: boolean }` —— 使用 WDA 的 MJPEG stream 作为 `agent.startObserving()` 的连续帧源。默认关闭；关闭时，观察逻辑会退回到连续调用 `screenshotBase64()`。
 - `autoDismissKeyboard?: boolean` —— 文本输入后自动隐藏键盘，默认 `true`。
 - `keyboardTypeDelay?: number` —— 按键间延迟，单位为毫秒。取值必须是有限的非负数。设为正数后，`legacy` 输入会通过 WDA 的 `/wda/keys` 接口逐个 Unicode 码点执行。适用于输入框在快速输入下丢字的场景。
@@ -2470,6 +2472,8 @@ const device = new IOSDevice({
 
 - 请确认已开启开发者模式且 WDA 能访问设备；真机转发端口时可借助 `iproxy`。
 - 通过 `wdaHost`/`wdaPort` 可指向远程设备或自建的 WDA。
+- 网关通过路径前缀转发 WDA 时，使用 `wdaBaseUrl`；WDA API 请求和就绪探测都会使用该前缀。
+- 网关提供原生 MJPEG 流时，单独设置 `wdaMjpegUrl`。WDA API 基地址不会决定画面流地址。
 - 多设备并发时，请为每个设备设置不同的 `wdaPort` 和 `wdaMjpegPort`，避免 WDA 命令和 MJPEG stream 端口冲突。
 - 通用交互方法请查阅 [API 参考（通用）](#interaction-methods)。
 

apps/studio/src/renderer/playground/selectors.ts

@@ -2,6 +2,7 @@ import type {
   PlaygroundRuntimeInfo,
   PlaygroundSessionTarget,
 } from '@midscene/playground';
+import { sha256Hex } from '@midscene/shared/utils';
 import type {
   DiscoveredDevice,
   PlatformDiscoveryError,
@@ -47,6 +48,40 @@ function buildHostPortId(host: string, port: number): string {
   return `${host}:${port}`;
 }
 
+function iosFormValue(formValues: Record<string, unknown>, key: string) {
+  const value = formValues[`ios.${key}`] ?? formValues[key];
+  return typeof value === 'string' && value.trim() ? value.trim() : undefined;
+}
+
+function normalizedGatewayUrl(value: string) {
+  const url = new URL(value);
+  return url.toString().replace(/\/+$/, '');
+}
+
+export function resolveSelectedIosGatewayId(
+  formValues: Record<string, unknown>,
+): string | undefined {
+  const baseUrl = iosFormValue(formValues, 'baseUrl');
+  if (!baseUrl) return undefined;
+  try {
+    const mjpegUrl = iosFormValue(formValues, 'mjpegUrl');
+    const sessionId = iosFormValue(formValues, 'sessionId');
+    const mjpegPort = normalizePort(
+      formValues['ios.mjpegPort'] ?? formValues.mjpegPort,
+    );
+    return `ios-gateway-${sha256Hex(
+      JSON.stringify({
+        baseUrl: normalizedGatewayUrl(baseUrl),
+        mjpegUrl: mjpegUrl ? new URL(mjpegUrl).toString() : undefined,
+        mjpegPort,
+        sessionId,
+      }),
+    )}`;
+  } catch {
+    return undefined;
+  }
+}
+
 /**
  * Map any incoming platform string (runtime metadata, form values, desktop
  * OS aliases like `macos`) to the canonical `StudioPlatformId`. Exported so
@@ -84,7 +119,7 @@ export function normalizeStudioPlatformId(
  * Platforms use different metadata keys for the device id:
  *   Android / Harmony → metadata.deviceId
  *   Computer          → metadata.displayId
- *   iOS               → metadata.wdaHost + metadata.wdaPort
+ *   iOS               → gateway URL digest, or metadata.wdaHost + metadata.wdaPort
  */
 export function resolveConnectedDeviceId(
   runtimeInfo: PlaygroundRuntimeInfo | null,
@@ -96,6 +131,9 @@ export function resolveConnectedDeviceId(
   if (isString(metadata.displayId)) {
     return metadata.displayId;
   }
+  if (isString(metadata.wdaGatewayId)) {
+    return metadata.wdaGatewayId;
+  }
   if (isString(metadata.wdaHost)) {
     const wdaPort = normalizePort(metadata.wdaPort);
     if (wdaPort !== undefined) {
@@ -108,6 +146,7 @@ export function resolveConnectedDeviceId(
 function resolveConnectedSessionValues(
   runtimeInfo: PlaygroundRuntimeInfo | null,
   platformKey: StudioSidebarPlatformKey,
+  formValues: Record<string, unknown> = {},
 ): Record<string, StudioSessionValue> | undefined {
   const metadata = runtimeInfo?.metadata || {};
 
@@ -126,7 +165,20 @@ function resolveConnectedSessionValues(
           }
         : undefined;
     case 'ios': {
+      if (isString(metadata.wdaGatewayId)) {
+        return resolveSelectedIosGatewayId(formValues) === metadata.wdaGatewayId
+          ? resolveSelectedSessionValues('ios', formValues)
+          : undefined;
+      }
       const wdaPort = normalizePort(metadata.wdaPort);
+      if (
+        isString(metadata.wdaHost) &&
+        wdaPort !== undefined &&
+        resolveSelectedDeviceId({ ...formValues, platformId: 'ios' }) ===
+          buildHostPortId(metadata.wdaHost, wdaPort)
+      ) {
+        return resolveSelectedSessionValues('ios', formValues);
+      }
       return isString(metadata.wdaHost) && wdaPort !== undefined
         ? {
             host: metadata.wdaHost,
@@ -155,6 +207,9 @@ export function resolveConnectedDeviceLabel(
   }
   const deviceId = resolveConnectedDeviceId(runtimeInfo);
   if (deviceId) {
+    if (isString(metadata.wdaGatewayId) && isString(metadata.wdaHost)) {
+      return `${metadata.wdaHost} (WDA gateway)`;
+    }
     // "Display 1" reads better than a bare numeric id for computer.
     return isString(metadata.displayId) && !isString(metadata.deviceId)
       ? `Display ${deviceId}`
@@ -169,13 +224,11 @@ export function resolveConnectedDeviceLabel(
 function buildGenericConnectedDeviceItem(
   runtimeInfo: PlaygroundRuntimeInfo | null,
   platformKey: StudioSidebarPlatformKey,
+  formValues: Record<string, unknown>,
 ): StudioAndroidDeviceItem | null {
   const metadata = runtimeInfo?.metadata || {};
   const deviceId = resolveConnectedDeviceId(runtimeInfo);
-  const label = isString(metadata.sessionDisplayName)
-    ? metadata.sessionDisplayName
-    : deviceId ||
-      (isString(runtimeInfo?.title) ? runtimeInfo.title : undefined);
+  const label = resolveConnectedDeviceLabel(runtimeInfo, { emptyLabel: '' });
 
   if (!label) {
     return null;
@@ -184,10 +237,17 @@ function buildGenericConnectedDeviceItem(
   return {
     id: deviceId || `${platformKey}-connected`,
     label,
-    description: deviceId && deviceId !== label ? deviceId : undefined,
+    description:
+      deviceId && deviceId !== label && !isString(metadata.wdaGatewayId)
+        ? deviceId
+        : undefined,
     selected: true,
     status: 'active',
-    sessionValues: resolveConnectedSessionValues(runtimeInfo, platformKey),
+    sessionValues: resolveConnectedSessionValues(
+      runtimeInfo,
+      platformKey,
+      formValues,
+    ),
   };
 }
 
@@ -214,6 +274,8 @@ export function resolveSelectedDeviceId(
   const selectedPlatform = normalizeStudioPlatformId(formValues.platformId);
 
   if (selectedPlatform === 'ios') {
+    const gatewayId = resolveSelectedIosGatewayId(formValues);
+    if (gatewayId) return gatewayId;
     const host = isString(formValues['ios.host'])
       ? formValues['ios.host']
       : isString(formValues.host)
@@ -300,13 +362,35 @@ function resolveSelectedSessionValues(
           ? { deviceId: formValues.deviceId }
           : undefined;
     case 'ios': {
+      const baseUrl = iosFormValue(formValues, 'baseUrl');
+      const mjpegUrl = iosFormValue(formValues, 'mjpegUrl');
+      const sessionId = iosFormValue(formValues, 'sessionId');
+      const mjpegPort = normalizePort(
+        formValues['ios.mjpegPort'] ?? formValues.mjpegPort,
+      );
+      if (baseUrl) {
+        return {
+          baseUrl,
+          ...(mjpegUrl ? { mjpegUrl } : {}),
+          ...(mjpegPort !== undefined ? { mjpegPort } : {}),
+          ...(sessionId ? { sessionId } : {}),
+        };
+      }
       const host = isString(formValues['ios.host'])
         ? formValues['ios.host']
         : isString(formValues.host)
           ? formValues.host
           : undefined;
       const port = normalizePort(formValues['ios.port'] ?? formValues.port);
-      return host && port !== undefined ? { host, port } : undefined;
+      return host && port !== undefined
+        ? {
+            host,
+            port,
+            ...(mjpegUrl ? { mjpegUrl } : {}),
+            ...(mjpegPort !== undefined ? { mjpegPort } : {}),
+            ...(sessionId ? { sessionId } : {}),
+          }
+        : undefined;
     }
     default:
       return undefined;
@@ -316,8 +400,20 @@ function resolveSelectedSessionValues(
 export function buildDeviceSelectionFormValues(
   platform: StudioSidebarPlatformKey,
   device: Pick<StudioAndroidDeviceItem, 'id' | 'sessionValues'>,
-): Record<string, StudioSessionValue> {
+): Record<string, StudioSessionValue | null> {
   if (device.sessionValues) {
+    if (platform === 'ios') {
+      return {
+        platformId: platform,
+        'ios.host': null,
+        'ios.port': null,
+        'ios.baseUrl': null,
+        'ios.mjpegUrl': null,
+        'ios.mjpegPort': null,
+        'ios.sessionId': null,
+        ...prefixSessionValues(platform, device.sessionValues),
+      };
+    }
     return {
       platformId: platform,
       ...prefixSessionValues(platform, device.sessionValues),
@@ -505,6 +601,7 @@ export function buildStudioSidebarDeviceBuckets({
     const connectedItem = buildGenericConnectedDeviceItem(
       runtimeInfo,
       runtimePlatformKey,
+      formValues,
     );
 
     if (connectedItem) {