Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions .github/workflows/extended-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@ name: Extended CI
on:
merge_group:
workflow_dispatch:
inputs:
windows_guest:
description: Run the real OpenSandbox Windows guest lifecycle test
required: false
default: false
type: boolean

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down Expand Up @@ -237,6 +243,109 @@ jobs:
if-no-files-found: ignore
retention-days: 14

e2e-opensandbox-windows:
name: E2E (OpenSandbox Windows guest)
if: github.event_name == 'workflow_dispatch' && inputs.windows_guest
runs-on: self-hosted
timeout-minutes: 35
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0

- name: Check Windows guest prerequisites
id: prerequisites
run: |
available=true
for device in /dev/kvm /dev/net/tun; do
if [ ! -r "$device" ] || [ ! -w "$device" ]; then
echo "::notice::Skipping Windows guest e2e: $device is unavailable or not readable/writable."
available=false
fi
done
if ! docker info >/dev/null 2>&1; then
echo "::notice::Skipping Windows guest e2e: Docker is unavailable."
available=false
fi
echo "available=$available" >> "$GITHUB_OUTPUT"

- uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6
if: steps.prerequisites.outputs.available == 'true'
with:
go-version: "1.25.x"
cache: true

- name: Install uv
if: steps.prerequisites.outputs.available == 'true'
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2

- name: Install OpenSandbox server with Windows profile
if: steps.prerequisites.outputs.available == 'true'
run: |
uv tool install 'opensandbox-server==0.2.1'
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

- name: Start OpenSandbox server
if: steps.prerequisites.outputs.available == 'true'
run: |
api_key="ci-$(openssl rand -hex 16)"
config="$RUNNER_TEMP/sandbox-windows.toml"
cat > "$config" <<EOF
[server]
host = "127.0.0.1"
port = 8081
api_key = "$api_key"

[runtime]
type = "docker"
execd_image = "opensandbox/execd:v1.0.21"

[docker]
network_mode = "host"
EOF
log="$RUNNER_TEMP/opensandbox-windows-server.log"
nohup opensandbox-server --config "$config" > "$log" 2>&1 &
echo $! > "$RUNNER_TEMP/opensandbox-windows-server.pid"
for i in $(seq 1 60); do
if curl -fsS http://127.0.0.1:8081/health 2>/dev/null | grep -q healthy; then
break
fi
if [ "$i" -eq 60 ]; then
echo "::error::OpenSandbox server did not become healthy within 120s"
cat "$log"
exit 1
fi
sleep 2
done
{
echo "OPENSANDBOX_API_KEY=$api_key"
echo "OPENSANDBOX_BASE_URL=http://127.0.0.1:8081"
echo "OPENSANDBOX_WINDOWS_IMAGE=dockurr/windows:latest"
} >> "$GITHUB_ENV"

- name: Run real Windows guest lifecycle test
if: steps.prerequisites.outputs.available == 'true'
run: go test -tags e2e -timeout 1800s -count=1 -v -run TestCustomEngine_OpenSandboxWindowsGuest ./e2e
env:
SKILL_UP_WINDOWS_GUEST_E2E: "1"
SKILL_UP_E2E_ARTIFACT_DIR: ${{ github.workspace }}/e2e-opensandbox-windows-artifacts

- name: Upload Windows guest diagnostics
if: always() && steps.prerequisites.outputs.available == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: opensandbox-windows-guest
path: |
${{ runner.temp }}/opensandbox-windows-server.log
e2e-opensandbox-windows-artifacts/
if-no-files-found: ignore
retention-days: 14

- name: Stop OpenSandbox server
if: always() && steps.prerequisites.outputs.available == 'true'
run: |
if [ -f "$RUNNER_TEMP/opensandbox-windows-server.pid" ]; then
kill "$(cat "$RUNNER_TEMP/opensandbox-windows-server.pid")" 2>/dev/null || true
fi

e2e-docker:
name: E2E (docker runtime)
runs-on: self-hosted
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- OpenSandbox Windows guest support through typed `environment.platform` and
optional `environment.resources` settings. The runtime now applies guest-OS
path and shell semantics across workspace setup, transfers, Custom Engine
execution, PowerShell script judging, and artifact collection. Extended CI
includes an explicit opt-in real Windows guest lifecycle test.

## [0.7.0] - 2026-07-16

### Added
Expand Down
50 changes: 35 additions & 15 deletions docs/guide/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ limitations, and the recommended workflow.
- **Build and unit tests** — `go build ./...` and `go test ./...` pass on
Windows. CI exercises a `windows-latest` runner alongside Linux.
- **The `none` runtime** — commands run on the host through `cmd.exe`.
- **The `opensandbox` runtime** — unaffected by the host OS; it always
executes inside a Linux sandbox.
- **The `opensandbox` runtime** — unaffected by the host OS; it can target a
Linux sandbox or an explicitly configured Windows guest.
- **The script judge** — dispatches by file extension (or shebang):

| Script | Interpreter on Windows |
Expand Down Expand Up @@ -42,21 +42,38 @@ generates. Users who want to drive script judges through WSL must arrange
path translation upstream and point `SKILL_UP_BASH` at a non-WSL bash — or
simply run skill-up inside WSL itself (see "Recommended workflow" below).

## OpenSandbox runtime on Windows
## OpenSandbox Windows guests

The `opensandbox` runtime talks to a remote OpenSandbox server over HTTP, so the
skill-up host and guest OS are independent. Configure a Windows guest with
`environment.platform`; when omitted, existing Linux behavior is unchanged.

```yaml
environment:
type: opensandbox
image: dockurr/windows:latest
platform:
os: windows
arch: amd64
resources:
memory: 16Gi
workspace_mount: C:/workspace
ready_timeout_seconds: 1800
```

The `opensandbox` runtime talks to a remote OpenSandbox server over HTTP and
never spawns a host shell. Running `skill-up.exe` on native Windows against a
remote sandbox works today: all host-side path handling already crosses the
host→sandbox boundary through `filepath.ToSlash`, and the sandbox itself is a
Linux container, so the script judge and any agent run inside it behave
exactly as they do on Linux.
Windows defaults are `C:\workspace`, 4 CPU, 8 GiB memory, and 64 GiB disk.
Users may override any resource field without repeating the others. Both Linux
and Windows use the OpenSandbox directory API first and fall back to their
native shell only when the API cannot create or verify a writable directory.
Uploads, downloads, command working directories, script judges, and artifact
collection all use guest OS path semantics even when skill-up runs on another
OS.

OpenSandbox also offers a [**Windows guest profile**](https://github.com/alibaba/OpenSandbox/blob/main/docs/windows-sandbox.md):
the server runs `dockur/windows` (Windows in KVM/QEMU inside a Linux container)
and the API accepts `platform: {"os": "windows", "arch": "amd64"}` on create.
At the time of writing the Go SDK does not yet expose the `Platform` field, so
driving a Windows-guest sandbox from skill-up is blocked on an upstream Go SDK
update — tracked separately.
The OpenSandbox server needs KVM, TUN, sufficient storage, and a Windows-capable
profile; see the upstream
[Windows Sandbox guide](https://github.com/opensandbox-group/OpenSandbox/blob/main/docs/guides/windows-sandbox.md).
Cold boot can take many minutes, so use a long `ready_timeout_seconds` and
persistent server-side storage where appropriate.

For a Windows machine that needs the full agent workflow **without** a remote
sandbox, run skill-up inside **WSL2**. WSL2 is a Linux environment, so both the
Expand Down Expand Up @@ -92,6 +109,9 @@ go test -race ./...
launched through a bash-based Node/nvm bootstrap. That bootstrap does not
run under `cmd.exe`. To run full agent evals on Windows, either install
Node.js and the agent CLIs yourself beforehand, or use WSL2.
- **Built-in Agent CLI bootstrap in a Windows OpenSandbox guest** — the runtime
lifecycle is supported, but automatic installation of built-in Agent CLIs is
not. Preinstall the CLI in the guest image or configure a Custom Engine.
- **`.ps1` script judges require a Windows target** — when the runtime target
is POSIX (for example the `opensandbox` Linux sandbox), only `.sh` scripts
are supported.
Expand Down
33 changes: 33 additions & 0 deletions docs/guide/writing-evals.md
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,39 @@ Common fields:
| `kwargs.request_timeout_seconds` | Request timeout for the OpenSandbox SDK. |
| `kwargs.file_transfer_parallelism` | Concurrency for directory download. |

OpenSandbox can also target a Windows guest independently of the OS running
skill-up:

```yaml
environment:
type: opensandbox
image: dockurr/windows:latest
platform:
os: windows
arch: amd64
resources: # Optional user overrides
cpu: "8"
memory: 16Gi
disk: 128Gi
workspace_mount: C:/workspace
ready_timeout_seconds: 1800
```

`platform.os` accepts `linux` or `windows`; `platform.arch` accepts `amd64` or
`arm64`, and the two fields must be set together. A Windows guest defaults to
`C:\workspace` plus 4 CPU, 8 GiB memory, and 64 GiB disk; partial `resources`
values override those defaults. Linux keeps the SDK defaults unless
`resources` is present, in which case partial values merge over them. An
explicit `workspace_mount` must be absolute for the guest OS (`C:/work` is
accepted and normalized for Windows). The OpenSandbox host must meet the
[Windows profile prerequisites](https://github.com/opensandbox-group/OpenSandbox/blob/main/docs/guides/windows-sandbox.md).

Windows guests execute `setup_steps` through `cmd.exe`, use Windows path rules
for fixture and Skill transfer, support PowerShell (`.ps1`) script judges, and
can run a locally configured Custom Engine. Automatic installation of the
built-in Agent CLIs into a Windows guest is not yet supported; install the CLI
in the image or use a Custom Engine.

#### Docker configuration

When `environment.type: docker` is used, the agent runs inside a local Docker container. This provides container-level isolation (filesystem, process, network) without any remote service dependency.
Expand Down
44 changes: 32 additions & 12 deletions docs/zh/guide/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ skill-up 原生支持 Windows。本页说明哪些功能可用、当前的限制
- **构建与单元测试** —— `go build ./...` 和 `go test ./...` 在 Windows 上通过。
CI 在 Linux 之外额外运行 `windows-latest` runner。
- **`none` runtime** —— 命令通过 `cmd.exe` 在宿主机上执行。
- **`opensandbox` runtime** —— 不受宿主机 OS 影响,始终在 Linux 沙箱内执行。
- **`opensandbox` runtime** —— 不受宿主机 OS 影响,可选择 Linux sandbox,
也可显式配置 Windows guest。
- **script judge** —— 按文件扩展名(或 shebang)分派解释器:

| 脚本 | Windows 上的解释器 |
Expand Down Expand Up @@ -37,19 +38,35 @@ skill-up 原生支持 Windows。本页说明哪些功能可用、当前的限制
需要走 WSL 的用户请自行处理路径翻译并把 `SKILL_UP_BASH` 指向非 WSL 的 bash,
或者直接在 WSL 内运行 skill-up(见下文「推荐工作流」)。

## Windows 上的 OpenSandbox runtime
## OpenSandbox Windows guest

`opensandbox` runtime 通过 HTTP 与远程 OpenSandbox 服务通信,因此 skill-up
宿主机与 guest OS 相互独立。通过 `environment.platform` 选择 Windows;省略时
保持原有 Linux 行为。

```yaml
environment:
type: opensandbox
image: dockurr/windows:latest
platform:
os: windows
arch: amd64
resources:
memory: 16Gi
workspace_mount: C:/workspace
ready_timeout_seconds: 1800
```

`opensandbox` runtime 通过 HTTP 与远程 OpenSandbox 服务器通信,不会启动任何
宿主机 shell。在原生 Windows 上运行 `skill-up.exe` 连接远程 sandbox 当前即可
工作:所有宿主机侧的路径处理都已通过 `filepath.ToSlash` 跨越「宿主机→sandbox」
边界,而 sandbox 本身是 Linux 容器,因此其中的 script judge 和 agent 行为与在
Linux 上完全一致
Windows 默认使用 `C:\workspace`、4 CPU、8 GiB 内存和 64 GiB 磁盘;用户可只覆盖
需要调整的资源字段。Linux 与 Windows 都优先调用 OpenSandbox 目录 API,只有
API 无法创建或验证可写目录时才回退到各自的原生 shell。上传、下载、命令工作
目录、script judge 和 artifact 收集均按 guest OS 的路径规则处理,即使 skill-up
运行在另一种 OS 上也一致

OpenSandbox 也提供 [**Windows guest profile**](https://github.com/alibaba/OpenSandbox/blob/main/docs/windows-sandbox.md):
服务端在 Linux 容器里通过 KVM/QEMU 运行 `dockur/windows`,创建 API 接受
`platform: {"os": "windows", "arch": "amd64"}`。撰写本文时 Go SDK 尚未暴露
`Platform` 字段,因此从 skill-up 驱动 Windows guest sandbox 依赖上游 Go SDK
补齐 —— 单独跟进。
OpenSandbox 服务端需要 KVM、TUN、足够的存储空间以及支持 Windows 的 profile;
详见上游 [Windows Sandbox 指南](https://github.com/opensandbox-group/OpenSandbox/blob/main/docs/guides/windows-sandbox.md)。
Windows 冷启动可能耗时数分钟,应配置较长的 `ready_timeout_seconds`,必要时在
服务端启用持久存储。

如果某台 Windows 机器需要在**没有**远程 sandbox 的情况下使用完整的 agent
工作流,请在 **WSL2** 中运行 skill-up。WSL2 是 Linux 环境,因此 `none` 与
Expand Down Expand Up @@ -82,6 +99,9 @@ go test -race ./...
- **原生运行真实 agent** —— Claude Code / Codex / Qoder CLI 通过基于 bash 的
Node/nvm 引导脚本启动,该脚本无法在 `cmd.exe` 下运行。要在 Windows 上运行
完整的 agent 评测,请预先自行安装 Node.js 和对应的 agent CLI,或使用 WSL2。
- **Windows OpenSandbox guest 中的内置 Agent CLI 引导** —— runtime 生命周期
已支持,但尚不会自动安装内置 Agent CLI。请将 CLI 预装进 guest 镜像,或配置
Custom Engine。
- **`.ps1` script judge 需要 Windows 目标** —— 当 runtime 目标是 POSIX
(例如 `opensandbox` 的 Linux 沙箱)时,仅支持 `.sh` 脚本。
- **`cmd.exe` 会展开参数里的 `%VAR%`** —— 当宿主未发现 bash、回退到
Expand Down
30 changes: 30 additions & 0 deletions docs/zh/guide/writing-evals.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,36 @@ environment:
| `kwargs.request_timeout_seconds` | OpenSandbox SDK 请求超时时间 |
| `kwargs.file_transfer_parallelism` | 目录下载并发度 |

OpenSandbox 还可以在不依赖 skill-up 宿主 OS 的情况下选择 Windows guest:

```yaml
environment:
type: opensandbox
image: dockurr/windows:latest
platform:
os: windows
arch: amd64
resources: # 可选的用户覆盖
cpu: "8"
memory: 16Gi
disk: 128Gi
workspace_mount: C:/workspace
ready_timeout_seconds: 1800
```

`platform.os` 支持 `linux`、`windows`,`platform.arch` 支持 `amd64`、`arm64`,
且两个字段必须同时配置。Windows guest 默认使用 `C:\workspace`、4 CPU、8 GiB
内存和 64 GiB 磁盘,`resources` 可按字段覆盖这些默认值。Linux 在没有
`resources` 时保持 SDK 默认值;提供部分字段时则在 SDK 默认值上覆盖。显式的
`workspace_mount` 必须是 guest OS 的绝对路径(Windows 可写成 `C:/work`,运行时
会规范化)。OpenSandbox 宿主机还需要满足
[Windows profile 前置条件](https://github.com/opensandbox-group/OpenSandbox/blob/main/docs/guides/windows-sandbox.md)。

Windows guest 的 `setup_steps` 通过 `cmd.exe` 执行;fixture 与 Skill 上传遵循
Windows 路径规则;script judge 支持 PowerShell(`.ps1`);本地 Custom Engine
也可以完成整条评测链路。当前尚不自动向 Windows guest 安装内置 Agent CLI;
请预装到镜像中,或使用 Custom Engine。

#### Docker 配置

使用 `environment.type: docker` 时,Agent 在本地 Docker 容器中运行,提供容器级别的隔离(文件系统、进程、网络),无需任何远程服务。
Expand Down
Loading
Loading