Skip to content

fix(amr): improve unavailable model gating #14650

fix(amr): improve unavailable model gating

fix(amr): improve unavailable model gating #14650

Triggered via pull request July 9, 2026 07:50
Status Cancelled
Total duration 9m 50s
Artifacts 4

ci.yml

on: pull_request
Resolve runner profiles
10s
Resolve runner profiles
Detect validation scopes
13s
Detect validation scopes
Static gate
12s
Static gate
Matrix: playwright_critical
Matrix: playwright_visual
Matrix: ui_p0
Validate workspace
7s
Validate workspace
Runtime summary
7s
Runtime summary
Fit to window
Zoom out
Zoom in

Annotations

28 errors, 3 warnings, and 2 notices
E2E Vitest
Process completed with exit code 1.
tests/amr/turn.test.ts > AMR chat-run end-to-end > drives /api/runs against vela ACP and the assistant message captures the fake stream: e2e/lib/vitest/runs.ts#L75
Error: run 7da7bf66-8ace-488d-8739-01dcd8cb15bd reached failed, expected succeeded ❯ waitForRunStatus lib/vitest/runs.ts:75:13 ❯ tests/amr/turn.test.ts:299:27 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/turn.test.ts:204:5
tests/amr/relogin-required.test.ts > AMR relogin-required run failures > uses daemon AMR runtime credentials for pre-run login status: e2e/lib/vitest/runs.ts#L75
Error: run bab40fd4-f52f-45f8-a9ec-ea4e83aa73c3 reached failed, expected succeeded ❯ waitForRunStatus lib/vitest/runs.ts:75:13 ❯ tests/amr/relogin-required.test.ts:137:11 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/relogin-required.test.ts:107:9 ❯ Object.withEnv [as env] lib/vitest/suite.ts:202:12 ❯ tests/amr/relogin-required.test.ts:100:5
tests/amr/relogin-required.test.ts > AMR relogin-required run failures > uses configured AMR profile env for pre-run login status: e2e/lib/vitest/runs.ts#L75
Error: run 5ba0b586-eb19-465a-a415-8391b313c4fc reached failed, expected succeeded ❯ waitForRunStatus lib/vitest/runs.ts:75:13 ❯ tests/amr/relogin-required.test.ts:91:9 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/relogin-required.test.ts:62:7 ❯ Object.withEnv [as env] lib/vitest/suite.ts:202:12 ❯ tests/amr/relogin-required.test.ts:60:5
tests/amr/logout-state-persistence.test.ts > AMR logout state persistence > a previously working AMR session stops working after local logout and requires re-login: e2e/lib/vitest/runs.ts#L75
Error: run 3ac7bda2-da67-41cb-91ba-240e3a344ae0 reached failed, expected succeeded ❯ waitForRunStatus lib/vitest/runs.ts:75:13 ❯ tests/amr/logout-state-persistence.test.ts:55:9 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/logout-state-persistence.test.ts:19:7 ❯ Object.withEnv [as env] lib/vitest/suite.ts:202:12 ❯ tests/amr/logout-state-persistence.test.ts:18:5
tests/amr/insufficient-balance.test.ts > AMR insufficient balance run failures > fails the run with a recharge-facing AMR error when fake vela reports insufficient balance: e2e/tests/amr/insufficient-balance.test.ts#L53
AssertionError: expected 'id: 1\nevent: pipeline_stage_started\…' to contain 'AMR_INSUFFICIENT_BALANCE' - Expected + Received - AMR_INSUFFICIENT_BALANCE + id: 1 + event: pipeline_stage_started + data: {"kind":"pipeline_stage_started","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"plan","iteration":0,"startedAt":1783583944102} + + id: 2 + event: pipeline_stage_completed + data: {"kind":"pipeline_stage_completed","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"plan","iteration":0,"completedAt":1783583944106,"converged":true} + + id: 3 + event: pipeline_stage_started + data: {"kind":"pipeline_stage_started","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"generate","iteration":0,"startedAt":1783583944106} + + id: 4 + event: pipeline_stage_completed + data: {"kind":"pipeline_stage_completed","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"generate","iteration":0,"completedAt":1783583944106,"converged":true} + + id: 5 + event: pipeline_stage_started + data: {"kind":"pipeline_stage_started","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"critique","iteration":0,"startedAt":1783583944107} + + id: 6 + event: pipeline_stage_completed + data: {"kind":"pipeline_stage_completed","runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","snapshotId":"2c1b0711-cedd-4e71-9c5c-9bd3b0b4fcb7","stageId":"critique","iteration":0,"completedAt":1783583944107,"converged":true} + + id: 7 + event: start + data: {"runId":"fd9bbe76-8320-41a8-83d3-d005c9d63875","agentId":"amr","bin":"amr","streamFormat":"acp-json-rpc","projectId":"d1802a64-9b82-4d90-acaa-fcc3204b9591","cwd":"/home/runner/_work/open-design/open-design/.tmp/e2e/e2e-amr-insufficient-balance-1783583941255-1gyyvh/scratch/data/projects/d1802a64-9b82-4d90-acaa-fcc3204b9591","model":"default","reasoning":null,"toolTokenExpiresAt":"2026-07-09T08:14:04.111Z"} + + id: 8 + event: agent + data: {"type":"status","label":"model","model":"glm-5"} + + id: 9 + event: agent + data: {"type":"status","label":"waiting_for_first_output","elapsedMs":16} + + id: 10 + event: error + data: {"message":"json-rpc id 3: session/set_model must be called before session/prompt"} + + id: 11 + event: run_retry_finished + data: {"page_name":"chat_panel","area":"chat_panel","project_id":"d1802a64-9b82-4d90-acaa-fcc3204b9591","conversation_id":"7e81d19f-110e-44bb-b53d-058f675bfd05","run_id":"fd9bbe76-8320-41a8-83d3-d005c9d63875","retry_of_run_id":"fd9bbe76-8320-41a8-83d3-d005c9d63875","retry_attempt_index":1,"retry_max_attempts":1,"retry_strategy":"same_run_transient","agent_provider_id":"amr","model_id":"default","failure_category":"process_exit","failure_detail":"terminated_unknown","failure_stage":"child_close","error_code":"AGENT_SIGNAL_SIGTERM","retry_result":"suppressed","retry_suppressed_reason":"non_retryable_category"} + + id: 12 + event: diagnostic + data: {"type":"runtime_close","rpc_close_reason":"fatal_rpc_error","status":"failed","exit_code":1,"signal":"SIGTERM"} + + id: 13 + event: end + data: {"code":1,"signal":"SIGTERM","status":"failed","resumable":false} + + ❯ tests/amr/insufficient-balance.test.ts:53:22 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/insufficient-balance.test.ts:17:5
tests/amr/auth-error-convergence.test.ts > AMR auth error convergence > marks the run and assistant message as failed when fake vela returns an auth error during prompt: e2e/tests/amr/auth-error-convergence.test.ts#L56
AssertionError: expected 'id: 1\nevent: pipeline_stage_started\…' to match /AMR_AUTH_REQUIRED/ - Expected: /AMR_AUTH_REQUIRED/ + Received: "id: 1 event: pipeline_stage_started data: {\"kind\":\"pipeline_stage_started\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"plan\",\"iteration\":0,\"startedAt\":1783583947624} id: 2 event: pipeline_stage_completed data: {\"kind\":\"pipeline_stage_completed\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"plan\",\"iteration\":0,\"completedAt\":1783583947627,\"converged\":true} id: 3 event: pipeline_stage_started data: {\"kind\":\"pipeline_stage_started\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"generate\",\"iteration\":0,\"startedAt\":1783583947627} id: 4 event: pipeline_stage_completed data: {\"kind\":\"pipeline_stage_completed\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"generate\",\"iteration\":0,\"completedAt\":1783583947627,\"converged\":true} id: 5 event: pipeline_stage_started data: {\"kind\":\"pipeline_stage_started\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"critique\",\"iteration\":0,\"startedAt\":1783583947627} id: 6 event: pipeline_stage_completed data: {\"kind\":\"pipeline_stage_completed\",\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"snapshotId\":\"92a635a5-a9fa-419c-b22b-4c0d9b1d16ec\",\"stageId\":\"critique\",\"iteration\":0,\"completedAt\":1783583947628,\"converged\":true} id: 7 event: start data: {\"runId\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"agentId\":\"amr\",\"bin\":\"amr\",\"streamFormat\":\"acp-json-rpc\",\"projectId\":\"9007a7ac-e88d-41c0-8bce-0cb714ff1b6e\",\"cwd\":\"/home/runner/_work/open-design/open-design/.tmp/e2e/e2e-amr-auth-error-convergence-1783583944882-kl5n25/scratch/data/projects/9007a7ac-e88d-41c0-8bce-0cb714ff1b6e\",\"model\":\"default\",\"reasoning\":null,\"toolTokenExpiresAt\":\"2026-07-09T08:14:07.631Z\"} id: 8 event: agent data: {\"type\":\"status\",\"label\":\"model\",\"model\":\"glm-5\"} id: 9 event: agent data: {\"type\":\"status\",\"label\":\"waiting_for_first_output\",\"elapsedMs\":16} id: 10 event: error data: {\"message\":\"json-rpc id 3: session/set_model must be called before session/prompt\"} id: 11 event: run_retry_finished data: {\"page_name\":\"chat_panel\",\"area\":\"chat_panel\",\"project_id\":\"9007a7ac-e88d-41c0-8bce-0cb714ff1b6e\",\"conversation_id\":\"0520bd61-d5a2-4c3f-9f8f-4aaefddb7ec2\",\"run_id\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"retry_of_run_id\":\"fef7a1ab-eec5-4abd-a32e-8478ea3483d3\",\"retry_attempt_index\":1,\"retry_max_attempts\":1,\"retry_strategy\":\"same_run_transient\",\"agent_provider_id\":\"amr\",\"model_id\":\"default\",\"failure_category\":\"process_exit\",\"failure_detail\":\"terminated_unknown\",\"failure_stage\":\"child_close\",\"error_code\":\"AGENT_SIGNAL_SIGTERM\",\"retry_result\":\"suppressed\",\"retry_suppressed_reason\":\"non_retryable_category\"} id: 12 event: diagnostic data: {\"type\":\"runtime_close\",\"rpc_close_reason\":\"fatal_rpc_error\",\"status\":\"failed\",\"exit_code\":1,\"signal\":\"SIGTERM\"} id: 13 event: end data: {\"code\":1,\"signal\":\"SIGTERM\",\"status\":\"failed\",\"resumable\":false} " ❯ tests/amr/auth-error-convergence.test.ts:56:52 ❯ runToolsDevSuite lib/vitest/suite.ts:265:5 ❯ tests/amr/auth-error-convergence.test.ts:17:5
Web workspace tests
Canceling since a higher priority waiting request for ci-5335 exists
Docker image build
Canceling since a higher priority waiting request for ci-5335 exists
Docker image build
The operation was canceled.
Validate Nix flake
Canceling since a higher priority waiting request for ci-5335 exists
Validate Nix flake
The operation was canceled.
UI P0 (workspace-restoration)
Canceling since a higher priority waiting request for ci-5335 exists
UI P0 (workspace-restoration)
The operation was canceled.
UI P0 smoke
Canceling since a higher priority waiting request for ci-5335 exists
UI P0 smoke
The operation was canceled.
UI P0 (project-runtime)
Canceling since a higher priority waiting request for ci-5335 exists
UI P0 (project-runtime)
The operation was canceled.
UI P0 (project-workspace)
Canceling since a higher priority waiting request for ci-5335 exists
UI P0 (project-workspace)
The operation was canceled.
UI P0 (entry-settings)
Canceling since a higher priority waiting request for ci-5335 exists
UI P0 (entry-settings)
The operation was canceled.
Playwright visual (entry-navigation)
Canceling since a higher priority waiting request for ci-5335 exists
Playwright visual (entry-navigation)
The operation was canceled.
ui/visual-entry.test.ts:48:1 β€Ί [P2] captures the visual home harness: e2e/lib/playwright/visual.ts#L578
1) ui/visual-entry.test.ts:48:1 β€Ί [P2] captures the visual home harness ────────────────────────── Error: locator.waitFor: Test timeout of 60000ms exceeded. Call log: - waiting for getByText('Loading Open Design…') to be hidden 10 Γ— locator resolved to visible <div class="od-loading-shell">Loading Open Design…</div> at ../lib/playwright/visual.ts:578 576 | 577 | export async function waitForVisualReady(page: Page): Promise<void> { > 578 | await page.getByText('Loading Open Design…').waitFor({ state: 'hidden', timeout: T.long }); | ^ 579 | await expect(page.getByTestId('home-hero')).toBeVisible({ timeout: T.medium }); 580 | await expect(page.getByTestId('home-hero-input')).toBeVisible({ timeout: T.medium }); 581 | await page.evaluate(async () => { at waitForVisualReady (/home/runner/_work/open-design/open-design/e2e/lib/playwright/visual.ts:578:48) at gotoVisualHome (/home/runner/_work/open-design/open-design/e2e/lib/playwright/visual.ts:597:9) at /home/runner/_work/open-design/open-design/e2e/ui/visual-entry.test.ts:50:3
ui/visual-entry.test.ts:48:1 β€Ί [P2] captures the visual home harness: e2e/ui/visual-entry.test.ts#L0
1) ui/visual-entry.test.ts:48:1 β€Ί [P2] captures the visual home harness ────────────────────────── Test timeout of 60000ms exceeded.
Validate workspace
Process completed with exit code 1.
ci
Canceling since a higher priority waiting request for ci-5335 exists
Detect validation scopes
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Docker image build
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: docker/build-push-action@v6, docker/metadata-action@v5, docker/setup-buildx-action@v3. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
Validate Nix flake
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/setup-node@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
🎭 Playwright Run Summary
19 passed (1.7m)
🎭 Playwright Run Summary
3 passed (1.2m)

Artifacts

Produced during runtime
Name Size Digest
handoff-report-visual-pr Expired
439 Bytes
sha256:abac93f6f1c7ed2ee9c2cf5cd22355f3bc3efe4ab805c01d2e9ad89ad1b19513
ui-p0-ci-29002806723-smoke Expired
269 KB
sha256:3939f84a535c918eadbf0181c1be651ec7b10185c8d1a5215bd2715de9441aae
visual-pr-capture-5335-29002806723-entry-navigation Expired
613 KB
sha256:e8fdbc6f53e702969beaabe00fe0a36fcd3604838e5958f45d25055cba33dfa1
visual-pr-capture-5335-29002806723-settings-workspace Expired
2.87 MB
sha256:4cb60f6b0f4d45a96bfe063cd6984a61b7a1de143d07f98c62aa590a1725b9dd