Skip to content

AI unit test

AI unit test #13

Manually triggered January 31, 2025 01:23
Status Failure
Total duration 7m 27s
Artifacts 1

ai-unit-test.yml

on: workflow_dispatch
Matrix: main
Fit to window
Zoom out
Zoom in

Annotations

11 errors
main (18.19.0)
Process completed with exit code 1.
tests/ai/web/puppeteer/showcase.test.ts > puppeteer integration > Sauce Demo by Swag Lab: packages/web-integration/src/common/tasks.ts#L519
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101295516252592278691151) Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101295516252592278691151) at Function.generate (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14) at OpenAI.makeStatusError (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21) at OpenAI.makeRequest (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24) at processTicksAndRejections (node:internal/process/task_queues:95:5) at call (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2142:20) at callToGetJSONObject (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2216:20) at callAiFn (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:1102:30) at Module.plan (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2518:30) at Object.executor (/home/runner/work/midscene/midscene/packages/web-integration/src/common/tasks.ts:519:24) at Executor.flush (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/index.js:119:25) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ Object.executor src/common/tasks.ts:519:24 ❯ PuppeteerAgent.aiAction src/common/agent.ts:185:15 ❯ tests/ai/web/puppeteer/showcase.test.ts:28:7
tests/ai/web/puppeteer/showcase.test.ts > puppeteer integration > search engine: packages/web-integration/src/common/tasks.ts#L519
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101301614433487146120264) Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101301614433487146120264) at Function.generate (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14) at OpenAI.makeStatusError (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21) at OpenAI.makeRequest (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24) at processTicksAndRejections (node:internal/process/task_queues:95:5) at call (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2142:20) at callToGetJSONObject (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2216:20) at callAiFn (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:1102:30) at Module.plan (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2518:30) at Object.executor (/home/runner/work/midscene/midscene/packages/web-integration/src/common/tasks.ts:519:24) at Executor.flush (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/index.js:119:25) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ Object.executor src/common/tasks.ts:519:24 ❯ PuppeteerAgent.aiAction src/common/agent.ts:185:15 ❯ it.timeout tests/ai/web/puppeteer/showcase.test.ts:102:9
tests/ai/web/puppeteer/showcase.test.ts > puppeteer integration > scroll: packages/web-integration/src/common/tasks.ts#L519
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101302410587303936518459) Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101302410587303936518459) at Function.generate (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14) at OpenAI.makeStatusError (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21) at OpenAI.makeRequest (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24) at processTicksAndRejections (node:internal/process/task_queues:95:5) at call (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2142:20) at callToGetJSONObject (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2216:20) at callAiFn (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:1102:30) at Module.plan (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2518:30) at Object.executor (/home/runner/work/midscene/midscene/packages/web-integration/src/common/tasks.ts:519:24) at Executor.flush (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/index.js:119:25) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ Object.executor src/common/tasks.ts:519:24 ❯ PuppeteerAgent.aiAction src/common/agent.ts:185:15 ❯ tests/ai/web/puppeteer/showcase.test.ts:121:7
tests/ai/web/puppeteer/showcase.test.ts > puppeteer integration > not tracking active tab: packages/web-integration/src/common/tasks.ts#L519
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101303482391462526827167) Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101303482391462526827167) at Function.generate (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14) at OpenAI.makeStatusError (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21) at OpenAI.makeRequest (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24) at processTicksAndRejections (node:internal/process/task_queues:95:5) at call (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2142:20) at callToGetJSONObject (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2216:20) at callAiFn (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:1102:30) at Module.plan (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2518:30) at Object.executor (/home/runner/work/midscene/midscene/packages/web-integration/src/common/tasks.ts:519:24) at Executor.flush (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/index.js:119:25) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ Object.executor src/common/tasks.ts:519:24 ❯ PuppeteerAgent.aiAction src/common/agent.ts:185:15 ❯ tests/ai/web/puppeteer/showcase.test.ts:135:7
tests/ai/web/puppeteer/showcase.test.ts > puppeteer integration > tracking active tab: packages/web-integration/src/common/tasks.ts#L519
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101304774137436303815045) Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101304774137436303815045) at Function.generate (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14) at OpenAI.makeStatusError (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21) at OpenAI.makeRequest (/home/runner/work/midscene/midscene/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24) at processTicksAndRejections (node:internal/process/task_queues:95:5) at call (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2142:20) at callToGetJSONObject (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2216:20) at callAiFn (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:1102:30) at Module.plan (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/chunk-FKK23XFF.js:2518:30) at Object.executor (/home/runner/work/midscene/midscene/packages/web-integration/src/common/tasks.ts:519:24) at Executor.flush (/home/runner/work/midscene/midscene/packages/midscene/dist/lib/index.js:119:25) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ Object.executor src/common/tasks.ts:519:24 ❯ PuppeteerAgent.aiAction src/common/agent.ts:185:15 ❯ tests/ai/web/puppeteer/showcase.test.ts:149:7
tests/ai/evaluate/plan/planning-input.test.ts > automation - planning input > input value: packages/midscene/src/ai-model/service-caller/index.ts#L204
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101301236994649652363773) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ call src/ai-model/service-caller/index.ts:204:20 ❯ Module.callToGetJSONObject src/ai-model/service-caller/index.ts:299:20 ❯ callAiFn src/ai-model/common.ts:31:30 ❯ Module.plan src/ai-model/llm-planning.ts:58:30 ❯ tests/ai/evaluate/plan/planning-input.test.ts:22:29 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { status: 400, headers: { connection: 'keep-alive', 'content-length': '385', 'content-type': 'application/json; charset=utf-8', date: 'Fri, 31 Jan 2025 01:30:17 GMT', 'x-aihubmix-request-id': '2025013101301236994649652363773' }, request_id: undefined, error: { message: 'Invalid schema for response_format \'action_items\': In context=(\'properties\', \'actions\', \'items\', \'properties\', \'param\', \'type\', \'0\'), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'direction\'. (request id: 2025013101301236994649652363773)', type: 'invalid_request_error', param: 'response_format', code: null }, code: null, param: 'response_format' }
tests/ai/evaluate/plan/planning-input.test.ts > automation - planning input > input value Add, delete, correct and check: packages/midscene/src/ai-model/service-caller/index.ts#L204
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101301712248727382801914) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ call src/ai-model/service-caller/index.ts:204:20 ❯ Module.callToGetJSONObject src/ai-model/service-caller/index.ts:299:20 ❯ callAiFn src/ai-model/common.ts:31:30 ❯ Module.plan src/ai-model/llm-planning.ts:58:30 ❯ tests/ai/evaluate/plan/planning-input.test.ts:39:29 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { status: 400, headers: { connection: 'keep-alive', 'content-length': '385', 'content-type': 'application/json; charset=utf-8', date: 'Fri, 31 Jan 2025 01:30:20 GMT', 'x-aihubmix-request-id': '2025013101301712248727382801914' }, request_id: undefined, error: { message: 'Invalid schema for response_format \'action_items\': In context=(\'properties\', \'actions\', \'items\', \'properties\', \'param\', \'type\', \'0\'), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'direction\'. (request id: 2025013101301712248727382801914)', type: 'invalid_request_error', param: 'response_format', code: null }, code: null, param: 'response_format' }
tests/ai/evaluate/plan/planning.test.ts > automation - planning > basic run: packages/midscene/src/ai-model/service-caller/index.ts#L204
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101295435521160264406258) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ call src/ai-model/service-caller/index.ts:204:20 ❯ Module.callToGetJSONObject src/ai-model/service-caller/index.ts:299:20 ❯ callAiFn src/ai-model/common.ts:31:30 ❯ Module.plan src/ai-model/llm-planning.ts:58:30 ❯ tests/ai/evaluate/plan/planning.test.ts:15:25 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { status: 400, headers: { connection: 'keep-alive', 'content-length': '385', 'content-type': 'application/json; charset=utf-8', date: 'Fri, 31 Jan 2025 01:29:58 GMT', 'x-aihubmix-request-id': '2025013101295435521160264406258' }, request_id: undefined, error: { message: 'Invalid schema for response_format \'action_items\': In context=(\'properties\', \'actions\', \'items\', \'properties\', \'param\', \'type\', \'0\'), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'direction\'. (request id: 2025013101295435521160264406258)', type: 'invalid_request_error', param: 'response_format', code: null }, code: null, param: 'response_format' }
tests/ai/evaluate/plan/planning.test.ts > automation - planning > instructions of to-do mvc: packages/midscene/src/ai-model/service-caller/index.ts#L204
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101295832910162273416479) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ call src/ai-model/service-caller/index.ts:204:20 ❯ Module.callToGetJSONObject src/ai-model/service-caller/index.ts:299:20 ❯ callAiFn src/ai-model/common.ts:31:30 ❯ Module.plan src/ai-model/llm-planning.ts:58:30 ❯ tests/ai/evaluate/plan/planning.test.ts:42:27 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { status: 400, headers: { connection: 'keep-alive', 'content-length': '385', 'content-type': 'application/json; charset=utf-8', date: 'Fri, 31 Jan 2025 01:30:02 GMT', 'x-aihubmix-request-id': '2025013101295832910162273416479' }, request_id: undefined, error: { message: 'Invalid schema for response_format \'action_items\': In context=(\'properties\', \'actions\', \'items\', \'properties\', \'param\', \'type\', \'0\'), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'direction\'. (request id: 2025013101295832910162273416479)', type: 'invalid_request_error', param: 'response_format', code: null }, code: null, param: 'response_format' }
tests/ai/evaluate/plan/planning.test.ts > automation - planning > scroll some element: packages/midscene/src/ai-model/service-caller/index.ts#L204
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'required' is required to be supplied and to be an array including every key in properties. Missing 'direction'. (request id: 2025013101300233039810468544680) ❯ Function.generate ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/error.ts:72:14 ❯ OpenAI.makeStatusError ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:435:21 ❯ OpenAI.makeRequest ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/openai/src/core.ts:499:24 ❯ call src/ai-model/service-caller/index.ts:204:20 ❯ Module.callToGetJSONObject src/ai-model/service-caller/index.ts:299:20 ❯ callAiFn src/ai-model/common.ts:31:30 ❯ Module.plan src/ai-model/llm-planning.ts:58:30 ❯ tests/ai/evaluate/plan/planning.test.ts:50:25 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Serialized Error: { status: 400, headers: { connection: 'keep-alive', 'content-length': '385', 'content-type': 'application/json; charset=utf-8', date: 'Fri, 31 Jan 2025 01:30:06 GMT', 'x-aihubmix-request-id': '2025013101300233039810468544680' }, request_id: undefined, error: { message: 'Invalid schema for response_format \'action_items\': In context=(\'properties\', \'actions\', \'items\', \'properties\', \'param\', \'type\', \'0\'), \'required\' is required to be supplied and to be an array including every key in properties. Missing \'direction\'. (request id: 2025013101300233039810468544680)', type: 'invalid_request_error', param: 'response_format', code: null }, code: null, param: 'response_format' }

Artifacts

Produced during runtime
Name Size
test-ai-output Expired
12.5 MB