[Bug] 400 Invalid schema for response_format

[SkyYap]

Have gone through the connectivity-test by implementing .env file on my folder. Still face this issue.

• Where are you using Midscene.js (Chrome extension, yaml with cli, Puppeteer,…)
  I am using Midscene.js with yaml with cli. I run C:\Users\(username)\Documents\GitHub\midscene-example\yaml-scripts-demo> midscene .\midscene-scripts\search-headphone-on-ebay.yaml --headed

Run npm run test on yaml-scripts-demo and all these 3 failed (sauce-demo.yaml, search-headphone-on-ebay.yaml and local-static-server.yml) but the extract-github-status.yaml work successfully and produce an output in json correctly.

• The version of Midscene.js or Extension
  Midscene chrome extension Version 0.26,
  @midscene/cli": "latest"

• The error message
  400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'additionalProperties' is required to be supplied and to be false. Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'additionalProperties' is required to be supplied and to be false.

• The model name and endpoint (if could be public）
  Using openai model.
  OPENAI_API_KEY="sk-project"

[DavidQL]

I am also getting this error as of last night when running mid.aiAction calls on @midscene/web@0.10.3.

[DavidQL]

ai-unit-test.yml fails in my fork with this same OpenAI error message.

[rublev]

same

[zhoushaw]

I'm unsure whether I fixed the issue in the new version, as I haven't encountered it myself. Could you please test it out?
https://github.com/web-infra-dev/midscene/releases/tag/v0.10.4 @DavidQL @rublev @SkyYap

[lmatello]

I still have the same problem.
Im using the demo project.

`> playwright-demo@0.0.1 e2e

playwright test --config=playwright.config.ts --headed

Running 4 tests using 4 workers
[chromium] › ebay-search-en.spec.ts:10:5 › search headphone on ebay

1. [chromium] › ebay-search-en.spec.ts:10:5 › search headphone on ebay ───────────────────────────

Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'additionalProperties' is required to be supplied and to be false.
Error: 400 Invalid schema for response_format 'action_items': In context=('properties', 'actions', 'items', 'properties', 'param', 'type', '0'), 'additionalProperties' is required to be supplied and to be false.

    at Function.generate (/Users/lmatello/Downloads/midscene-example-main/playwright-demo/node_modules/openai/src/error.ts:72:14)
    at OpenAI.makeStatusError (/Users/lmatello/Downloads/midscene-example-main/playwright-demo/node_modules/openai/src/core.ts:435:21)
    at OpenAI.makeRequest (/Users/lmatello/Downloads/midscene-example-main/playwright-demo/node_modules/openai/src/core.ts:499:24)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)`

My package-lock:

"devDependencies": { "@midscene/web": "latest", "@playwright/test": "1.48.0", "@types/jest": "~29.5.14", "@types/node": "~22.7.9", "dotenv": "16.4.5", "eslint-plugin-prettier": "~5.2.1", "rimraf": "~6.0.1", "typescript": "~5.6.3" }

[zhoushaw]

@lmatello Could you send the report from your execution directory? I haven't been able to replicate the issue on my end, and I need to review what the AI actually returned. The report won't include your token information.

[zhoushaw]

I suspect the issue might be due to an anomaly in the model's data. You could manually set the model's version to gpt-4o-2024-11-20.

[DavidQL]

Still failing for me with same error message on @midscene/web@0.10.4. Here is the JSON from my midscene_run/report/web-2025-01-30_09-55-38-143.html.

[lmatello]

Sure @zhoushaw
I send you the report!

playwright-merged-2025-01-30_08-28-04-835.html.zip

[lmatello]

@zhoushaw Same issue with this:
MIDSCENE_MODEL_NAME='gpt-4o-2024-11-20'

[zhoushaw]

https://github.com/web-infra-dev/midscene/releases/tag/v0.10.5 @lmatello @DavidQL cc

[lmatello]

@zhoushaw Now its working!
Thanks so much! 💪