500 error from API. Unable to get any prompts to complete. Several steps run, but never completes.

[jstjoe]

Issue / Repro steps

I just cloned the repo, installed requirements, updated Playwright, and tried running a basic prompt (full examples below).

No matter the prompt, it seems to fail on a final step with a 500 error. I suspect an invalid call or issue with one of the tool calls? But with very little information.

Steps: (details in logs below)

• clone this repository
• run setup steps
• run cli command
• see Chrome window open to bing
• enter prompt to interactive shell
• see steps happen
• on what seems to be the last step, error

Browser:

local playwright
version: latest (cloned today)

Here's the API error:

Error: 500 {
  "error": {
    "message": "The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_540c6eaab7f4556b2baa22bcba915d51 in your email.)",
    "type": "server_error",
    "param": null,
    "code": null
  }
}

Question / expected behavior:

Is this indicative of overloading or genuine API-side issues, or something on my end? It's clearly completing some steps but maybe getting tripped up trying to get content or use one of the tools?

Am I doing something wrong?

Full logs

One example:

.venv➜  openai-cua-sample-app git:(main) python cli.py --computer local-playwright
New page created
> visit news.google.com and find recent news stories relating to global politics from the past week
screenshot({})
click({'button': 'left', 'x': 231, 'y': 176})
click({'button': 'left', 'x': 282, 'y': 450})
click({'button': 'left', 'x': 308, 'y': 147})
click({'button': 'left', 'x': 208, 'y': 177})
type({'text': 'news.google.com'})
keypress({'keys': ['ENTER']})
wait({})
click({'button': 'left', 'x': 246, 'y': 301})
wait({})
New page created
click({'button': 'left', 'x': 255, 'y': 301})
click({'button': 'left', 'x': 567, 'y': 93})
New page created
click({'button': 'left', 'x': 576, 'y': 12})
click({'button': 'left', 'x': 562, 'y': 14})
Error: 500 {
  "error": {
    "message": "The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_540c6eaab7f4556b2baa22bcba915d51 in your email.)",
    "type": "server_error",
    "param": null,
    "code": null
  }
}
Page closed
Page closed
Page closed
Warning: All pages have been closed.
Traceback (most recent call last):
  File "/Users/joseph/Code/openai-cua-sample-app/cli.py", line 82, in <module>
    main()
    ~~~~^^
  File "/Users/joseph/Code/openai-cua-sample-app/cli.py", line 71, in main
    output_items = agent.run_full_turn(
        items,
    ...<2 lines>...
        debug=args.debug,
    )
  File "/Users/joseph/Code/openai-cua-sample-app/agent/agent.py", line 138, in run_full_turn
    new_items += response["output"]
                 ~~~~~~~~^^^^^^^^^^
KeyError: 'output'

Another example:

.venv➜  openai-cua-sample-app git:(main) python cli.py --computer local-playwright
New page created
> search for global news this week
screenshot({})
click({'button': 'left', 'x': 202, 'y': 188})
keypress({'keys': ['ENTER']})
wait({})
click({'button': 'left', 'x': 361, 'y': 188})
wait({})
New page created
click({'button': 'left', 'x': 315, 'y': 191})
Error: 500 {
  "error": {
    "message": "The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_b143575ab325bad18c5dfcb20c4a84ca in your email.)",
    "type": "server_error",
    "param": null,
    "code": null
  }
}
Page closed
Page closed
Warning: All pages have been closed.
Traceback (most recent call last):
  File "/Users/joseph/Code/openai-cua-sample-app/cli.py", line 82, in <module>
    main()
    ~~~~^^
  File "/Users/joseph/Code/openai-cua-sample-app/cli.py", line 71, in main
    output_items = agent.run_full_turn(
        items,
    ...<2 lines>...
        debug=args.debug,
    )
  File "/Users/joseph/Code/openai-cua-sample-app/agent/agent.py", line 138, in run_full_turn
    new_items += response["output"]
                 ~~~~~~~~^^^^^^^^^^
KeyError: 'output'

[realadeel]

I am getting the same error even with exponential backoff and many seconds between some requests, well below my tier 4 limits.

[CadeInsights]

i get this a lot as well. can't figure it out. seems to happen randomly.. sometimes my code executes, other times i get this. even with backoffs and retries.