fix(llm): surface the response body when a completion has no choices#36
Open
wmeddie wants to merge 2 commits into
Open
fix(llm): surface the response body when a completion has no choices#36wmeddie wants to merge 2 commits into
wmeddie wants to merge 2 commits into
Conversation
Some OpenAI-compatible gateways return upstream error bodies (rate limits, provider failures) with HTTP 200. The SDK constructs those leniently, leaving choices None/absent, and response.choices[0] then raised a bare "'NoneType' object is not subscriptable" that hid the actual upstream error. Raise a RuntimeError carrying the body instead, so callers and logs see the real reason.
json.loads in the tool-call path was strict, so two recoverable cases
dropped tool calls with only a log line to show for it:
- raw control characters (literal newlines) inside JSON string values —
models emit these routinely and strict=False parses them fine
- empty arguments from parameterless tool calls, which must become {}
instead of being skipped
Truly irrecoverable arguments (garbage, non-dict JSON) keep the existing
skip-with-warning behavior.
treo
approved these changes
Jun 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two resilience fixes in the OpenAI adapter's response handling, both observed with OpenAI-compatible gateways in production use.
1. A completion without
choicesnow raises a useful errorSome OpenAI-compatible gateways return upstream error bodies (rate limits, provider failures) with HTTP 200. The SDK constructs those leniently, so
choicesends upNone/absent andresponse.choices[0]raises a bare'NoneType' object is not subscriptable— the actual upstream error (sitting right there in the body) never reaches the caller or the logs.OpenAILLM.generatenow checks before subscripting and raises aRuntimeErrorcarrying the response body (truncated to 500 chars):The streaming path already guards (
if chunk.choices and ...), so onlygenerateneeded it.2. Tool-call argument parsing tolerates recoverable malformations
The strict
json.loadsin the tool-call path dropped two recoverable cases (skipping the call with only a log line):strict=Falseparses them fine;json.loads("")raises, but the right reading is{}, not dropping the call.Truly irrecoverable arguments (garbage, non-dict JSON like a bare list) keep the existing skip-with-warning behavior from #33; a bare non-dict additionally can't crash
LLMFunctionCallvalidation anymore.Tests
test_openai_no_choices.py(incl. the SDK's real lenientconstruct()path) andtest_openai_tool_arguments.py.Note: commit
70f52e6on the already-merged #35 branch is a stray push of the first fix made before noticing the merge — ignore that branch head; this PR supersedes it, rebased onto currentmain(composes with thefinish_reasonhandling from #33).