Empty response from trustcall

[Chajf]

I'm encountering an issue where Trustcall returns empty outputs when structuring responses in the following environment:

While using trustcall for structing output in configuration with

• Langchain Ollama - 0.3.6
• Langgraph - 0.3.5
• Qwen3:8b and GPT-oss:20b
• Trustcall - 0.0.39

When invoking, I receive responses with no content in either the messages or responses fields, even though token usage and execution times indicate the model generated something.

Example empty output

{
    "messages": [
        {
            "content": "",
            "additional_kwargs": {},
            "response_metadata": {
                "model": "qwen3:latest",
                "created_at": "2025-08-14T12:19:40.795352902Z",
                "done": true,
                "done_reason": "stop",
                "total_duration": 102373450948,
                "load_duration": 85624850,
                "prompt_eval_count": 7667,
                "prompt_eval_duration": 3286289025,
                "eval_count": 5008,
                "eval_duration": 98940289640,
                "model_name": "qwen3:latest"
            },
            "type": "ai",
            "name": null,
            "id": "run--617ad51e-0611-4f60-a167-a8b545926c4d-0",
            "example": false,
            "tool_calls": [],
            "invalid_tool_calls": [],
            "usage_metadata": {
                "input_tokens": 7667,
                "output_tokens": 5008,
                "total_tokens": 12675
            }
        }
    ],
    "responses": [],
    "response_metadata": [],
    "attempts": 1
}

Increasing the model context (num_ctx) sometimes resolves the issue for specific inputs, but it's inconsistent. Importantly, the problem seems related to tool calls: for the same or smaller context and identical input, the behavior changes—sometimes I get valid tool call responses.

For same input as provided before we can get valid answer:

{
    "messages": [
        {
            "content": "",
            "additional_kwargs": {},
            "response_metadata": {
                "model": "qwen3:latest",
                "created_at": "2025-08-14T12:58:48.480010548Z",
                "done": true,
                "done_reason": "stop",
                "total_duration": 115513915397,
                "load_duration": 107242045,
                "prompt_eval_count": 7667,
                "prompt_eval_duration": 3792590206,
                "eval_count": 5404,
                "eval_duration": 111556287803,
                "model_name": "qwen3:latest"
            },
            "type": "ai",
            "name": null,
            "id": "run--4f9b619c-b015-408d-a9b1-d19a3ec5f1cf-0",
            "example": false,
            "tool_calls": [
                {
                    "id": "c3d88e80-9632-4c2d-964d-49cf13f7e3d3",
                    "name": "Epic",
                    "args": { [...] }
        }
    ],
    "response_metadata": [
        {
            "id": "c3d88e80-9632-4c2d-964d-49cf13f7e3d3"
        }
    ],
    "attempts": 2
}

Configuration Snippet:

extractor = create_extractor(llm, tools=[Epic], tool_choice="Epic")
chain = prompt | extractor
ans = chain.invoke(
    input={
        "chat_history": [],
        "user_input": msg,
    }
)

After printing ans on empty response

{'messages': [AIMessage(content='', additional_kwargs={}, response_metadata={'model': 'qwen3:latest', 'created_at': '2025-08-14T10:41:53.470475984Z', 'done': True, 'done_reason': 'stop', 'total_duration': 97660323124, 'load_duration': 97297347, 'prompt_eval_count': 4519, 'prompt_eval_duration': 1898189101, 'eval_count': 4036, 'eval_duration': 95598085728, 'model_name': 'qwen3:latest'}, id='run--f57dadae-a667-4bb5-9fa1-fff9ed43fb05-0', usage_metadata={'input_tokens': 4519, 'output_tokens': 4036, 'total_tokens': 8555})], 'responses': [], 'response_metadata': [], 'attempts': 1}

[onestardao]

this looks like no.13 – multi-agent chaos from our 16-item problem map: the model/provider stack is generating tokens (usage shows output tokens, stop: "stop"), but the adapter layer that trustcall reads from doesn’t find the fields it expects, so it returns empty messages/responses.

quick ways to unstick it

• dump the raw provider payload right before trustcall parses it; if you’re routing via langchain/ollama/hf, make sure the final object has the content where trustcall expects it.
• turn off streaming for a baseline run; some adapters only populate messages when not streaming.
• set an explicit schema/response_model in trustcall and enable a lenient parse (capture raw text on validation failure).
• single-provider baseline: run the same prompt through just one client (e.g., openai only) to confirm the parser path, then add the others back.
• guard the return path: if messages.length === 0 && raw_text, surface raw_text as a fallback so you don’t drop successful generations on the floor.

we track and fix this class a lot; project is MIT-licensed, part of a set of 16 common failure patterns, built from a cold start → ~600 stars in ~60 days. if you want the tiny “adapter guard + raw capture” snippet we use for trustcall-style parsers, say the word and i’ll share.

[Chajf]

Thank you for the detailed breakdown and thoughtful suggestions - really helpful analysis.

I'd appreciate it if you could share the "adapter guard + raw capture" snippet you mentioned. It sounds like a solid way to handle this type of issue.

[onestardao]

Thanks for the context. This is the “provider/adapter shape mismatch → empty messages” class. A small adapter guard prevents silent drops and gives you a safe fallback for debugging.

resp = provider.invoke(prompt, **kwargs)

# adapter guard + raw capture
msgs = resp.get("messages")
if not msgs:  # empty, None, or missing
    raw_text = resp.get("raw_text") or resp.get("content") or str(resp)
    return [{"role": "assistant", "content": raw_text}]  # surface raw payload

return msgs

Quick checks that usually unstick it (no infra changes required):

• run a baseline with streaming off (some adapters only populate messages when not streaming)
• set an explicit schema/response_model so Trustcall fails fast on mismatch
• start with a single provider baseline, then add others back
• log the raw provider payload right before Trustcall parses it

This maps to the ProblemMap item for empty-message responses. Full checklist (MIT):
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

Happy to share a slightly stricter guard (types + shape assertions) if you want.