Skip to content

OpenAIChatClient sends tool results and arguments to the model as indented JSON #7921

Open

Description

@jguizarj

Description

OpenAIChatClient serializes the tool results (FunctionResultContent.Result) and the tool-call arguments (FunctionCallContent.Arguments) it sends to the model with AIJsonUtilities.DefaultOptions, which has WriteIndented = true:

The caller can't change this. DefaultOptions is read-only. FunctionInvokingChatClient has no serializer options. Setting WriteIndented = false in AIFunctionFactoryOptions.SerializerOptions doesn't change what reaches the model.

The model bills the indentation as input tokens on every call of the tool loop. In one run, a tool returned 10 products and the whitespace was 151 of 484 input tokens (31%). Returning the same result as a compact string brought that call down to 333 input tokens.

Reproduction Steps

The repro is a file-based app (dotnet run repro.cs). A fake HTTP handler captures the request body that OpenAIChatClient sends, and the app prints the tool-call arguments and the tool result from that body. The tool result comes from an AIFunctionFactory function, as it would for FunctionInvokingChatClient.

#:package Microsoft.Extensions.AI.OpenAI@10.10.1
#:property PublishAot=false

using System.ClientModel;
using System.ClientModel.Primitives;
using System.Net;
using System.Text;
using System.Text.Json.Nodes;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Chat;

string? body = null;
var http = new HttpClient(new CaptureHandler(b => body = b));
IChatClient client = new ChatClient("some-model", new ApiKeyCredential("fake"), new OpenAIClientOptions
{
    Transport = new HttpClientPipelineTransport(http),
}).AsIChatClient();

var searchProducts = AIFunctionFactory.Create(
    (string query, int max) => new
    {
        products = new[]
        {
            new { sku = "NB-001", name = "Spiral notebook", available = 12 },
            new { sku = "NB-002", name = "Grid notebook", available = 3 },
        },
    },
    "SearchProducts");
var result = await searchProducts.InvokeAsync(new AIFunctionArguments { ["query"] = "notebook", ["max"] = 10 });

List<Microsoft.Extensions.AI.ChatMessage> messages =
[
    new(ChatRole.User, "Do you have notebooks?"),
    new(ChatRole.Assistant, [new FunctionCallContent("call_1", "SearchProducts", new Dictionary<string, object?> { ["query"] = "notebook", ["max"] = 10 })]),
    new(ChatRole.Tool, [new FunctionResultContent("call_1", result)]),
];

await client.GetResponseAsync(messages);

foreach (var message in JsonNode.Parse(body!)!["messages"]!.AsArray())
{
    if ((string?)message!["role"] == "assistant")
    {
        Console.WriteLine("tool_calls[0].function.arguments sent to the model:");
        Console.WriteLine((string?)message["tool_calls"]![0]!["function"]!["arguments"]);
    }
    else if ((string?)message["role"] == "tool")
    {
        Console.WriteLine("tool message content sent to the model:");
        var content = message["content"]!;
        Console.WriteLine(content is JsonArray parts ? (string?)parts[0]!["text"] : (string?)content);
    }
}

sealed class CaptureHandler(Action<string> capture) : HttpMessageHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        capture(await request.Content!.ReadAsStringAsync(cancellationToken));
        const string response = """
            {"id":"x","object":"chat.completion","created":0,"model":"some-model",
             "choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],
             "usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}
            """;
        return new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(response, Encoding.UTF8, "application/json") };
    }
}

PublishAot=false is there because file-based apps enable AOT by default and the anonymous types need reflection-based serialization.

Passing serializerOptions: new JsonSerializerOptions(AIJsonUtilities.DefaultOptions) { WriteIndented = false } to AIFunctionFactory.Create gives the same output.

Expected behavior

The JSON that OpenAIChatClient sends to the model for tool results and tool-call arguments should be compact. If the default has to stay indented, the caller should be able to pass the JsonSerializerOptions the client uses for them. AIJsonUtilities.Defaults.cs already has a JsonContextNoIndentation (#L152-L154).

Actual behavior

Both the arguments and the result reach the model indented:

tool_calls[0].function.arguments sent to the model:
{
  "query": "notebook",
  "max": 10
}
tool message content sent to the model:
{
  "products": [
    {
      "sku": "NB-001",
      "name": "Spiral notebook",
      "available": 12
    },
    {
      "sku": "NB-002",
      "name": "Grid notebook",
      "available": 3
    }
  ]
}

Regression?

No response

Known Workarounds

For the result, MarshalResult can return a compact string:

var searchProducts = AIFunctionFactory.Create(SearchProducts, new AIFunctionFactoryOptions
{
    MarshalResult = (value, type, ct) => new ValueTask<object?>(
        JsonSerializer.Serialize(value, type!, new JsonSerializerOptions(AIJsonUtilities.DefaultOptions) { WriteIndented = false })),
});

With it, the repro prints the result as {"products":[{"sku":"NB-001",...}]} and the arguments stay indented. For the arguments, the only workaround I found is a pipeline policy that rewrites the request body.

Configuration

  • Microsoft.Extensions.AI.OpenAI 10.10.1 (Microsoft.Extensions.AI.Abstractions 10.10.1, OpenAI 2.14.0)
  • .NET SDK 10.0.403, Windows 11
  • Not specific to the configuration. The indentation comes from AIJsonUtilities.DefaultOptions, and the linked lines are the same on main.

Other information

Related:

Activity

  1. added
    bugThis issue describes a behavior which is not expected - a bug.
    on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI librariesbugThis issue describes a behavior which is not expected - a bug.untriaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions