Repository navigation
Description
Description
OpenAIChatClient serializes the tool results (FunctionResultContent.Result) and the tool-call arguments (FunctionCallContent.Arguments) it sends to the model with AIJsonUtilities.DefaultOptions, which has WriteIndented = true:
- Result: OpenAIChatClient.cs#L192
- Arguments: OpenAIChatClient.cs#L219-L220
The caller can't change this. DefaultOptions is read-only. FunctionInvokingChatClient has no serializer options. Setting WriteIndented = false in AIFunctionFactoryOptions.SerializerOptions doesn't change what reaches the model.
The model bills the indentation as input tokens on every call of the tool loop. In one run, a tool returned 10 products and the whitespace was 151 of 484 input tokens (31%). Returning the same result as a compact string brought that call down to 333 input tokens.
Reproduction Steps
The repro is a file-based app (dotnet run repro.cs). A fake HTTP handler captures the request body that OpenAIChatClient sends, and the app prints the tool-call arguments and the tool result from that body. The tool result comes from an AIFunctionFactory function, as it would for FunctionInvokingChatClient.
#:package Microsoft.Extensions.AI.OpenAI@10.10.1
#:property PublishAot=false
using System.ClientModel;
using System.ClientModel.Primitives;
using System.Net;
using System.Text;
using System.Text.Json.Nodes;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Chat;
string? body = null;
var http = new HttpClient(new CaptureHandler(b => body = b));
IChatClient client = new ChatClient("some-model", new ApiKeyCredential("fake"), new OpenAIClientOptions
{
Transport = new HttpClientPipelineTransport(http),
}).AsIChatClient();
var searchProducts = AIFunctionFactory.Create(
(string query, int max) => new
{
products = new[]
{
new { sku = "NB-001", name = "Spiral notebook", available = 12 },
new { sku = "NB-002", name = "Grid notebook", available = 3 },
},
},
"SearchProducts");
var result = await searchProducts.InvokeAsync(new AIFunctionArguments { ["query"] = "notebook", ["max"] = 10 });
List<Microsoft.Extensions.AI.ChatMessage> messages =
[
new(ChatRole.User, "Do you have notebooks?"),
new(ChatRole.Assistant, [new FunctionCallContent("call_1", "SearchProducts", new Dictionary<string, object?> { ["query"] = "notebook", ["max"] = 10 })]),
new(ChatRole.Tool, [new FunctionResultContent("call_1", result)]),
];
await client.GetResponseAsync(messages);
foreach (var message in JsonNode.Parse(body!)!["messages"]!.AsArray())
{
if ((string?)message!["role"] == "assistant")
{
Console.WriteLine("tool_calls[0].function.arguments sent to the model:");
Console.WriteLine((string?)message["tool_calls"]![0]!["function"]!["arguments"]);
}
else if ((string?)message["role"] == "tool")
{
Console.WriteLine("tool message content sent to the model:");
var content = message["content"]!;
Console.WriteLine(content is JsonArray parts ? (string?)parts[0]!["text"] : (string?)content);
}
}
sealed class CaptureHandler(Action<string> capture) : HttpMessageHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
capture(await request.Content!.ReadAsStringAsync(cancellationToken));
const string response = """
{"id":"x","object":"chat.completion","created":0,"model":"some-model",
"choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],
"usage":{"prompt_tokens":1,"completion_tokens":1,"total_tokens":2}}
""";
return new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(response, Encoding.UTF8, "application/json") };
}
}PublishAot=false is there because file-based apps enable AOT by default and the anonymous types need reflection-based serialization.
Passing serializerOptions: new JsonSerializerOptions(AIJsonUtilities.DefaultOptions) { WriteIndented = false } to AIFunctionFactory.Create gives the same output.
Expected behavior
The JSON that OpenAIChatClient sends to the model for tool results and tool-call arguments should be compact. If the default has to stay indented, the caller should be able to pass the JsonSerializerOptions the client uses for them. AIJsonUtilities.Defaults.cs already has a JsonContextNoIndentation (#L152-L154).
Actual behavior
Both the arguments and the result reach the model indented:
tool_calls[0].function.arguments sent to the model:
{
"query": "notebook",
"max": 10
}
tool message content sent to the model:
{
"products": [
{
"sku": "NB-001",
"name": "Spiral notebook",
"available": 12
},
{
"sku": "NB-002",
"name": "Grid notebook",
"available": 3
}
]
}
Regression?
No response
Known Workarounds
For the result, MarshalResult can return a compact string:
var searchProducts = AIFunctionFactory.Create(SearchProducts, new AIFunctionFactoryOptions
{
MarshalResult = (value, type, ct) => new ValueTask<object?>(
JsonSerializer.Serialize(value, type!, new JsonSerializerOptions(AIJsonUtilities.DefaultOptions) { WriteIndented = false })),
});With it, the repro prints the result as {"products":[{"sku":"NB-001",...}]} and the arguments stay indented. For the arguments, the only workaround I found is a pipeline policy that rewrites the request body.
Configuration
- Microsoft.Extensions.AI.OpenAI 10.10.1 (Microsoft.Extensions.AI.Abstractions 10.10.1, OpenAI 2.14.0)
- .NET SDK 10.0.403, Windows 11
- Not specific to the configuration. The indentation comes from
AIJsonUtilities.DefaultOptions, and the linked lines are the same onmain.
Other information
Related:
- Allow controlling JSON formatting for gen_ai input/output message telemetry #7514 asks for the same control over the JSON in the
gen_ai.input.messages/gen_ai.output.messagestelemetry, and Allow OpenTelemetry*Client.JsonSerializerOptions to control full OTel message formatting #7558 tried to address it. Both are about telemetry, not about what is sent to the model. - [M.E.AI] Tool call format problems #6483 mentioned the indentation in passing ("It costs a little more money. So is the indentation problem."), but the discussion there was about
additionalProperties.
Activity