Description
Some connectors (for example OpenAI and Mistral) return token usage in ChatMessageContent.Metadata, which might be used later in application's logic (usually for request/usage limiting).
Currently Amazon Bedrock connector does not do that, even though that would be trivial to implement:
Index: dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs
<+>UTF-8
===================================================================
diff --git a/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs b/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs
--- a/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs (revision 71ea643dee10486fd1fff93ade09cab85f6b44b0)
+++ b/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs (date 1747666877685)
@@ -130,7 +130,11 @@
{
Role = BedrockClientUtilities.MapConversationRoleToAuthorRole(message.Role.Value),
Items = CreateChatMessageContentItemCollection(message.Content),
- InnerContent = response
+ InnerContent = response,
+ Metadata = new Dictionary<string, object?>
+ {
+ { "Usage", response.Usage }
+ }
}
];
}
I can volunteer to add this and update tests.
Notes:
I don't like how currently token usage is stored in ChatMessageContent - it is hidden in Metadata under Usage key and it leaks model specific usage structure (for example OpenAI.Chat.ChatTokenUsage or Microsoft.SemanticKernel.Connectors.MistralAI.Client.MistralUsage), but that could be separate discussion and larger change.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Backlog