Skip to content

[QUERY] About chat context management tools in Azure OpenAI SDK #43359

Open
@kissycn

Description

@kissycn

Query/Question
Is there a built-in tool or utility in the Azure OpenAI SDK for managing chat context (history and message truncation)?
Currently, I'm manually managing the chat messages list and context, like this:

List<ChatRequestMessage> chatMessages = new ArrayList<>();
chatMessages.add(new ChatRequestSystemMessage("..."));
chatMessages.add(new ChatRequestUserMessage("..."));
chatMessages.add(new ChatRequestAssistantMessage("..."));
chatMessages.add(new ChatRequestUserMessage("..."));


IterableStream<ChatCompletions> chatCompletionsStream = client.getChatCompletionsStream(deploymentOrModelId,
        new ChatCompletionsOptions(chatMessages));

I'm looking for functionality that could help with:

Managing conversation history
Automatic message truncation when context length is exceeded
Smart context window management

If such tools don't exist in the SDK, are there any recommended patterns or best practices for implementing this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    ClientThis issue points to a problem in the data-plane of the library.OpenAIcustomer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-triageWorkflow: This issue needs the team to triage.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions