Save FunctionResult to ContextVariables and not to ChatHistory

[HuskyDanny]

Can we configure such case so that the function result invoked by an agent will be saved in contextvariables and not to the chat history which is by default.

This is important for my scenario, since the agent will run multiple retrieval, and the response is large, putting in chat history will slow down the downstream tasks, in addition, I also need downstream tasks to use the data directly, having the llm to infer the parameters to put from chat history is simply inaccurate and slow.

So, the core intentions are to:

1. Saving FunctionResult as key-value pair in a central config store, best if kernel can provide the store for downstream task use.
2. Delete the ChatHistory's messages for FunctionResult, as I don't want the network to be overwhelmed.

After some explorations, I found
https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/caching/semantic_caching.py
I am thinking to use the filter to save the function result, and use the in memory store to save and retrieve, but not found an elegant way to remove the unneeded messages for FunctionResult, unless I loop through the ChatHistory from the FunctionContext passed from the filter parameter and remove the FunctionalCall message.

Any ideas or examples? Thanks!

[sophialagerkranspandey]

We'd recommend writing a chat history reducer for this. https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/chat-history?pivots=programming-language-csharp#chat-history-reduction

[moonbox3]

@HuskyDanny these are the Python Chat History reducer samples along with the learn site doc.

[HuskyDanny]

Thanks @sophialagerkranspandey and @moonbox3, I will use it for reduce the history messages. But one other thing is does kernel support the shared memory k-v store out of box, the k-v store should be global store serving any k-v pair during any point of the workflow.

[HuskyDanny]

And I see ChatHistory Reducer is more like a general strategy to remove long history. There is no built-in method to say remove some plulgins function result, I guess that will be custom function extending this class right?

[moonbox3]

Hi @HuskyDanny, we don't have a shared memory k-v store out of the box. Regarding the Chat History Reducer: this is the second time today I have seen someone ask about functionality to get a chat history without the FunctionCallContent/FunctionResultContent. I was planning to file an issue to investigate this type of behavior. If you were to use a ChatHistorySummarizationReducer, it will summarize the text either including the function call content or without it (IIRC, there's a flag to toggle this). Some people may want to leave that function calling content as part of the chat history and some may not care about it. Please have a look at the current implementation and provide feedback if anything falls short, or there's functionality you'd like to see.

[HuskyDanny]

Hi @moonbox3, if I understand correctly, current implementations are treating the FunctionResultContent as the message in the ChatHistory, so the llm could have the context by the list of previous messages including the FunctionResultContent. However, for my case, I have a downstream task called PromptAugmentPlugin which is to augment the propmt in a certain format with the exact FunctionResultContent, once I have augmented my prompt, I don't need the FunctionResultContent to be in the ChatHistory.

In addition, the agent will use PromptAugmentPlugin, but I don't want the functional calling to parse all these FunctionResultContent in the history to the parameters of PromptAugmentPlugin, as it takes very much long time if my FunctionResultContent is big.

So, I feel it is a general need that someone might just want the result to processed and not in the history, but I didn't see much discussion about this, so not sure if there are some other ways I just don't see. If there are no other ways, I do feel it is a feature request.

Looking at the current implementations, the function results just get stored in the history without other options, I think it could resolve our need if there is an option to save to memory and not in the history.

[HuskyDanny]

Also @sophialagerkranspandey It seems we cannot get ChatHistory from FunctionInvocationContext in the filter hook

 async def on_function_invocation(
        self, 
        context: FunctionInvocationContext, 
        next: Callable[[FunctionInvocationContext], Awaitable[None]]
    ) -> None:
        """Intercept function calls, check cache, and store results."""
        plugin_name = context.function.plugin_name
        function_name = context.function.name
        # Only process TextMemoryPlugin functions
        if plugin_name != "TextMemoryPlugin":
            await next(context)
            return
   
        # Execute the function
        await next(context)

        chat_history : FunctionResultReducer = context.arguments["chat_history"] #ERROR!!!
        
        
        # Cache the result
        if context.result:
            logger.info(f"Caching result for {plugin_name}.{function_name}")
            self.memory_store.store(plugin_name, function_name, context.result.value)
            chat_history.reduce()

[eavanvalkenburg]

@HuskyDanny try the AutoFunctionInvocationFilter instead, that has a field for chatHistory.

[HuskyDanny]

@eavanvalkenburg I cannot search it under python sdk, is it supported in python?

[moonbox3]

@HuskyDanny, Eduard is referring to this filter:

• SK Python Learn Site Doc about Auto Function Invoke Filter
• auto_function_invoke_filters.py code sample
• The AutoFunctionInvocationFilter uses the AutoFunctionInvocationContext

[HuskyDanny]

Thanks, I will look into it!

[HuskyDanny]

Another question related to this is even if I use the ChatHistoryReducer, the ChatHistory is not reduced in ChatCompletionAgent.

And looking through the code, I found internally it dumps the message to the ChatHistory at _inner_invoke:

The function:

Is it by design that the ChatHistory can only be maintained within the agent? How to use ChatHistoryReducer which automatically reduce in this case?

[moonbox3]

The chat history is managed by the caller. Therefore, if you choose to use a history reducer, there will be some point when you're adding messages to it (before or after invoking a message on the agent). You can then call await history.reduce().

[HuskyDanny]

Now it seems I got the design. Basically we would have two ChatHistories, one is internally in the agent, and another one is the outer one as we passed in, and we could reduce the outside ChatHistory and we could not directly modify the internal one unless we use filter, we can control the internal flow by the filter and internal chathistory, and we will manage the overall user messages and response in the outer one , do I understand it correctly?