.Net: PromptRenderFilter that can inspect the prompt, and the response from the LLM?

[Daniellled]

If I have a PromptRenderFilter in .NET am I am able to inspect the RenderPrompt, and modify the prompt prior to it being sent to the LLM. While in the same class I would like to look at the response from the LLM and see if it matches the prompt. (or do some work on the response based on the initial prompt)

Is it possible to do that in one class? Or do I need a separate IFunctionInvocationFilter to handle reviewing the response? I thought about having a class implement both IFunctionInvocationFilter and IPromptRenderFilter but since both methods are separate with different context I would have to track which function call belongs to which prompt, so it didn't feel like a good solution.

[markwallace-microsoft]

@dmytrostruk can you suggest an approach to resolve this?

[dmytrostruk]

@Daniellled thanks for your question!

If I have a PromptRenderFilter in .NET am I am able to inspect the RenderPrompt, and modify the prompt prior to it being sent to the LLM.

Yes, IPromptRenderFilter is a correct place where to modify the prompt before the request.

While in the same class I would like to look at the response from the LLM and see if it matches the prompt.

It should be possible to do it in a single class that implements IFunctionInvocationFilter. In FunctionInvocationContext.Result.RenderedPrompt property, you should access the prompt that was used to get a result, so you should be able to compare the request and response and process your response as needed. Here is an example line:

semantic-kernel/dotnet/samples/Concepts/Caching/SemanticCachingWithFilters.cs

Line 234 in 4f0bf16

if (!string.IsNullOrEmpty(context.Result.RenderedPrompt))

We also have this example which demonstrates how to implement Semantic Caching with Filters and it's a similar scenario when you need to correlate the prompt and response:
https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Caching/SemanticCachingWithFilters.cs

Please let us know if it helps, thank you!