Skip to content

Agents with Vision or returning ImageContent from KernelFunction #11145

Answered by RogerBarreto
floboc asked this question in Q&A
Discussion options

You must be logged in to vote

@floboc That's a very interesting question, thanks for bringing it in. So currently when the Plugin is invoked by the AIModel according to the function calling pattern the answer needs to go back to the model as a message.role=tool where there isn't on option to identify the function result in a multi modal way that the AI Model will recognize as image/audio.

BUT, is it possible to be a bit creative here using Semantic Kernel where basically you can inject that generated image in the Chat History everytime you have one in your kernel context.

Here's how I would do it:

  1. inject the Kernel into your RenderPreview(Kernel kernel) function.

  2. Once you get the imageContent created you can lever…

Replies: 2 comments 2 replies

Comment options

You must be logged in to vote
0 replies
Comment options

You must be logged in to vote
2 replies
@floboc
Comment options

@mbenson01
Comment options

Answer selected by sophialagerkranspandey
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
4 participants