What does setting InScope actually do to instruct the chat service to limit responses to data found in the Azure Search Chat Data Source?

[EMadd2015]

I am trying to replicate some functionality I saw while demo-ing SK using the AzureChatDataSource parameter in AzureOpenAIPromptExecutionSettings, and that is restricting responses strictly to only what's found in the data source provided (as well as adding citations).

I have multiple indexes in Azure Search that I would like to be able to query, and AzureOpenAIPromptExecutionSettings only supports one datasource, so I chose to use VectorStores instead, and inject the results of the query into a system message within the chat history of a call to a chat completion service (as opposed to creating a plugin and invoking a prompt directly from the kernel).

I would like to know how exactly setting the InScope parameter for AzureChatDataSource instructs the chat completion service to only respond with information found in the data source provided, as well as how exactly citations are added to the response, and how I can replicate those functionalities using multiple vector stores instead.

[sophialagerkranspandey]

Tagging @dmytrostruk

[dmytrostruk]

@EMadd2015

I would like to know how exactly setting the InScope parameter for AzureChatDataSource instructs the chat completion service to only respond with information found in the data source provided, as well as how exactly citations are added to the response, and how I can replicate those functionalities using multiple vector stores instead.

I'm not sure how exactly InScope parameter is implemented but taking into account that you are providing your data to AI model with system message, you can update your system message to ask model to answer questions only using the provided data, otherwise provide a general response that the information is not available. Same for citations, you would need to share with AI model the information about your documents, not only their content, but also a metadata, such as file name and so on. In this case, AI model will be able to provide a response based on your data together with citations.

We have a RAG example in our repo where we store the data and its metadata (like reference link) in vector store, and then we share it with AI model and ask questions about the data:

semantic-kernel/dotnet/samples/Demos/VectorStoreRAG/DataLoader.cs

Lines 57 to 65 in 87bc3d6

	// Map each paragraph to a TextSnippet and generate an embedding for it.
	var recordTasks = textContent.Select(async content => new TextSnippet<TKey>
	{
	Key = uniqueKeyGenerator.GenerateKey(),
	Text = content.Text,
	ReferenceDescription = $"{new FileInfo(pdfPath).Name}#page={content.PageNumber}",
	ReferenceLink = $"{new Uri(new FileInfo(pdfPath).FullName).AbsoluteUri}#page={content.PageNumber}",
	TextEmbedding = await GenerateEmbeddingsWithRetryAsync(textEmbeddingGenerationService, content.Text!, cancellationToken: cancellationToken).ConfigureAwait(false)
	});

Here is the prompt we are using:

semantic-kernel/dotnet/samples/Demos/VectorStoreRAG/RAGChatService.cs

Lines 118 to 133 in 87bc3d6

	var response = kernel.InvokePromptStreamingAsync(
	promptTemplate: """
	Please use this information to answer the question:
	{{#with (SearchPlugin-GetTextSearchResults question)}}
	{{#each this}}
	Name: {{Name}}
	Value: {{Value}}
	Link: {{Link}}
	-----------------
	{{/each}}
	{{/with}}
	Include citations to the relevant information where it is referenced in the response.
	Question: {{question}}
	""",

As mentioned before, you can extend similar prompt and ask model to provide some general message if the data is not available. Hope this helps!