VectorStoreRAG demo with Azure AI Search, my implementation fails because IVectorSearch<T> doesn't know how to generate embeddings

[godefroi]

I'm going through the VectorStoreRAG demo, and using Azure AI Search for the vector store and Azure OpenAI for the chat and embeddings. I cannot successfully search, though. When I attempt to actually ask questions of SK, I get this:

Call to LLM failed with error: Microsoft.Extensions.VectorData.VectorStoreOperationException: Call to vector store failed.
---> Azure.RequestFailedException: Field 'TextEmbedding' does not have a vectorizer defined in it's vector profile.

(note the spelling error; the message should say "its" instead of "it's")

Looking through the source, it seems to me that two strategies have been set up here in VectorStoreTextSearch<T>.ExecuteVectorSearchAsync(). The first one uses the _textEmbeddingGeneration field to generate embeddings, but the constructors that set that field are all marked deprecated, so I assume that's the "old" way. The second one uses the _vectorSearch field (of type IVectorSearch<T>) which the deprecation message says should be "configured to perform embedding generation". My IVectorSearch<T> however is not configured to perform embedding generation (I have confirmed this by making the call directly), and I see nothing in the demo source that would configure it to do that.

For what it's worth, my IVectorSearch<T> implementation is a Microsoft.SemanticKernel.Connectors.AzureAISearch.AzureAISearchVectorStoreRecordCollection<string, T>, but I can't seem to find that class in any "official" SK repository. The copies I can find (for example, in the elastic repo) don't seem to have any obvious way that they could be configured to have a non-server-side vectorizer.

What am I missing? How do I configure my IVectorSearch<T> to do embedding generation?

[sophialagerkranspandey]

Tagging @westey-m

[westey-m]

@godefroi, thank for raising this issue. We have been making quite a lot of improvements to the vector store area, and we missed updating this sample. I have updated it now with some changes to fix any issues. See pr #11903.

Specifically, we now support embedding generation inside of the vector store classes. The AzureAISearchVectorStoreRecordCollection class is here: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.Memory.AzureAISearch/AzureAISearchVectorStoreRecordCollection.cs
If you check the options that can be supplied to its constructor, you can see that it supports passing an embedding generator:
https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.Memory.AzureAISearch/AzureAISearchVectorStoreRecordCollectionOptions.cs#L46

You can read more about the built in embedding generation we now support here:
https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/embedding-generation

You can also read more about the April 2025 changes we made here: https://learn.microsoft.com/en-us/semantic-kernel/support/migration/vectorstore-april-2025?pivots=programming-language-csharp

[godefroi]

That helps quite a bit! I have, I guess, a couple remaining questions. First, it really seemed to me that I was already getting the "generate embeddings automatically on upsert" functionality, when this documentation implies it's a new feature and requires this configuration. Second, when I'm configuring this via DI, how do I get access to the IEmbeddingGenerator during DI setup? I.e.:

var collectionOptions = new AzureAISearchVectorStoreRecordCollectionOptions<DocumentChunk>() {
	EmbeddingGenerator = ??? // *** I don't have an embedding generator here ***
};

var kernelBuilder = builder.Services.AddKernel()
	.AddAzureOpenAIChatCompletion("my-chat-model")
	.AddAzureOpenAITextEmbeddingGeneration("my-embedding-model")
	.AddAzureAISearchVectorStoreRecordCollection<DocumentChunk>("my-index-name", new Uri("https://my-ai-search.search.windows.net/"), new DefaultAzureCredential(), collectionOptions)
	.AddVectorStoreTextSearch<DocumentChunk>(
		new TextSearchStringMapper(result => (result as DocumentChunk)!.Text!),
		new TextSearchResultMapper(result => {
			var chunk = (result as DocumentChunk)!;

			return new TextSearchResult(chunk.Text!) {
				Name = chunk.ReferenceDescription,
				Link = chunk.ReferenceLink
			};
		})
	);

Should I construct the AzureOpenAIClient separately before this point in order to use it in the collectionOptions?

Finally, if I do that (construct an AzureOpenAIClient outside of DI and set it on the EmbeddingGenerator property), I get this:

Unhandled exception. System.NotSupportedException: The Azure AI Search connector does not currently support a
custom embedding generator (configured for property 'TextEmbedding' on type 'DocumentChunk'). However, you
can configure embedding generation in Azure AI Search itself, without requiring a .NET IEmbeddingGenerator.

... so it seems like this isn't possible yet at this point anyway.

[westey-m]

@godefroi, firstly on adding the embedding generator during DI. If you register an IEmbeddingGenerator before you register your record collection, the IEmbeddingGenerator will automatically be added to your options.
See https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/Connectors/Connectors.Memory.AzureAISearch/AzureAISearchServiceCollectionExtensions.cs#L140

You of course also don't have to use the DI registration helper methods we provide and can easily do something like the following yourself if you want to further customize the registration. There's no secret source in our registration helpers, they are just convenience methods.

        services.AddKeyedSingleton<IVectorStoreRecordCollection<string, TRecord>>(
            serviceId,
            (sp, obj) =>
            {
                var searchIndexClient = sp.GetRequiredService<SearchIndexClient>();
                return new AzureAISearchVectorStoreRecordCollection<string, TRecord>(
                    searchIndexClient,
                     collectionName,
                     new() { EmbeddingGenerator = sp.GetRequiredService<IEmbeddingGenerator>() });
            });

Secondly, you are right! We do have an outstanding issue for AzureAISearch (all the other connectors that we maintain ourselves fully support this feature already) in that it also supports generating vectors on the server side for search at least. We have to still come up with a way of deciding whether to use the local embedding or defer to the server in a way that is intuitive to consumers. We hope to resolve this soon. See the following issue for tracking: #11941

Also note that we still support, and will always support, generating vectors outside of the vector store and providing them both for upsert and for search.

CC @roji

[godefroi]

Thank you! For the upsert, I think I actually was generating my own embeddings (by setting the appropriate property before calling UpsertAsync), but I'd commented that out at some point for reasons I no longer remember...

As for search, I can see that for IVectorSearch<T> I can call SearchEmbeddingAsync after generating my own embeddings; how do I set this up in the SK plugin (as created by VectorStoreTextSearch<T>, for example)? Would I need to create my own custom plugin that does the embedding generation before calling the search?

[westey-m]

If you use one of the obsolete constructors that allows passing in an embedding generator, it will still get used.
You will have to suppress the obsolete warning though.
Thanks for bringing this up. We should definitely reconsider obsolescence of these constructors, for scenarios where someone doesn't want to or cannot use the built in embedding generation. I've added a new issue to track and decide what to do: #11946

Note that we are in the process of obsoleting ITextEmbeddingGenerationService in SK in favor of IEmbeddingGenerator from Microsoft.Extensions.AI. You will therefore see a mixture of these types required by different parts of the code. E.g. the vector stores use IEmbeddingGenerator but the obsoleted constructors for VectorStoreTextSearch use ITextEmbeddingGenerationService.
It is however possible to easily convert from one to the other using the extension methods AsEmbeddingGenerator and AsTextEmbeddingGenerationService if needed.

[westey-m]

@godefroi, for yesterday's release of Semantic Kernel we added a new VectorStoreTextSearch constructor overload that allows an IEmbeddingGenerator to be provided and will generate embeddings with this before search. This mechanism can be used with AzureAISearch until we have full support for built-in embedding for the AzureAISearch connector.
See:
https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Core/Data/TextSearch/VectorStoreTextSearch.cs#L34