-
Notifications
You must be signed in to change notification settings - Fork 540
Embedding: Adds Microsoft.Azure.Cosmos.AI package scaffold #5845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| <?xml version="1.0" encoding="utf-8"?> | ||
| <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> | ||
| <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" /> | ||
| </Project> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Azure Cosmos DB .NET SDK AI Extension Library | ||
|
|
||
| `Microsoft.Azure.Cosmos.AI` is an optional, provider-style extension package for the [Azure Cosmos DB .NET SDK](https://www.nuget.org/packages/Microsoft.Azure.Cosmos/) that adds AI-powered capabilities to applications built on Azure Cosmos DB for NoSQL. | ||
|
|
||
| > **Status:** This package is in early development. It currently ships as a scaffold only — no public APIs are exposed yet. The first capability (an `AzureOpenAIEmbeddingGenerator` provider for automatic vector embedding generation in hybrid and vector search queries) is tracked in [#5842](https://github.com/Azure/azure-cosmos-dotnet-v3/pull/5842). Breaking changes may occur prior to GA. | ||
|
|
||
| ## Why a Separate Package | ||
|
|
||
| - **Zero cost for non-AI workloads.** Applications that don't use AI features pull in no AI dependencies. `Azure.AI.OpenAI` and related dependencies are only present for customers who explicitly opt in to this package. | ||
| - **Independent versioning.** This package ships on its own cadence, allowing rapid iteration on AI capabilities without coupling to the core SDK release cycle. | ||
| - **Established extensibility model.** Follows the same pattern proven by `Microsoft.Azure.Cosmos.Encryption` — a separate, optional, provider-style package that extends the core SDK without polluting it. | ||
| - **Clear separation of concerns.** The core SDK owns the data plane. This package owns AI service integration. | ||
|
|
||
| ## Planned Capabilities | ||
|
|
||
| - **Automatic vector embedding generation** — provide raw text to vector / hybrid search queries; the SDK calls a configured embedding service (e.g., Azure OpenAI) on your behalf. | ||
| - **Semantic re-ranking** — re-rank query results using AI services. | ||
| - **AI-assisted query construction** — generate Cosmos DB queries from natural language. | ||
| - Additional AI-powered features as the landscape evolves. | ||
|
|
||
| ## Getting Started | ||
|
|
||
| Once the first capability lands in a published preview, install from NuGet: | ||
|
|
||
| ```bash | ||
| dotnet add package Microsoft.Azure.Cosmos.AI --prerelease | ||
| ``` | ||
|
|
||
| Usage examples will be added here as APIs ship. | ||
|
|
||
| ## Related Links | ||
|
|
||
| - [Azure Cosmos DB documentation](https://learn.microsoft.com/azure/cosmos-db/) | ||
| - [Azure Cosmos DB .NET SDK on NuGet](https://www.nuget.org/packages/Microsoft.Azure.Cosmos/) | ||
| - [Changelog](./changelog.md) | ||
| - [Repository](https://github.com/Azure/azure-cosmos-dotnet-v3) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| Preview features are treated as a separate branch and will not be included in the official release until the feature is ready. Each preview release lists all the additional features that are enabled. | ||
|
|
||
| The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), | ||
| and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). | ||
|
|
||
| ### <a name="unreleased"/> Unreleased | ||
|
|
||
| #### Added | ||
| - Initial scaffold of the `Microsoft.Azure.Cosmos.AI` extension package. No public APIs are shipped in this release; the package serves as the home for AI-powered Cosmos DB capabilities (automatic embedding generation, semantic ranking, and future AI integrations) that ship in subsequent releases. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| // ------------------------------------------------------------ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // ------------------------------------------------------------ | ||
|
|
||
| using System.Runtime.CompilerServices; | ||
|
|
||
| [assembly: InternalsVisibleTo("Microsoft.Azure.Cosmos.AI.Tests" + Microsoft.Azure.Cosmos.AI.AssemblyKeys.TestPublicKey)] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The PR description says the This matters because Adding both projects to the
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in db7ea2e — both Microsoft.Azure.Cosmos.AI and Microsoft.Azure.Cosmos.AI.Tests are now in Microsoft.Azure.Cosmos.sln under a new AI solution folder (mirrors the existing Encryption / Encryption.Custom folders). Verified locally with dotnet build Microsoft.Azure.Cosmos.sln -p:IsPreview=true -p:SdkProjectRef=true --configuration Release — 0 errors, 16 pre-existing warnings (all Microsoft.Azure.Cosmos.Performance.Tests net6.0 EOL warnings, unrelated). This now means every PR build that runs the solution will catch core-SDK changes that break the AI package, same as the Encryption guardrail. |
||
| <PropertyGroup> | ||
| <TargetFramework>netstandard2.0</TargetFramework> | ||
| <AssemblyName>Microsoft.Azure.Cosmos.AI</AssemblyName> | ||
| <RootNamespace>Microsoft.Azure.Cosmos.AI</RootNamespace> | ||
| <LangVersion>$(LangVersion)</LangVersion> | ||
| <CosmosAIVersion Condition=" '$(IsPreview)' != 'true' ">$(CosmosAIOfficialVersion)</CosmosAIVersion> | ||
| <CosmosAIVersion Condition=" '$(IsPreview)' == 'true' ">$(CosmosAIPreviewVersion)</CosmosAIVersion> | ||
| <CosmosAIVersionSuffix Condition=" '$(IsPreview)' == 'true' ">$(CosmosAIPreviewSuffixVersion)</CosmosAIVersionSuffix> | ||
| <Version Condition=" '$(CosmosAIVersionSuffix)' == '' ">$(CosmosAIVersion)</Version> | ||
| <Version Condition=" '$(CosmosAIVersionSuffix)' != '' ">$(CosmosAIVersion)-$(CosmosAIVersionSuffix)</Version> | ||
| <CurrentDate>$([System.DateTime]::Now.ToString(yyyyMMdd))</CurrentDate> | ||
| <Company>Microsoft Corporation</Company> | ||
| <Authors>Microsoft</Authors> | ||
| <Description>AI extensions library for Azure Cosmos DB for NoSQL. Provides an Azure OpenAI / Foundry-backed embedding provider for automatic vector embedding generation in hybrid and vector search queries. Designed to be extensible for broader AI scenarios including semantic re-ranking, OCR, and model integrations. For more information, refer to https://aka.ms/CosmosDB</Description> | ||
| <Copyright>© Microsoft Corporation. All rights reserved.</Copyright> | ||
| <Title>Microsoft Azure Cosmos DB AI extension library</Title> | ||
| <PackageId>Microsoft.Azure.Cosmos.AI</PackageId> | ||
| <PackageRequireLicenseAcceptance>true</PackageRequireLicenseAcceptance> | ||
| <PackageLicenseExpression>MIT</PackageLicenseExpression> | ||
| <PackageProjectUrl>https://github.com/Azure/azure-cosmos-dotnet-v3</PackageProjectUrl> | ||
| <PackageReleaseNotes>The change log for this SDK is made available at https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.AI/changelog.md at the time of release.</PackageReleaseNotes> | ||
| <PackageReadmeFile>README.md</PackageReadmeFile> | ||
| <GeneratePackageOnBuild>false</GeneratePackageOnBuild> | ||
| <GenerateDocumentationFile>true</GenerateDocumentationFile> | ||
| <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||
| <PackageTags>microsoft;azure;cosmos;cosmosdb;documentdb;nosql;azureofficial;dotnetcore;netcore;netstandard;ai;embedding;vector;openai;semanticranking</PackageTags> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Two artifacts worth seeding now, before the first preview push to nuget.org:
Both are essentially free at the scaffold stage and load-bearing once the package is on nuget.org.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in db7ea2e:
#5842 inherits the changelog Unreleased bucket — its description should add a customer-facing entry under ### Added for AzureOpenAIEmbeddingGenerator, per the repo's changelog classifier. |
||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <AdditionalFiles Include="..\..\Microsoft.Azure.Cosmos\src\stylecop.json" Link="stylecop.json" /> | ||
| <None Include="..\README.md" Pack="true" PackagePath="\" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Core SDK reference: project ref when building from source, otherwise NuGet pin --> | ||
| <ItemGroup Condition=" '$(SdkProjectRef)' != 'True' AND '$(IsPreview)' != 'True' "> | ||
| <PackageReference Include="Microsoft.Azure.Cosmos" Version="[3.59.0,)" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup Condition=" '$(SdkProjectRef)' != 'True' AND '$(IsPreview)' == 'True' "> | ||
| <PackageReference Include="Microsoft.Azure.Cosmos" Version="3.60.0-preview.0" /> | ||
|
ananth7592 marked this conversation as resolved.
|
||
| </ItemGroup> | ||
|
|
||
| <ItemGroup Condition=" '$(SdkProjectRef)' == 'True' "> | ||
| <ProjectReference Include="..\..\Microsoft.Azure.Cosmos\src\Microsoft.Azure.Cosmos.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <None Include="..\..\Icon.png" Pack="true" PackagePath=""/> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Required by Microsoft.Azure.Cosmos SDK; PrivateAssets=All keeps it off the public dependency graph --> | ||
| <ItemGroup> | ||
| <PackageReference Include="Newtonsoft.Json" Version="10.0.2" NoWarn="NU1903" PrivateAssets="All" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Azure OpenAI provider dependency --> | ||
| <ItemGroup> | ||
| <PackageReference Include="Azure.AI.OpenAI" Version="2.1.0" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Analyzers --> | ||
| <ItemGroup> | ||
| <PackageReference Include="StyleCop.Analyzers" Version="1.1.118" PrivateAssets="All" /> | ||
| <PackageReference Include="Microsoft.VisualStudio.Threading.Analyzers" Version="16.0.102" PrivateAssets="All" /> | ||
| </ItemGroup> | ||
|
|
||
| <!-- Strong-name signing --> | ||
| <PropertyGroup> | ||
| <SigningType>Product</SigningType> | ||
| <SignAssembly>true</SignAssembly> | ||
| <DelaySign>true</DelaySign> | ||
| <AssemblyOriginatorKeyFile>..\..\35MSSharedLib1024.snk</AssemblyOriginatorKeyFile> | ||
| </PropertyGroup> | ||
|
|
||
| <PropertyGroup Condition=" '$(SdkProjectRef)' == 'True' "> | ||
| <DefineSdkProjectRefSymbol Condition=" '$(DefineSdkProjectRefSymbol)' == '' ">true</DefineSdkProjectRefSymbol> | ||
| </PropertyGroup> | ||
|
|
||
| <PropertyGroup Condition=" '$(DefineSdkProjectRefSymbol)' == 'true' "> | ||
| <DefineConstants>$(DefineConstants);SDKPROJECTREF</DefineConstants> | ||
| </PropertyGroup> | ||
| </Project> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| // ------------------------------------------------------------ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // ------------------------------------------------------------ | ||
|
|
||
| namespace Microsoft.Azure.Cosmos.AI | ||
| { | ||
| internal static class AssemblyKeys | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This makes three copies of the same hard-coded test public key in the repo (Encryption, Encryption.Custom, and now AI), all under "Mirrored" folders with no canonical source. If the key is ever rotated, three files diverge. Out of scope for this PR, but worth a follow-up: extract a single
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed and noted — leaving the 3 Mirrored\AssemblyKeys.cs copies as-is in this PR per your out-of-scope call. I'll file a follow-up issue to extract a single canonical AssemblyKeys.cs under Microsoft.Azure.Cosmos/build/ and it from all three packages (Encryption, Encryption.Custom, AI), and link the issue from this thread once filed. |
||
| { | ||
| /// <summary>TestPublicKey is an unsupported strong key for testing and internal use only.</summary> | ||
| internal const string TestPublicKey = ", PublicKey=0024000004800000940000000602000000240000525341310004000001000100197c25d0a04f73cb271e8181dba1c0c713df8deebb25864541a66670500f34896d280484b45fe1ff6c29f2ee7aa175d8bcbd0c83cc23901a894a86996030f6292ce6eda6e6f3e6c74b3c5a3ded4903c951e6747e6102969503360f7781bf8bf015058eb89b7621798ccc85aaca036ff1bc1556bb7f62de15908484886aa8bbae"; | ||
|
ananth7592 marked this conversation as resolved.
|
||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| namespace Microsoft.Azure.Cosmos.AI.Tests | ||
| { | ||
| using Microsoft.VisualStudio.TestTools.UnitTesting; | ||
|
|
||
| [TestCategory("Windows")] | ||
| [TestCategory("UpdateContract")] | ||
| [TestClass] | ||
| public class ContractEnforcementTests | ||
| { | ||
| /// <summary> | ||
| /// Locks the public API surface of <c>Microsoft.Azure.Cosmos.AI</c> against a | ||
| /// versioned baseline so accidental renames, visibility changes, or removals | ||
| /// surface at PR review time instead of shipping to a preview NuGet. | ||
| /// | ||
| /// IMPORTANT: Because tests can run on multiple .NET versions, the contract | ||
| /// validation uses framework-specific baselines to ensure consistency: | ||
| /// | ||
| /// - When running on net8.0: validates against DotNetSDKCosmosAIAPI.net8.json | ||
| /// | ||
| /// To update baselines, run: UpdateContracts.ps1 from the repository root. | ||
| /// </summary> | ||
| [TestMethod] | ||
| public void ContractChanges() | ||
| { | ||
| Cosmos.Tests.Contracts.ContractEnforcement.ValidateContract( | ||
| dllName: "Microsoft.Azure.Cosmos.AI", | ||
| contractType: Cosmos.Tests.Contracts.ContractType.Standard, | ||
| baselinePattern: "DotNetSDKCosmosAIAPI", | ||
| breakingChangesPattern: "DotNetSDKCosmosAIAPIChanges"); | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| { | ||
| "Subclasses": {}, | ||
| "Members": { | ||
| "Boolean Equals(System.Object)": { | ||
| "Type": "Method", | ||
| "Attributes": [], | ||
| "MethodInfo": "Boolean Equals(System.Object);IsAbstract:False;IsStatic:False;IsVirtual:True;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "Int32 GetHashCode()": { | ||
| "Type": "Method", | ||
| "Attributes": [], | ||
| "MethodInfo": "Int32 GetHashCode();IsAbstract:False;IsStatic:False;IsVirtual:True;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "System.Boolean System.Object.Equals(System.Object, System.Object)": { | ||
| "Type": "Method", | ||
| "Attributes": [], | ||
| "MethodInfo": "System.Boolean System.Object.Equals(System.Object, System.Object);IsAbstract:False;IsStatic:True;IsVirtual:False;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "System.Boolean System.Object.ReferenceEquals(System.Object, System.Object)": { | ||
| "Type": "Method", | ||
| "Attributes": [], | ||
| "MethodInfo": "System.Boolean System.Object.ReferenceEquals(System.Object, System.Object);IsAbstract:False;IsStatic:True;IsVirtual:False;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "System.String ToString()": { | ||
| "Type": "Method", | ||
| "Attributes": [], | ||
| "MethodInfo": "System.String ToString();IsAbstract:False;IsStatic:False;IsVirtual:True;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "System.Type GetType()[System.Runtime.CompilerServices.NullableContextAttribute((Byte)1)]-[System.Runtime.CompilerServices.IntrinsicAttribute()]": { | ||
| "Type": "Method", | ||
| "Attributes": [ | ||
| "IntrinsicAttribute", | ||
| "NullableContextAttribute" | ||
| ], | ||
| "MethodInfo": "System.Type GetType();IsAbstract:False;IsStatic:False;IsVirtual:False;IsGenericMethod:False;IsConstructor:False;IsFinal:False;" | ||
| }, | ||
| "Void .ctor()": { | ||
| "Type": "Constructor", | ||
| "Attributes": [], | ||
| "MethodInfo": "Void .ctor()" | ||
| } | ||
| }, | ||
| "NestedTypes": {} | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <IsTestProject>true</IsTestProject> | ||
| <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||
| <Platform>AnyCPU</Platform> | ||
| <TargetFramework>net8.0</TargetFramework> | ||
| <IsPackable>false</IsPackable> | ||
| <GenerateAssemblyInfo>false</GenerateAssemblyInfo> | ||
| <RootNamespace>Microsoft.Azure.Cosmos.AI.Tests</RootNamespace> | ||
| <LangVersion>$(LangVersion)</LangVersion> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Compile Include="..\..\..\Microsoft.Azure.Cosmos\tests\Microsoft.Azure.Cosmos.Tests\Contracts\ContractEnforcement.cs" Link="Contracts\ContractEnforcement.cs" /> | ||
| <Compile Include="..\..\..\Microsoft.Azure.Cosmos\tests\Microsoft.Azure.Cosmos.Tests\Contracts\ContractType.cs" Link="Contracts\ContractType.cs" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" /> | ||
| <PackageReference Include="Moq" Version="4.8.3" /> | ||
| <PackageReference Include="MSTest.TestAdapter" Version="3.7.3" /> | ||
| <PackageReference Include="MSTest.TestFramework" Version="3.7.3" /> | ||
| <PackageReference Include="Newtonsoft.Json" Version="13.0.3" /> | ||
| <PackageReference Include="coverlet.collector" Version="6.0.2" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\src\Microsoft.Azure.Cosmos.AI.csproj" /> | ||
|
ananth7592 marked this conversation as resolved.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
A new GA-track package is the cheapest moment to wire this up — once #5842 lands <Compile Include="..\..\..\Microsoft.Azure.Cosmos\tests\Microsoft.Azure.Cosmos.Tests\Contracts\ContractEnforcement.cs" Link="Contracts\ContractEnforcement.cs" />
<Compile Include="..\..\..\Microsoft.Azure.Cosmos\tests\Microsoft.Azure.Cosmos.Tests\Contracts\ContractType.cs" Link="Contracts\ContractType.cs" />plus an empty
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in db7ea2e:
Test run: Passed 1/1 (ContractChanges). |
||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <None Include="Contracts\DotNetSDKCosmosAIAPI*.json"> | ||
| <CopyToOutputDirectory>Always</CopyToOutputDirectory> | ||
| </None> | ||
| </ItemGroup> | ||
|
|
||
| <PropertyGroup> | ||
| <SignAssembly>true</SignAssembly> | ||
| <DelaySign>true</DelaySign> | ||
| <AssemblyOriginatorKeyFile>..\..\..\testkey.snk</AssemblyOriginatorKeyFile> | ||
| </PropertyGroup> | ||
|
|
||
| <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'"> | ||
| <PlatformTarget>x64</PlatformTarget> | ||
| <AllowUnsafeBlocks>true</AllowUnsafeBlocks> | ||
| </PropertyGroup> | ||
|
|
||
| </Project> | ||
Uh oh!
There was an error while loading. Please reload this page.