Integration for `Azure.AI.Inference` #9103

aaronpowell · 2025-05-05T04:17:44Z

Description

Implementing an integration for the Azure AI Inference NuGet package.

For the method, I've added Azure to the name, even though the type being registered is a ChatCompletionsClient and it can be used without Azure (it's GitHub Models compatible). This aims to be consistent with the other integrations like Table.

Fixes #9011

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No
Does the change require an update in our Aspire docs?
- Yes
  - [New article]: New Azure.AI.Inference integration docs-aspire#3394
- No

Fixes dotnet#9011

davidfowl · 2025-05-05T04:27:00Z

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

+            if (builder.DisableTracing)
+            {
+                return result;
+            }
+
+            var loggerFactory = services.GetService<ILoggerFactory>();
+            return new OpenTelemetryChatClient(result, loggerFactory?.CreateLogger(typeof(OpenTelemetryChatClient)));


Why does it do this instead of using the UseOpenTelemetry helper method?

Didn't know it existed. I copied that from the OpenAI integration

Me as the author of the OpenAI integration: 😅

See #9103 (comment)

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

src/Components/Aspire.Azure.AI.Inference/AspireChatCompletionsClientBuilder.cs

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj

eerhardt · 2025-05-05T14:51:16Z

src/Components/Aspire.Azure.AI.Inference/ChatCompletionsClientSettings.cs

+    /// <summary>
+    /// Gets or sets the name of the AI model to use for chat completions.
+    /// </summary>
+    public string? ModelName { get; set; }


Where is this used?

Possibly isn't, I think it was from an older implementation

eerhardt · 2025-05-05T14:52:03Z

src/Components/Aspire.Azure.AI.Inference/ChatCompletionsClientSettings.cs

+    /// <summary>
+    /// Gets or sets the ID of the AI model to use for chat completions.
+    /// </summary>
+    public string? ModelId { get; set; }


The Azure.AI.Inference library calls this "deploymentId", right? We also use the term DeploymentId in AspireChatCompletionsClientBuilder. We should consolidate on a single term here.

Missed that in my refactor, will fix.

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj

Co-authored-by: Eric Erhardt <[email protected]>

eerhardt · 2025-05-07T14:04:28Z

src/Components/Aspire.Azure.AI.Inference/ChatCompletionsClientSettings.cs

+    /// Gets or sets a boolean value that indicates whether the OpenTelemetry tracing is disabled or not.
+    /// </summary>
+    /// <remarks>
+    /// ServiceBus ActivitySource support in Azure SDK is experimental, the shape of Activities may change in the future without notice.


Suggested change

/// ServiceBus ActivitySource support in Azure SDK is experimental, the shape of Activities may change in the future without notice.

/// Azure AI Inference client library ActivitySource support in Azure SDK is experimental, the shape of Activities may change in the future without notice.

eerhardt · 2025-05-07T14:06:15Z

src/Components/Aspire.Azure.AI.Inference/ChatCompletionsClientSettings.cs

+    /// Gets or sets the connection string used to connect to the AI Foundry account.
+    /// </summary>
+    /// <remarks>
+    /// If <see cref="ConnectionString"/> is set, it overrides <see cref="Endpoint"/> and <see cref="Credential"/>.


This comment is a bit misleading, since setting ConnectionString will populate Endpoint in ParseConnectionString below.

I think the comment was lifted from here:

aspire/src/Components/Aspire.Azure.Data.Tables/AzureDataTablesSettings.cs

Line 18 in 3859112

/// If <see cref="ConnectionString"/> is set, it overrides <see cref="ServiceUri"/> and <see cref="Credential"/>.

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj

…ther integrations

danmoseley · 2025-05-12T19:03:23Z

@eerhardt could you please review too?

eerhardt · 2025-05-12T19:23:07Z

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

+    public static AspireChatCompletionsClientBuilder AddAzureChatCompletionsClient(
+        this IHostApplicationBuilder builder,
+        string connectionName,
+        Action<ChatCompletionsClientSettings>? configureClient = null,


configureClient => configureSettings

eerhardt · 2025-05-12T19:29:20Z

src/Components/Aspire.Azure.AI.Inference/AspireChatCompletionsClientBuilder.cs

+    /// <summary>
+    /// The ID of the deployment in Azure AI Foundry.
+    /// </summary>
+    public string? DeploymentId { get; } = deploymentId;


Where does this get used?

eerhardt · 2025-05-12T19:31:13Z

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

+    /// <param name="builder">An <see cref="AspireChatCompletionsClientBuilder" />.</param>
+    /// <param name="serviceKey">The service key with which the <see cref="IChatClient"/> will be registered.</param>
+    /// <returns></returns>
+    public static ChatClientBuilder AddKeyedChatClient(this AspireChatCompletionsClientBuilder builder, string? serviceKey)


serviceKey shouldn't be nullable if you are calling AddKeyedChatClient.

The builder has a service key:

serviceKey = serviceKey ?? builder.ServiceKey;

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs

Use DeploymentId when calling AsIChatClient. Make serviceKey required.

sebastienros · 2025-05-12T20:35:54Z

I am doing the required changes, will merge after

danmoseley · 2025-05-12T23:25:52Z

@joperezr we want this one even though it was just after 4pm.

joperezr · 2025-05-14T15:23:30Z

@joperezr we want this one even though it was just after 4pm.

Sure, let's put up a backport PR then. The target is to have the final build by EoD today.

joperezr · 2025-05-14T15:25:03Z

Oh nvm, just realized this made the cutoff

Initial implementation of Inference SDK as an integration

cacc720

Fixes dotnet#9011

aaronpowell requested review from radical, eerhardt and sebastienros as code owners May 5, 2025 04:17

dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label May 5, 2025

github-actions bot added the area-integrations Issues pertaining to Aspire Integrations packages label May 5, 2025

davidfowl reviewed May 5, 2025

View reviewed changes

RussKie reviewed May 5, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Show resolved Hide resolved

src/Components/Aspire.Azure.AI.Inference/AspireChatCompletionsClientBuilder.cs Show resolved Hide resolved

RussKie mentioned this pull request May 5, 2025

Integration for Azure.AI.projects #7819

Open

18 tasks

aaronpowell commented May 5, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Outdated Show resolved Hide resolved

aaronpowell added 2 commits May 5, 2025 16:02

Making properties public

5f10191

Fixing missed refactor code

3a2cc94

eerhardt reviewed May 5, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj Outdated Show resolved Hide resolved

eerhardt reviewed May 5, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj Show resolved Hide resolved

aaronpowell and others added 5 commits May 6, 2025 09:30

Apply suggestions from code review

51ac74c

Co-authored-by: Eric Erhardt <[email protected]>

Updating from code review feedback

f7230ed

Merge branch 'main' into aaronpowell/issue-9011

7f4889d

Adding tests

e08b444

Updating based on feedback

efdc6f8

eerhardt reviewed May 7, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Outdated Show resolved Hide resolved

eerhardt reviewed May 7, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Outdated Show resolved Hide resolved

eerhardt reviewed May 7, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/Aspire.Azure.AI.Inference.csproj Show resolved Hide resolved

aaronpowell added 4 commits May 10, 2025 09:50

Adding readme

ede53da

Adding tests

137f02f

Adding change from feedback

9f9c025

Adding change from feedback

1357337

aaronpowell requested a review from eerhardt May 10, 2025 00:10

aaronpowell and others added 4 commits May 10, 2025 10:12

Renaming the method to include Azure as that's more consistent with o…

c7cf933

…ther integrations

Merge branch 'main' into aaronpowell/issue-9011

70ad18f

Fix unit test

dfbaed1

Fix README inclusion

d7e69de

sebastienros approved these changes May 12, 2025

View reviewed changes

danmoseley approved these changes May 12, 2025

View reviewed changes

danmoseley added this to the 9.3 milestone May 12, 2025

danmoseley assigned sebastienros May 12, 2025

eerhardt reviewed May 12, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Outdated Show resolved Hide resolved

eerhardt reviewed May 12, 2025

View reviewed changes

src/Components/Aspire.Azure.AI.Inference/AspireAzureAIInferenceExtensions.cs Outdated Show resolved Hide resolved

Tweak AddChatClient APIs to take an optional DeploymentId.

5e2d407

Use DeploymentId when calling AsIChatClient. Make serviceKey required.

sebastienros added 3 commits May 12, 2025 14:26

Merge branch 'main' into aaronpowell/issue-9011

32c281a

Rename argument

30fb1ec

Add unit test for deploymentId

b19df43

sebastienros mentioned this pull request May 12, 2025

[New article]: New Azure.AI.Inference integration dotnet/docs-aspire#3394

Closed

sebastienros merged commit 524992e into dotnet:main May 12, 2025
174 checks passed

aaronpowell deleted the aaronpowell/issue-9011 branch May 13, 2025 00:03

davidfowl mentioned this pull request May 13, 2025

.NET Aspire release 9.3 content dotnet/docs-aspire#3368

Merged

github-actions bot locked and limited conversation to collaborators Jun 14, 2025

	/// ServiceBus ActivitySource support in Azure SDK is experimental, the shape of Activities may change in the future without notice.
	/// Azure AI Inference client library ActivitySource support in Azure SDK is experimental, the shape of Activities may change in the future without notice.

Integration for Azure.AI.Inference #9103

Integration for Azure.AI.Inference #9103

Uh oh!

Conversation

aaronpowell commented May 5, 2025 • edited by sebastienros Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danmoseley commented May 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sebastienros commented May 12, 2025

Uh oh!

Uh oh!

danmoseley commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joperezr commented May 14, 2025

Uh oh!

joperezr commented May 14, 2025

Uh oh!

Uh oh!

Integration for `Azure.AI.Inference` #9103

Integration for `Azure.AI.Inference` #9103

aaronpowell commented May 5, 2025 •

edited by sebastienros

Loading

danmoseley commented May 12, 2025 •

edited

Loading