Enrich and Filter support for metrics [ASP.NET Core] and [HttpClient]

[vishweshbankwar]

Currently, ASP.NET Core metric instrumentation has added support for Enrich and Filter (https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry.Instrumentation.AspNetCore/AspNetCoreMetricsInstrumentationOptions.cs). A similar functionality for HttpClient does not exist today.

These features will be removed prior to stable release of the instrumentation library. With native support for metrics added to ASP.NET Core and HttpClient in .NET8.0, any futher enhancements are expected to be added directly to these libraries. For ASP.NET Core a similar functionality is already added https://learn.microsoft.com/en-us/aspnet/core/log-mon/metrics/metrics?view=aspnetcore-8.0#enrich-the-aspnet-core-request-metric. For HttpClient, dotnet/runtime#88460 is proposed.

This issue to document the plan for these APIs with respect to stable release of these libraries.

This does not affect the Enrich and Filter APIs that are part of Tracing.

[vishweshbankwar]

@open-telemetry/dotnet-approvers @open-telemetry/dotnet-maintainers FYI

[sshumakov]

@vishweshbankwar, could you clarify what is the plan for http client metric tags enrichment in Otel targeting .NET 8? Is it something that is going to be available in the next release?

I am trying to understand if we should expect this functionality or if this is something that is not going to be available in the initial .NET8.0 and OTel release.

[cijothomas]

If you are in .NET 8.0 or newer, the .NET's built-in mechanism would help with Enrich, so you won't miss anything.

If you pre .NET 8.0 world, then no luck...

[vishweshbankwar]

could you clarify what is the plan for http client metric tags enrichment in Otel targeting .NET 8? Is it something that is going to be available in the next release?

The functionality is available today but there is no official doc for it. @JamesNK @antonfirsov - Could you please confirm the recommendation from .NET side.

using var httpclient = new HttpClient();
var message = new HttpRequestMessage();
message.RequestUri = new Uri("http://www.bing.com/");
HttpMetricsEnrichmentContext.AddCallback(message, (ctx) =>
{
ctx.AddCustomTag("MyCustomTag", "MyCustomValue");
});

[sshumakov]

The functionality is available today but there is no official doc for it. @JamesNK @antonfirsov - Could you please confirm the recommendation from .NET side.

using var httpclient = new HttpClient();
var message = new HttpRequestMessage();
message.RequestUri = new Uri("http://www.bing.com/");
HttpMetricsEnrichmentContext.AddCallback(message, (ctx) =>
{
ctx.AddCustomTag("MyCustomTag", "MyCustomValue");
});

Trying to use this way, but callback provided is not getting called. Not sure what is missing there:

    using var httpclient = new HttpClient();
    var message = new HttpRequestMessage();
    message.RequestUri = new Uri("http://www.bing.com/");
    HttpMetricsEnrichmentContext.AddCallback(message, (ctx) =>
    {
        ctx.AddCustomTag("MyCustomTag", "MyCustomValue");
    });
    await httpclient.SendAsync(message);

[JamesNK]

@antonfirsov Please take a look

[antonfirsov]

@sshumakov you need to make sure the http.client.request.duration instrument is enabled. Is anything collecting the System.Net.Http metrics?

[antonfirsov]

Could you please confirm the recommendation from .NET side.

@vishweshbankwar yes, the quoted code is correct for Encrichment. Not sure what is your definition of Filtering, but there is no built-in support for removing/renaming tags.

Note that HttpMetricsEnrichmentContext is a low-level primitive that needs a callback registration call for each HttpRequestMessage, and the assumption was that higher-level libraries will build their own (more convenient) enrichment experiences around it.

[sshumakov]

My assumption was that this code:

services.AddOpenTelemetry().WithMetrics(metrics => metrics.AddHttpClientInstrumentation());

Would enable instrument http.client.request.duration internally, but I can't find this code.

@vishweshbankwar - do we have examples how to use http client instrumentation with .NET 8, using native metric http.client.request.duration?

[sshumakov]

These features will be removed prior to stable release of the instrumentation library. With native support for metrics added to ASP.NET Core and HttpClient in .NET8.0, any futher enhancements are expected to be added directly to these libraries. For ASP.NET Core a similar functionality is already added https://learn.microsoft.com/en-us/aspnet/core/log-mon/metrics/metrics?view=aspnetcore-8.0#enrich-the-aspnet-core-request-metric. For HttpClient, dotnet/runtime#88460 is proposed.

This issue to document the plan for these APIs with respect to stable release of these libraries.

This does not affect the Enrich and Filter APIs that are part of Tracing.

Could you elaborate more on the reasons of removing metrics filtering/enrichments APIs from OTel .NET SDK? It sounds like this should be internal implementation details of how OTel .NET SDK is providing metering (diagnostic listener vs .NET native metric).

@vishweshbankwar

[sshumakov]

If you are in .NET 8.0 or newer, the .NET's built-in mechanism would help with Enrich, so you won't miss anything.

If you pre .NET 8.0 world, then no luck...

I found it confusing that AspNetCore instrumentation does provide enrichments/filtering, but not HttpClient instrumentation. The same feature set is available for http in/out tracing.

There are at least two PRs which attempted to introduce this feature, but never got merged:

• Add enrichment & filter for http client metrics opentelemetry-dotnet#4889
• Add enrichment and filtering to HttpClient metrics opentelemetry-dotnet#4374

Do you have any plans to make this feature available?

@cijothomas