Skip to content

Add support for serializing KeyValue lists#7015

Open
juliuskoval wants to merge 12 commits into
open-telemetry:mainfrom
juliuskoval:keyvalue-lists
Open

Add support for serializing KeyValue lists#7015
juliuskoval wants to merge 12 commits into
open-telemetry:mainfrom
juliuskoval:keyvalue-lists

Conversation

@juliuskoval

@juliuskoval juliuskoval commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

According to the spec, attribute values should be AnyValue. One of the possible values of AnyValue is KeyValueList, which we currently aren't able to serialize properly.

Changes

This PR adds support for serializing KeyValue lists to the OTLP exporter.

Merge requirement checklist

  • CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
  • Unit tests added/updated
  • Appropriate CHANGELOG.md files updated for non-trivial changes
  • Changes in public API reviewed (if applicable)

@github-actions github-actions Bot added pkg:OpenTelemetry.Exporter.Console Issues related to OpenTelemetry.Exporter.Console NuGet package pkg:OpenTelemetry.Exporter.OpenTelemetryProtocol Issues related to OpenTelemetry.Exporter.OpenTelemetryProtocol NuGet package pkg:OpenTelemetry.Exporter.Zipkin Issues related to OpenTelemetry.Exporter.Zipkin NuGet package pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package labels Mar 29, 2026
@juliuskoval

Copy link
Copy Markdown
Contributor Author

Question: As far as I know, the spec doesn't say anything about limits to have many nested layers there can be, but shouldn't we try to guard against self-referencing IEnumerables?

@codecov

codecov Bot commented Mar 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 30.76923% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.77%. Comparing base (7ef0ebc) to head (d3141e4).
⚠️ Report is 19 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...xporter.Console/Implementation/ConsoleTagWriter.cs 0.00% 49 Missing ⚠️
....Exporter.Zipkin/Implementation/ZipkinTagWriter.cs 0.00% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #7015      +/-   ##
==========================================
- Coverage   89.98%   89.77%   -0.21%     
==========================================
  Files         277      278       +1     
  Lines       14622    14778     +156     
==========================================
+ Hits        13157    13267     +110     
- Misses       1465     1511      +46     
Flag Coverage Δ
unittests-Project-Experimental 89.73% <30.76%> (-0.20%) ⬇️
unittests-Project-Stable 89.74% <30.76%> (-0.12%) ⬇️
unittests-Solution 89.70% <30.76%> (-0.28%) ⬇️
unittests-UnstableCoreLibraries-Experimental 48.85% <ø> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...Implementation/Serializer/ProtobufOtlpTagWriter.cs 97.22% <100.00%> (+0.79%) ⬆️
....Exporter.Zipkin/Implementation/ZipkinTagWriter.cs 50.00% <0.00%> (-10.87%) ⬇️
...xporter.Console/Implementation/ConsoleTagWriter.cs 31.25% <0.00%> (-49.40%) ⬇️

... and 28 files with indirect coverage changes

@martincostello

Copy link
Copy Markdown
Member

shouldn't we try to guard against self-referencing IEnumerables

I think so, otherwise it would cause a stack overflow? Could just add a recursion limit to track such occurences.

@github-actions

github-actions Bot commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day.

@github-actions github-actions Bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Apr 8, 2026
@juliuskoval

Copy link
Copy Markdown
Contributor Author

I added a limit to recursion depth and a unit test for the behavior. For now I arbitrarily set the limit to 3, because I'm guessing it would be subject to discussion anyway.

@juliuskoval juliuskoval marked this pull request as ready for review April 12, 2026 09:05
@juliuskoval juliuskoval requested a review from a team as a code owner April 12, 2026 09:05
@juliuskoval juliuskoval changed the title Add support for serializing KeyValue lists - WIP Add support for serializing KeyValue lists Apr 12, 2026
@github-actions github-actions Bot removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Apr 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or pushing will instruct the bot to automatically remove the label. This bot runs once per day.

@github-actions github-actions Bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Apr 21, 2026
@Kielek Kielek removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Apr 21, 2026
@martincostello martincostello added the keep-open Prevents issues and pull requests being closed as stale label Apr 21, 2026

@martincostello martincostello left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also extend the fuzz tests to cover this functionality please?

Comment thread src/Shared/TagWriter/TagWriter.cs Outdated
Comment thread src/OpenTelemetry.Exporter.Zipkin/Implementation/ZipkinTagWriter.cs
Comment on lines +308 to +315
from severity in Gen.Elements(
LogRecordSeverity.Debug,
LogRecordSeverity.Error,
LogRecordSeverity.Fatal,
LogRecordSeverity.Info,
LogRecordSeverity.Trace,
LogRecordSeverity.Unspecified,
LogRecordSeverity.Warn)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be simplified using Enum.GetValues()?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enum.GetValues() doesn't exist on .NET Framework, so we'd have to put it behind a preprocessor directive which wouldn't be very elegant.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does, you just have to use the GetValues(Type) version not GetValues<T>(). But we can leave it if it's still too ugly.

@SalvoDeveloper

Copy link
Copy Markdown

We are having trouble sending complex nested objects to an OpenTelemetryCollector. I have tested the fork of @juliuskoval by connecting NLog.Target.OpenTelemetryProtocol with the OpenTelemetry-dotnet. I only need to traslate the object into IEnumerable<KeyValuePair<string, object?>>, and the everything works. I would kindly ask you to merge this and release new version. This would be really important for us.

@martincostello

Copy link
Copy Markdown
Member

@SalvoDeveloper I'm waiting for at least one other approving review from one of the other maintainers before merging this. Once merged it will be part of the next release (which has no specific date at this time).

After it's merged you can use consume prerelease builds from MyGet if you can't wait until then.

@rajkumar-rangaraj

Copy link
Copy Markdown
Member

Before merging, I'd like us to align on:

  1. Is implicit type-sniffing the path we want, or should we wait for typed AnyValue API so we don't ship two redundant contracts?
  2. How do we address Blanch's reference-aliasing concern from [feature request] support extensible tag serialization #6052 - should this require SDK-level deep-copy on span end?

Strong support for the underlying use case, but want to avoid locking in behavior we can't revisit.

juliuskoval and others added 7 commits June 22, 2026 20:59
Co-authored-by: Martin Costello <martin@martincostello.com>
…ListAttributeTests.cs

Co-authored-by: Martin Costello <martin@martincostello.com>
@github-actions github-actions Bot added the perf Performance related label Jun 22, 2026
@juliuskoval

Copy link
Copy Markdown
Contributor Author

@rajkumar-rangaraj @martincostello I added the benchmarks that we talked about 2 weeks ago. It measures serializing a log with a varying number of attributes and optionally a KvList attribute with no nesting and also with a varying number of attributes.

BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.8457/25H2/2025Update/HudsonValley2)
AMD Ryzen 7 9800X3D 4.70GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 10.0.301
[Host] : .NET 10.0.9 (10.0.9, 10.0.926.27113), X64 RyuJIT x86-64-v4
DefaultJob : .NET 10.0.9 (10.0.9, 10.0.926.27113), X64 RyuJIT x86-64-v4

Method AttributeCount KvListAttributeCount Mean Error StdDev Median Allocated
WriteLogRecord 4 0 113.8 ns 1.87 ns 1.57 ns 113.9 ns 32 B
WriteLogRecord 4 2 168.9 ns 3.15 ns 8.31 ns 166.4 ns 32 B
WriteLogRecord 4 4 224.2 ns 4.55 ns 12.84 ns 219.1 ns 32 B
WriteLogRecord 4 8 254.4 ns 5.11 ns 10.99 ns 252.6 ns 32 B
WriteLogRecord 8 0 193.0 ns 3.88 ns 9.58 ns 188.7 ns 32 B
WriteLogRecord 8 2 239.4 ns 2.89 ns 2.26 ns 239.3 ns 32 B
WriteLogRecord 8 4 297.5 ns 5.87 ns 10.58 ns 294.2 ns 32 B
WriteLogRecord 8 8 340.9 ns 8.21 ns 23.03 ns 332.2 ns 32 B
WriteLogRecord 16 0 365.1 ns 7.95 ns 22.16 ns 357.4 ns 32 B
WriteLogRecord 16 2 427.9 ns 11.96 ns 34.70 ns 417.5 ns 32 B
WriteLogRecord 16 4 476.7 ns 15.13 ns 44.62 ns 480.0 ns 32 B
WriteLogRecord 16 8 471.3 ns 9.11 ns 12.16 ns 466.0 ns 32 B

Comment on lines +209 to +211
=> maxLength.HasValue && value.Length > maxLength
? value.Slice(0, maxLength.Value)
: value;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
=> maxLength.HasValue && value.Length > maxLength
? value.Slice(0, maxLength.Value)
: value;
=> maxLength is { } maxLengthValue && value.Length > maxLengthValue
? value.Slice(0, maxLengthValue)
: value;

Comment thread test/Benchmarks/Exporter/ProtobufOtlpLogSerializerBenchmarks.cs Outdated
@martincostello

Copy link
Copy Markdown
Member

These changes seem reasonable to me. Raj is on PTO until August so let's wait on another +1 from another maintainer.

Can you also update the CHANGELOGs as-appropriate please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

keep-open Prevents issues and pull requests being closed as stale perf Performance related pkg:OpenTelemetry.Exporter.Console Issues related to OpenTelemetry.Exporter.Console NuGet package pkg:OpenTelemetry.Exporter.OpenTelemetryProtocol Issues related to OpenTelemetry.Exporter.OpenTelemetryProtocol NuGet package pkg:OpenTelemetry.Exporter.Zipkin Issues related to OpenTelemetry.Exporter.Zipkin NuGet package pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants