Skip to content

[Hybrid Agent] Decorate OpenTelemetry dimensional metrics with otlp_resource_attributes from connect response#2956

Open
jasonjkeller wants to merge 9 commits into
mainfrom
add-otel-metric-attributes
Open

[Hybrid Agent] Decorate OpenTelemetry dimensional metrics with otlp_resource_attributes from connect response#2956
jasonjkeller wants to merge 9 commits into
mainfrom
add-otel-metric-attributes

Conversation

@jasonjkeller

@jasonjkeller jasonjkeller commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Resolves #2875

This PR is currently pending the agent spec being approved: https://source.datanerd.us/agents/agent-specs/pull/821

Dynamically updates the OTel resource with the otlp_resource_attributes received in the connect response so that OTel dimensional metrics get decorated with the same attributes as timeslice metrics/events.

The general approach is:

  • Adds a new internal getServiceMetadata API to the AgentBridge that provides a map of otlp_resource_attributes parsed from the latest connect response.
  • The hybrid agent instrumentation module configures the OTel SDK with a wrapped MetricExporter which delegates to the original exporter and allows for adding custom logic to the export calls.
  • When it intercepts the export calls, it polls the AgentBridge API for the latest connect attributes and builds a new Resource (if one isn’t already cached) with the attributes that we want to decorate onto the metrics. That Resource is cached until a subsequent call to export occurs and it detects that the connect attributes have changed, which will trigger an updated Resource to be built and cached.
  • There’s also an extended wrapper class for MetricData that delegates to the original MetricData but overrides the getResource method. Effectively, it merges the existing Resource for each MetricData with the customized Resource and exports a batch of the extended wrapper MetricData instances so that getResource returns the merged attributes.

Example otlp_resource_attributes field:

        "otlp_resource_attributes": {
            "tags.foo": "bar",
            "instanceName": "PKXVHV7MV1:8080",
            "appName": "Production",
            "k8s.containerName": "k8_container_name",
            "k8s.nodeName": "k8_node_name",
            "tags.releaseTag": "v1.2.3",
            "tags.service.version": "2.0",
            "k8s.podName": "k8_pod_name",
            "k8s.clusterName": "k8_cluster_name",
            "tags.commit": "xxxxxxx",
            "licenseKey": "07624511",
            "k8s.namespaceName": "k8_namespace_name",
            "k8s.deploymentName": "k8_deployment_name",
            "host": "PKXVHV7MV1",
            "entity.guid": "MTA0OTc3N3xBUE18QVBQTElDQVRJT058OTg4MTk2NjE2",
            "host.displayName": "PKXVHV7MV1",
            "agent.version": "9.4.0-SNAPSHOT",
            "realAgentId": "969827753",
            "tags.tagName1": "tagValue1",
            "tags.tagName3": "tagValue3",
            "tags.tagName2": "tagValue2"
        }

Related:

@codecov-commenter

codecov-commenter commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.68421% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.79%. Comparing base (57b05ef) to head (adb7fe3).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
...t/src/main/java/com/newrelic/agent/RPMService.java 66.66% 3 Missing and 1 partial ⚠️
...ent/introspec/internal/IntrospectorRPMService.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2956      +/-   ##
============================================
+ Coverage     70.76%   70.79%   +0.02%     
- Complexity    10659    10664       +5     
============================================
  Files           879      874       -5     
  Lines         42971    42970       -1     
  Branches       6501     6503       +2     
============================================
+ Hits          30409    30419      +10     
+ Misses         9627     9613      -14     
- Partials       2935     2938       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs Review

Development

Successfully merging this pull request may close these issues.

Backend and agent spec work related to adding agent metadata to OTLP dimensional metrics

2 participants