Skip to content

feat(outputs.opentelemetry): Add a config option for specifying name/field separator #19156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mchrome wants to merge 6 commits into
base: master
Choose a base branch
from
Open

feat(outputs.opentelemetry): Add a config option for specifying name/field separator #19156

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d5aff96
feat(outputs.opentelemetry): Add a config option for specifying name/…
mchrome Jun 24, 2026
af4cbc5
Add description of the new config field to sample.conf & readme
mchrome Jul 3, 2026
dfcab49
Rename otel_name_separator to name_separator
mchrome Jul 3, 2026
894adfa
Use Connect() in all test functions instead of injecting converters d…
mchrome Jul 6, 2026
9fd0978
Add status 500 header to response when failing to parse request body …
mchrome Jul 6, 2026
2eb45e5
Add a new test that uses the default name separator
mchrome Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ go 1.26.0

godebug x509negativeserial=1

replace github.com/influxdata/influxdb-observability/influx2otel => github.com/mchrome/influxdb-observability/influx2otel v0.0.0-20260623155732-db200540d054

@skartikey skartikey Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't land on master. The real behavior lives in influxdb-observability#341, so that one needs to be reviewed, merged, and released first; then bump influx2otel here and drop the replace. Note influx2otel, common, and otel2influx are versioned together (currently v0.5.12), so the bump will likely move all three.

@mchrome mchrome Jul 6, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got you. Is there anything i can do to get that PR reviewed and merged?


require (
cloud.google.com/go/auth v0.20.0
cloud.google.com/go/bigquery v1.77.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1661,8 +1661,6 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/influxdata/influxdb-observability/common v0.5.12 h1:4YwZ+vsodz6VfoiX+ZqVotmnyCa9vCCPksSBK/WLjBs=
github.com/influxdata/influxdb-observability/common v0.5.12/go.mod h1:u+CABnGO/F1IK51pDlZQroh4+igJNo695XrbLGDBhVc=
github.com/influxdata/influxdb-observability/influx2otel v0.5.12 h1:u0lNE3+63rILk4mtmCYsNyczH/1wEXnM+1aBzBe5akk=
github.com/influxdata/influxdb-observability/influx2otel v0.5.12/go.mod h1:bM407XIJYnrJYJ9Q3q2ytDSOyFhiYmGm0Sz1Qf48RPk=
github.com/influxdata/influxdb-observability/otel2influx v0.5.12 h1:t9gmVOOHbZyEAvIYSoO97Tde1KArVtiYdM0/0Dhmuio=
github.com/influxdata/influxdb-observability/otel2influx v0.5.12/go.mod h1:YGsb8xYfjHvcr2y0+Nj7kOHMTw7fWDbAA4g/qJKkvaU=
github.com/influxdata/line-protocol-corpus v0.0.0-20210519164801-ca6fa5da0184/go.mod h1:03nmhxzZ7Xk2pdG+lmMd7mHDfeVOYFyhOgwO61qWU98=
Expand Down Expand Up @@ -1866,6 +1864,8 @@ github.com/mattn/go-runewidth v0.0.21 h1:jJKAZiQH+2mIinzCJIaIG9Be1+0NR+5sz/lYEEj
github.com/mattn/go-runewidth v0.0.21/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mchrome/influxdb-observability/influx2otel v0.0.0-20260623155732-db200540d054 h1:WXWL5QDbST9SMghx58SgxKadDTS1YqFyngMS+7BHnPk=
github.com/mchrome/influxdb-observability/influx2otel v0.0.0-20260623155732-db200540d054/go.mod h1:YgNF+ESSKXizCNRvkIAFMS8rWaP8t/86GLuycJ7vOTw=
github.com/mdlayher/apcupsd v0.0.0-20220319200143-473c7b5f3c6a h1:JOlLsLUQnokTyWWwEvOVoKH3XUl6oDMP8jisO54l6J8=
github.com/mdlayher/apcupsd v0.0.0-20220319200143-473c7b5f3c6a/go.mod h1:960H6oqSawdujauTeLX9BOx+ZdYX0TdG9xE9br5bino=
github.com/mdlayher/ethtool v0.0.0-20210210192532-2b88debcdd43/go.mod h1:+t7E0lkKfbBsebllff1xdTmyJt8lH37niI6kwFk9OTo=
Expand Down
29 changes: 16 additions & 13 deletions plugins/outputs/opentelemetry/opentelemetry.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,13 @@ type OpenTelemetry struct {
EncodingType string `toml:"encoding_type"`

tls.ClientConfig
Timeout config.Duration `toml:"timeout"`
Compression string `toml:"compression"`
Token config.Secret `toml:"token"`
Headers map[string]string `toml:"headers"`
Attributes map[string]string `toml:"attributes"`
Coralogix *CoralogixConfig `toml:"coralogix"`
Timeout config.Duration `toml:"timeout"`
Compression string `toml:"compression"`
OtelNameSeparator string `toml:"otel_name_separator"`

@skartikey skartikey Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otel_name_separator repeats "otel". name_separator (or just separator) reads better. Config is user-facing API and renaming after release is breaking, so worth settling now.

@mchrome mchrome Jul 6, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed it here dfcab49.

Token config.Secret `toml:"token"`
Headers map[string]string `toml:"headers"`
Attributes map[string]string `toml:"attributes"`
Coralogix *CoralogixConfig `toml:"coralogix"`
proxy.HTTPProxy
proxy.TCPProxy

Expand Down Expand Up @@ -105,7 +106,7 @@ func (o *OpenTelemetry) Connect() error {
o.Headers["Authorization"] = "Bearer " + o.Coralogix.PrivateKey
}

metricsConverter, err := influx2otel.NewLineProtocolToOtelMetrics(logger)
metricsConverter, err := influx2otel.NewLineProtocolToOtelMetricsWithSeparator(logger, o.OtelNameSeparator)
if err != nil {
return err
}
Expand Down Expand Up @@ -232,17 +233,19 @@ func (o *OpenTelemetry) sendBatch(metrics []telegraf.Metric) error {
}

const (
defaultServiceAddress = "localhost:4317"
defaultTimeout = config.Duration(5 * time.Second)
defaultCompression = "gzip"
defaultServiceAddress = "localhost:4317"
defaultTimeout = config.Duration(5 * time.Second)
defaultCompression = "gzip"
defaultOtelNameSeparator = "_"
)

func init() {
outputs.Add("opentelemetry", func() telegraf.Output {
return &OpenTelemetry{
ServiceAddress: defaultServiceAddress,
Timeout: defaultTimeout,
Compression: defaultCompression,
ServiceAddress: defaultServiceAddress,
Timeout: defaultTimeout,
Compression: defaultCompression,
OtelNameSeparator: defaultOtelNameSeparator,
}
})
}
255 changes: 255 additions & 0 deletions plugins/outputs/opentelemetry/opentelemetry_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,261 @@ func TestOpenTelemetryHTTPJSON(t *testing.T) {
require.JSONEq(t, string(expectJSON), string(gotJSON))
}

func TestOpenTelemetrySeparator(t *testing.T) {

@skartikey skartikey Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new tests only cover .. Backward compatibility relies on the default _ path through Connect() producing the same output as before, but nothing asserts that (the existing default tests inject the converter directly / use the old constructor). A test that calls Connect() with the default and asserts mem_used_percent would lock that in.

@mchrome mchrome Jul 6, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rewrote the old tests in 894adfa because using the old constructor doesn't make much sense anymore since the plugin only uses the new one. Now they don't inject converters directly and create them through Connect(). Also added the test you described in 2eb45e5.

expect := pmetric.NewMetrics()
{
rm := expect.ResourceMetrics().AppendEmpty()
rm.Resource().Attributes().PutStr("host.name", "potato")
rm.Resource().Attributes().PutStr("attr-key", "attr-val")
ilm := rm.ScopeMetrics().AppendEmpty()
ilm.Scope().SetName("My Library Name")
m := ilm.Metrics().AppendEmpty()
m.SetName("mem.used_percent")
m.SetEmptyGauge()
dp := m.Gauge().DataPoints().AppendEmpty()
dp.Attributes().PutStr("foo", "bar")
dp.SetTimestamp(pcommon.Timestamp(1622848686000000000))
dp.SetDoubleValue(87.332)
}
m := newMockOtelService(t)
t.Cleanup(m.Cleanup)

metricsConverter, err := influx2otel.NewLineProtocolToOtelMetricsWithSeparator(common.NoopLogger{}, ".")
require.NoError(t, err)
plugin := &OpenTelemetry{
ServiceAddress: m.Address(),
Timeout: config.Duration(time.Second),
Headers: map[string]string{"test": "header1"},
Attributes: map[string]string{"attr-key": "attr-val"},
metricsConverter: metricsConverter,
otlpMetricClient: &gRPCClient{
grpcClientConn: m.GrpcClient(),
metricsServiceClient: pmetricotlp.NewGRPCClient(m.GrpcClient()),
},
Log: testutil.Logger{},
}

input := metric.New(
"mem",
map[string]string{
"foo": "bar",
"otel.library.name": "My Library Name",
"host.name": "potato",
},
map[string]interface{}{
"used_percent": 87.332,
},
time.Unix(0, 1622848686000000000),
telegraf.Gauge,
)

require.NoError(t, plugin.Write([]telegraf.Metric{input}))

got := m.GotMetrics()

marshaller := pmetric.JSONMarshaler{}
expectJSON, err := marshaller.MarshalMetrics(expect)
require.NoError(t, err)

gotJSON, err := marshaller.MarshalMetrics(got)
require.NoError(t, err)

require.JSONEq(t, string(expectJSON), string(gotJSON))
}

func TestOpenTelemetrySeparatorHTTPProtobuf(t *testing.T) {
expect := pmetric.NewMetrics()
{
rm := expect.ResourceMetrics().AppendEmpty()
rm.Resource().Attributes().PutStr("host.name", "potato")
rm.Resource().Attributes().PutStr("attr-key", "attr-val")
ilm := rm.ScopeMetrics().AppendEmpty()
ilm.Scope().SetName("My Library Name")
m := ilm.Metrics().AppendEmpty()
m.SetName("mem.used_percent")
m.SetEmptyGauge()
dp := m.Gauge().DataPoints().AppendEmpty()
dp.Attributes().PutStr("foo", "bar")
dp.SetTimestamp(pcommon.Timestamp(1622848686000000000))
dp.SetDoubleValue(87.332)
}

var receivedMetrics pmetric.Metrics
var receivedContentType string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
receivedContentType = r.Header.Get("Content-Type")
body, err := io.ReadAll(r.Body)
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}

req := pmetricotlp.NewExportRequest()
err = req.UnmarshalProto(body)
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}

receivedMetrics = pmetric.NewMetrics()
req.Metrics().CopyTo(receivedMetrics)

resp := pmetricotlp.NewExportResponse()
respBytes, err := resp.MarshalProto()
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}

w.Header().Set("Content-Type", "application/x-protobuf")
w.WriteHeader(http.StatusOK)
_, err = w.Write(respBytes)
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}
}))
defer server.Close()

plugin := &OpenTelemetry{
ServiceAddress: server.URL,
EncodingType: "protobuf",
Timeout: config.Duration(time.Second),
Attributes: map[string]string{"attr-key": "attr-val"},
Compression: "none",
OtelNameSeparator: ".",
Log: testutil.Logger{},
}
require.NoError(t, plugin.Connect())

input := metric.New(
"mem",
map[string]string{
"foo": "bar",
"otel.library.name": "My Library Name",
"host.name": "potato",
},
map[string]interface{}{
"used_percent": 87.332,
},
time.Unix(0, 1622848686000000000),
telegraf.Gauge,
)

require.NoError(t, plugin.Write([]telegraf.Metric{input}))

require.Equal(t, "application/x-protobuf", receivedContentType)

marshaller := pmetric.JSONMarshaler{}
expectJSON, err := marshaller.MarshalMetrics(expect)
require.NoError(t, err)

gotJSON, err := marshaller.MarshalMetrics(receivedMetrics)
require.NoError(t, err)

require.JSONEq(t, string(expectJSON), string(gotJSON))
}

func TestOpenTelemetrySeparatorHTTPJSON(t *testing.T) {

@skartikey skartikey Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the io.ReadAll error branch returns without w.WriteHeader(http.StatusInternalServerError), unlike the protobuf variant.

@mchrome mchrome Jul 6, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 9fd0978

expect := pmetric.NewMetrics()
{
rm := expect.ResourceMetrics().AppendEmpty()
rm.Resource().Attributes().PutStr("host.name", "potato")
rm.Resource().Attributes().PutStr("attr-key", "attr-val")
ilm := rm.ScopeMetrics().AppendEmpty()
ilm.Scope().SetName("My Library Name")
m := ilm.Metrics().AppendEmpty()
m.SetName("mem.used_percent")
m.SetEmptyGauge()
dp := m.Gauge().DataPoints().AppendEmpty()
dp.Attributes().PutStr("foo", "bar")
dp.SetTimestamp(pcommon.Timestamp(1622848686000000000))
dp.SetDoubleValue(87.332)
}

var receivedMetrics pmetric.Metrics
var receivedContentType string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
receivedContentType = r.Header.Get("Content-Type")
body, err := io.ReadAll(r.Body)
if err != nil {
t.Error(err)
return
}

req := pmetricotlp.NewExportRequest()
err = req.UnmarshalJSON(body)
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}

receivedMetrics = pmetric.NewMetrics()
req.Metrics().CopyTo(receivedMetrics)

resp := pmetricotlp.NewExportResponse()
respBytes, err := resp.MarshalJSON()
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}

w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
_, err = w.Write(respBytes)
if err != nil {
t.Error(err)
w.WriteHeader(http.StatusInternalServerError)
return
}
}))
defer server.Close()

plugin := &OpenTelemetry{
ServiceAddress: server.URL,
EncodingType: "json",
Timeout: config.Duration(time.Second),
Attributes: map[string]string{"attr-key": "attr-val"},
Compression: "none",
OtelNameSeparator: ".",
Log: testutil.Logger{},
}
require.NoError(t, plugin.Connect())

input := metric.New(
"mem",
map[string]string{
"foo": "bar",
"otel.library.name": "My Library Name",
"host.name": "potato",
},
map[string]interface{}{
"used_percent": 87.332,
},
time.Unix(0, 1622848686000000000),
telegraf.Gauge,
)

require.NoError(t, plugin.Write([]telegraf.Metric{input}))

require.Equal(t, "application/json", receivedContentType)

marshaller := pmetric.JSONMarshaler{}
expectJSON, err := marshaller.MarshalMetrics(expect)
require.NoError(t, err)

gotJSON, err := marshaller.MarshalMetrics(receivedMetrics)
require.NoError(t, err)

require.JSONEq(t, string(expectJSON), string(gotJSON))
}

var _ pmetricotlp.GRPCServer = (*mockOtelService)(nil)

type mockOtelService struct {
Expand Down
Loading