Skip to content

Commit 85daf49

Browse files
[scraper/scraperhelper] Add scraper ID to error logs (#14461)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description This PR adds the `scraperID` to the error logs for scrapers (metrics, logs, and profiles). Why is this important? Currently if an error occurs in any of the scrapers in `hostmetricsreceiver/internal/scraper/...` there is no info about the which scraper errored in the logs: ```bash 2026-01-21T10:42:42.830Z error [email protected]/obs_metrics.go:61 Error scraping metrics {"resource": {"service.instance.id": "63071771-056d-40ca-83e3-585dc1a00d57", "service.name": "otelcontribcol", "service.version": "0.144.0-dev"}, "otelcol.component.id": "hostmetrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "error": "forced test error for scraper identification"} ``` The update in this PR add's the scraper ID (In this case the memory scraper errored) to the logs which then gives us: ```bash 2026-01-21T10:23:52.759Z error scraperhelper/obs_metrics.go:62 Error scraping metrics {"resource": {"service.instance.id": "7137f19d-706f-4095-8367-0d68b2732402", "service.name": "otelcontribcol", "service.version": "0.144.0-dev"}, "otelcol.component.id": "hostmetrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "scraper": "memory", "error": "forced test error for scraper identification"} ``` <!-- Issue number if applicable --> #### Link to tracking issue Fixes Issue in [collector-contrib](open-telemetry/opentelemetry-collector-contrib#35814 (comment)). <!--Describe what testing was performed and which tests were added.--> #### Testing **Metrics scrapers:** Locally tested with `collector-contrib` by updating the mod file... **Logs scrapers:** The same pattern was applied. Logs scrapers (MySQL, PostgreSQL, Oracle, SQL Server receivers) require database connections for integration testing. **Profiles scrapers (xscraperhelper):** The same pattern was applied. Note: There are currently no profiles scrapers in collector-contrib to test against. **Metrics scraper local test**: Locally this was updated and included in the `collector-contrib` via updating the mod file. An error was forced in the `hostmetricsreceiver/internal/scraper/...` both memory and cpu, we can now see which scraper component had an error: ```bash 2026-01-21T11:51:34.176Z error scraperhelper/obs_metrics.go:61 Error scraping metrics {"resource": {"service.instance.id": "e6832705-bc54-43dd-9b1f-0c06d58b7ead", "service.name": "otelcontribcol", "service.version": "0.144.0-dev"}, "otelcol.component.id": "hostmetrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "scraper": "memory", "error": "forced test error for scraper identification"} go.opentelemetry.io/collector/scraper/scraperhelper.wrapObsMetrics.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/obs_metrics.go:61 go.opentelemetry.io/collector/scraper.ScrapeMetricsFunc.ScrapeMetrics /home/dos/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/metrics.go:24 go.opentelemetry.io/collector/scraper/scraperhelper.scrapeMetrics /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/controller.go:167 go.opentelemetry.io/collector/scraper/scraperhelper.NewMetricsController.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/controller.go:139 go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller.(*Controller[...]).startScraping.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/internal/controller/controller.go:118 2026-01-21T11:51:34.177Z error scraperhelper/obs_metrics.go:61 Error scraping metrics {"resource": {"service.instance.id": "e6832705-bc54-43dd-9b1f-0c06d58b7ead", "service.name": "otelcontribcol", "service.version": "0.144.0-dev"}, "otelcol.component.id": "hostmetrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "scraper": "cpu", "error": "forced test error for scraper identification"} go.opentelemetry.io/collector/scraper/scraperhelper.wrapObsMetrics.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/obs_metrics.go:61 go.opentelemetry.io/collector/scraper.ScrapeMetricsFunc.ScrapeMetrics /home/dos/go/pkg/mod/go.opentelemetry.io/collector/[email protected]/metrics.go:24 go.opentelemetry.io/collector/scraper/scraperhelper.scrapeMetrics /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/controller.go:167 go.opentelemetry.io/collector/scraper/scraperhelper.NewMetricsController.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/controller.go:139 go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller.(*Controller[...]).startScraping.func1 /home/dos/Documents/opentelemetry-collector/scraper/scraperhelper/internal/controller/controller.go:118 2026-01-21T11:51:34.199Z info Metrics {"resource": {"service.instance.id": "e6832705-bc54-43dd-9b1f-0c06d58b7ead", "service.name": "otelcontribcol", "service.version": "0.144.0-dev"}, "otelcol.component.id": "debug", "otelcol.component.kind": "exporter", "otelcol.signal": "metrics", "resource metrics": 1, "metrics": 1, "data points": 1} 2026-01-21T11:51:34.199Z info ResourceMetrics #0 Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0 ``` Closes open-telemetry/opentelemetry-collector-contrib#35814
1 parent c4695a7 commit 85daf49

File tree

6 files changed

+179
-2
lines changed

6 files changed

+179
-2
lines changed
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. receiver/otlp)
7+
component: pkg/scraperhelper
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: ScraperID has been added to the logs for metrics, logs, and profiles
11+
12+
# One or more tracking issues or pull requests related to the change
13+
issues: [14461]

scraper/scraperhelper/controller_test.go

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ import (
1616
"go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest"
1717
sdktrace "go.opentelemetry.io/otel/sdk/trace"
1818
"go.uber.org/multierr"
19+
"go.uber.org/zap"
20+
"go.uber.org/zap/zaptest/observer"
1921

2022
"go.opentelemetry.io/collector/component"
2123
"go.opentelemetry.io/collector/component/componenttest"
@@ -763,3 +765,59 @@ func TestNewDefaultControllerConfig(t *testing.T) {
763765
intControllerConfig := controller.NewDefaultControllerConfig()
764766
require.Equal(t, intControllerConfig, controllerConfig)
765767
}
768+
769+
func TestNewMetricsController_ScraperIDInErrorLogs(t *testing.T) {
770+
t.Parallel()
771+
772+
core, observedLogs := observer.New(zap.ErrorLevel)
773+
tel := componenttest.NewTelemetry()
774+
t.Cleanup(func() { require.NoError(t, tel.Shutdown(context.Background())) })
775+
telset := tel.NewTelemetrySettings()
776+
telset.Logger = zap.New(core)
777+
778+
receiverID := component.MustNewID("fakeReceiver")
779+
scraperType := component.MustNewType("fakeScraper")
780+
scrapeErr := errors.New("scrape error")
781+
782+
scrapeCh := make(chan int, 1)
783+
ts := &testScrape{ch: scrapeCh, err: scrapeErr}
784+
scp, err := scraper.NewMetrics(ts.scrapeMetrics)
785+
require.NoError(t, err)
786+
787+
cfg := newTestNoDelaySettings()
788+
tickerCh := make(chan time.Time)
789+
790+
recv, err := NewMetricsController(
791+
cfg,
792+
receiver.Settings{ID: receiverID, TelemetrySettings: telset, BuildInfo: component.NewDefaultBuildInfo()},
793+
new(consumertest.MetricsSink),
794+
AddMetricsScraper(scraperType, scp),
795+
WithTickerChannel(tickerCh),
796+
)
797+
require.NoError(t, err)
798+
require.NoError(t, recv.Start(context.Background(), componenttest.NewNopHost()))
799+
defer func() { require.NoError(t, recv.Shutdown(context.Background())) }()
800+
801+
<-scrapeCh
802+
803+
require.Eventually(t, func() bool {
804+
return observedLogs.Len() >= 1
805+
}, time.Second, 10*time.Millisecond)
806+
errorLogs := observedLogs.FilterLevelExact(zap.ErrorLevel).All()
807+
require.Len(t, errorLogs, 1)
808+
809+
assert.Equal(t, "Error scraping metrics", errorLogs[0].Message)
810+
assert.Equal(t, scraperType.String(), errorLogs[0].ContextMap()["scraper"])
811+
assert.Equal(t, scrapeErr.Error(), errorLogs[0].ContextMap()["error"])
812+
813+
// Verify the original receiver telemetry settings logger was NOT mutated
814+
// by logging something and checking it doesn't have the scraper field
815+
telset.Logger.Error("test log from receiver")
816+
817+
allLogs := observedLogs.FilterLevelExact(zap.ErrorLevel).All()
818+
require.Len(t, allLogs, 2)
819+
820+
receiverLog := allLogs[1]
821+
assert.Equal(t, "test log from receiver", receiverLog.Message)
822+
assert.NotContains(t, receiverLog.ContextMap(), "scraper")
823+
}

scraper/scraperhelper/internal/controller/controller.go

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import (
1111
"time"
1212

1313
"go.uber.org/multierr"
14+
"go.uber.org/zap"
1415

1516
"go.opentelemetry.io/collector/component"
1617
"go.opentelemetry.io/collector/receiver"
@@ -124,9 +125,12 @@ func (sc *Controller[T]) startScraping() {
124125
}
125126

126127
func GetSettings(sType component.Type, rSet receiver.Settings) scraper.Settings {
128+
id := component.NewID(sType)
129+
telemetry := rSet.TelemetrySettings
130+
telemetry.Logger = telemetry.Logger.With(zap.String("scraper", id.String()))
127131
return scraper.Settings{
128-
ID: component.NewID(sType),
129-
TelemetrySettings: rSet.TelemetrySettings,
132+
ID: id,
133+
TelemetrySettings: telemetry,
130134
BuildInfo: rSet.BuildInfo,
131135
}
132136
}

scraper/scraperhelper/obs_logs_test.go

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,16 @@ import (
1414
"go.opentelemetry.io/otel/codes"
1515
"go.opentelemetry.io/otel/sdk/metric/metricdata"
1616
"go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest"
17+
"go.uber.org/zap"
18+
"go.uber.org/zap/zaptest/observer"
1719

1820
"go.opentelemetry.io/collector/component"
1921
"go.opentelemetry.io/collector/component/componenttest"
2022
"go.opentelemetry.io/collector/pdata/plog"
2123
"go.opentelemetry.io/collector/pdata/testdata"
24+
"go.opentelemetry.io/collector/receiver"
2225
"go.opentelemetry.io/collector/scraper"
26+
"go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller"
2327
"go.opentelemetry.io/collector/scraper/scraperhelper/internal/metadatatest"
2428
)
2529

@@ -97,6 +101,36 @@ func TestCheckScraperLogs(t *testing.T) {
97101
checkScraperLogs(t, tel, receiverID, scraperID, 7, 0)
98102
}
99103

104+
func TestScrapeLogsDataOp_LogsScraperID(t *testing.T) {
105+
tel := componenttest.NewTelemetry()
106+
t.Cleanup(func() { require.NoError(t, tel.Shutdown(context.Background())) })
107+
108+
core, observedLogs := observer.New(zap.ErrorLevel)
109+
telset := tel.NewTelemetrySettings()
110+
telset.Logger = zap.New(core)
111+
112+
rSet := receiver.Settings{
113+
ID: receiverID,
114+
TelemetrySettings: telset,
115+
}
116+
set := controller.GetSettings(scraperID.Type(), rSet)
117+
118+
sm, err := scraper.NewLogs(func(context.Context) (plog.Logs, error) {
119+
return plog.NewLogs(), errFake
120+
})
121+
require.NoError(t, err)
122+
sf, err := wrapObsLogs(sm, receiverID, scraperID, set.TelemetrySettings)
123+
require.NoError(t, err)
124+
_, err = sf.ScrapeLogs(context.Background())
125+
require.ErrorIs(t, err, errFake)
126+
127+
errorLogs := observedLogs.FilterLevelExact(zap.ErrorLevel).All()
128+
require.Len(t, errorLogs, 1)
129+
assert.Equal(t, "Error scraping logs", errorLogs[0].Message)
130+
assert.Equal(t, scraperID.String(), errorLogs[0].ContextMap()["scraper"])
131+
assert.Equal(t, errFake.Error(), errorLogs[0].ContextMap()["error"])
132+
}
133+
100134
func checkScraperLogs(t *testing.T, tel *componenttest.Telemetry, receiver, scraper component.ID, scrapedLogRecords, erroredLogRecords int64) {
101135
metadatatest.AssertEqualScraperScrapedLogRecords(t, tel,
102136
[]metricdata.DataPoint[int64]{

scraper/scraperhelper/obs_metrics_test.go

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,17 @@ import (
1414
"go.opentelemetry.io/otel/codes"
1515
"go.opentelemetry.io/otel/sdk/metric/metricdata"
1616
"go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest"
17+
"go.uber.org/zap"
18+
"go.uber.org/zap/zaptest/observer"
1719

1820
"go.opentelemetry.io/collector/component"
1921
"go.opentelemetry.io/collector/component/componenttest"
2022
"go.opentelemetry.io/collector/pdata/pmetric"
2123
"go.opentelemetry.io/collector/pdata/testdata"
24+
"go.opentelemetry.io/collector/receiver"
2225
"go.opentelemetry.io/collector/scraper"
2326
"go.opentelemetry.io/collector/scraper/scrapererror"
27+
"go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller"
2428
"go.opentelemetry.io/collector/scraper/scraperhelper/internal/metadatatest"
2529
)
2630

@@ -110,6 +114,36 @@ func TestCheckScraperMetrics(t *testing.T) {
110114
checkScraperMetrics(t, tel, receiverID, scraperID, 7, 0)
111115
}
112116

117+
func TestScrapeMetricsDataOp_LogsScraperID(t *testing.T) {
118+
tel := componenttest.NewTelemetry()
119+
t.Cleanup(func() { require.NoError(t, tel.Shutdown(context.Background())) })
120+
121+
core, observedLogs := observer.New(zap.ErrorLevel)
122+
telset := tel.NewTelemetrySettings()
123+
telset.Logger = zap.New(core)
124+
125+
rSet := receiver.Settings{
126+
ID: receiverID,
127+
TelemetrySettings: telset,
128+
}
129+
set := controller.GetSettings(scraperID.Type(), rSet)
130+
131+
sm, err := scraper.NewMetrics(func(context.Context) (pmetric.Metrics, error) {
132+
return pmetric.NewMetrics(), errFake
133+
})
134+
require.NoError(t, err)
135+
sf, err := wrapObsMetrics(sm, receiverID, scraperID, set.TelemetrySettings)
136+
require.NoError(t, err)
137+
_, err = sf.ScrapeMetrics(context.Background())
138+
require.ErrorIs(t, err, errFake)
139+
140+
errorLogs := observedLogs.FilterLevelExact(zap.ErrorLevel).All()
141+
require.Len(t, errorLogs, 1)
142+
assert.Equal(t, "Error scraping metrics", errorLogs[0].Message)
143+
assert.Equal(t, scraperID.String(), errorLogs[0].ContextMap()["scraper"])
144+
assert.Equal(t, errFake.Error(), errorLogs[0].ContextMap()["error"])
145+
}
146+
113147
func checkScraperMetrics(t *testing.T, tt *componenttest.Telemetry, receiver, scraper component.ID, scrapedMetricPoints, erroredMetricPoints int64) {
114148
metadatatest.AssertEqualScraperScrapedMetricPoints(t, tt,
115149
[]metricdata.DataPoint[int64]{

scraper/scraperhelper/xscraperhelper/obs_profiles_test.go

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,16 @@ import (
1515
"go.opentelemetry.io/otel/codes"
1616
"go.opentelemetry.io/otel/sdk/metric/metricdata"
1717
"go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest"
18+
"go.uber.org/zap"
19+
"go.uber.org/zap/zaptest/observer"
1820

1921
"go.opentelemetry.io/collector/component"
2022
"go.opentelemetry.io/collector/component/componenttest"
2123
"go.opentelemetry.io/collector/pdata/pprofile"
2224
"go.opentelemetry.io/collector/pdata/testdata"
25+
"go.opentelemetry.io/collector/receiver"
2326
"go.opentelemetry.io/collector/scraper/scrapererror"
27+
"go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller"
2428
"go.opentelemetry.io/collector/scraper/scraperhelper/xscraperhelper/internal/metadatatest"
2529
"go.opentelemetry.io/collector/scraper/xscraper"
2630
)
@@ -112,6 +116,36 @@ func TestCheckScraperProfiles(t *testing.T) {
112116
checkScraperProfiles(t, tel, receiverID, scraperID, 7, 0)
113117
}
114118

119+
func TestScrapeProfilesDataOp_LogsScraperID(t *testing.T) {
120+
tel := componenttest.NewTelemetry()
121+
t.Cleanup(func() { require.NoError(t, tel.Shutdown(context.Background())) })
122+
123+
core, observedLogs := observer.New(zap.ErrorLevel)
124+
telset := tel.NewTelemetrySettings()
125+
telset.Logger = zap.New(core)
126+
127+
rSet := receiver.Settings{
128+
ID: receiverID,
129+
TelemetrySettings: telset,
130+
}
131+
set := controller.GetSettings(scraperID.Type(), rSet)
132+
133+
sm, err := xscraper.NewProfiles(func(context.Context) (pprofile.Profiles, error) {
134+
return pprofile.NewProfiles(), errFake
135+
})
136+
require.NoError(t, err)
137+
sf, err := wrapObsProfiles(sm, receiverID, scraperID, set.TelemetrySettings)
138+
require.NoError(t, err)
139+
_, err = sf.ScrapeProfiles(context.Background())
140+
require.ErrorIs(t, err, errFake)
141+
142+
errorLogs := observedLogs.FilterLevelExact(zap.ErrorLevel).All()
143+
require.Len(t, errorLogs, 1)
144+
assert.Equal(t, "Error scraping profiles", errorLogs[0].Message)
145+
assert.Equal(t, scraperID.String(), errorLogs[0].ContextMap()["scraper"])
146+
assert.Equal(t, errFake.Error(), errorLogs[0].ContextMap()["error"])
147+
}
148+
115149
func checkScraperProfiles(t *testing.T, tel *componenttest.Telemetry, receiver, scraper component.ID, scrapedProfileRecords, erroredProfileRecords int64) {
116150
metadatatest.AssertEqualScraperScrapedProfileRecords(t, tel,
117151
[]metricdata.DataPoint[int64]{

0 commit comments

Comments
 (0)