Skip to content

Commit 1f92d42

Browse files
authored
feat(receiver/azuremonitorreceiver)! : Collect metrics labels from Azure Resource tags (#41117)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Updated `append_tags_as_attributes` configuration type from boolean to array. It controls which Azure resource tags are added as resource attributes to the metrics. The values can be a list of specific tag names or `["*"]` to include all tags. <!-- Issue number (e.g. #1234) or full URL to issue, if applicable. --> #### Link to tracking issue Fixes #40988 <!--Describe what testing was performed and which tests were added.--> #### Testing Comprehensive unit tests have been added/updated to validate the changes introduced in this PR. All newly added code is covered with 100% unit test coverage to ensure correctness. Please note that I do not currently have access to an Azure account, so I have not been able to perform end-to-end testing against live Azure resources. All changes have been validated through unit tests. If verification with an actual Azure environment is required, please let me know, and I will coordinate accordingly. <!--Describe the documentation added.--> #### Documentation The `azuremonitorreceiver` README has been updated to reflect the configuration changes introduced in this PR <!--Please delete paragraphs that you did not use before submitting.-->
1 parent b4fbd61 commit 1f92d42

File tree

13 files changed

+1576
-22
lines changed

13 files changed

+1576
-22
lines changed
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
change_type: breaking
2+
3+
component: azuremonitorreceiver
4+
5+
note: |
6+
Updated `append_tags_as_attributes` configuration type from boolean to array. It controls which Azure resource tags are added as resource attributes to the metrics. The values can be a list of specific tag names or `["*"]` to include all tags.
7+
8+
issues: [40988]
9+
10+
subtext:
11+
12+
change_logs: [user]

receiver/azuremonitorreceiver/README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ The following settings are optional:
3737
- `cloud` (default = `AzureCloud`): defines which Azure cloud to use. Valid values: `AzureCloud`, `AzureUSGovernment`, `AzureChinaCloud`.
3838
- `dimensions.enabled` (default = `true`): allows to opt out from automatically split by all the dimensions of the resource type.
3939
- `dimensions.overrides` (default = `{}`): if dimensions are enabled, it allows you to specify a set of dimensions for a particular metric. This is a two levels map with first key being the resource type and second key being the metric name. Programmatic value should be used for metric name https://learn.microsoft.com/en-us/azure/azure-monitor/reference/metrics-index
40+
- `append_tags_as_attributes` (default = `[]`): Controls which Azure resource tags are added as resource attributes to the metrics. Can be a list of specific tag names or `["*"]` to include all tags.
4041
- `use_batch_api` (default = `false`): Use the batch API to fetch metrics. This is useful when the number of subscriptions is high and the API calls are rate limited.
4142
- `maximum_resources_per_batch` (default = 50): If batch is enabled, the maximum number of unique resource IDs to fetch per API call, current limit is 50 (as of 06/16/2025) https://learn.microsoft.com/en-us/azure/azure-monitor/metrics/migrate-to-batch-api?tabs=individual-response
4243

@@ -182,6 +183,18 @@ receivers:
182183
"NetworkRuleHit": [Reason, Status]
183184
```
184185

186+
Selectively including resource tags as attributes:
187+
188+
```yaml
189+
receivers:
190+
azuremonitor:
191+
# Include all tags
192+
append_tags_as_attributes: ["*"]
193+
194+
# Or include only specific tags
195+
append_tags_as_attributes: ["service", "environment"]
196+
```
197+
185198
## Metrics
186199

187200
Details about the metrics scraped by this receiver can be found in [Supported metrics with Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported). This receiver adds the prefix "azure_" to all scraped metrics.

receiver/azuremonitorreceiver/config.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ type Config struct {
252252
CacheResourcesDefinitions float64 `mapstructure:"cache_resources_definitions"`
253253
MaximumNumberOfMetricsInACall int `mapstructure:"maximum_number_of_metrics_in_a_call"`
254254
MaximumNumberOfRecordsPerResource int32 `mapstructure:"maximum_number_of_records_per_resource"`
255-
AppendTagsAsAttributes bool `mapstructure:"append_tags_as_attributes"`
255+
AppendTagsAsAttributes []string `mapstructure:"append_tags_as_attributes"`
256256
UseBatchAPI bool `mapstructure:"use_batch_api"`
257257
Dimensions DimensionsConfig `mapstructure:"dimensions"`
258258
MaximumResourcesPerBatch int `mapstructure:"maximum_resources_per_batch"`

receiver/azuremonitorreceiver/config_test.go

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,26 @@ func TestLoadConfig(t *testing.T) {
4646
return cfg
4747
}(),
4848
},
49+
{
50+
id: component.NewIDWithName(metadata.Type, "append_tags_all"),
51+
expected: func() component.Config {
52+
cfg := createDefaultConfig().(*Config)
53+
cfg.SubscriptionIDs = []string{"test"}
54+
cfg.Credentials = defaultCredentials
55+
cfg.AppendTagsAsAttributes = []string{"*"}
56+
return cfg
57+
}(),
58+
},
59+
{
60+
id: component.NewIDWithName(metadata.Type, "append_tags_specific"),
61+
expected: func() component.Config {
62+
cfg := createDefaultConfig().(*Config)
63+
cfg.SubscriptionIDs = []string{"test"}
64+
cfg.Credentials = defaultCredentials
65+
cfg.AppendTagsAsAttributes = []string{"service", "environment"}
66+
return cfg
67+
}(),
68+
},
4969
{
5070
id: component.NewIDWithName(metadata.Type, "missing_subscription"),
5171
expectedErr: errMissingSubscriptionIDs.Error(),

receiver/azuremonitorreceiver/scraper.go

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,8 @@ func (s *azureScraper) getResources(ctx context.Context, subscriptionID string)
279279
Filter: &filter,
280280
}
281281

282+
tagsFilterMap := getTagsFilterMap(s.cfg.AppendTagsAsAttributes)
283+
282284
pager := clientResources.NewListPager(opts)
283285

284286
for pager.More() {
@@ -300,7 +302,7 @@ func (s *azureScraper) getResources(ctx context.Context, subscriptionID string)
300302
}
301303
s.resources[subscriptionID][*resource.ID] = &azureResource{
302304
attributes: attributes,
303-
tags: resource.Tags,
305+
tags: filterResourceTags(tagsFilterMap, resource.Tags),
304306
resourceType: resource.Type,
305307
}
306308
}
@@ -449,11 +451,9 @@ func (s *azureScraper) getResourceMetricsValues(ctx context.Context, subscriptio
449451
name := metadataPrefix + *value.Name.Value
450452
attributes[name] = value.Value
451453
}
452-
if s.cfg.AppendTagsAsAttributes {
453-
for tagName, value := range res.tags {
454-
name := tagPrefix + tagName
455-
attributes[name] = value
456-
}
454+
for tagName, value := range res.tags {
455+
name := tagPrefix + tagName
456+
attributes[name] = value
457457
}
458458
for _, metricValue := range timeseriesElement.Data {
459459
s.processTimeseriesData(resourceID, metric, metricValue, attributes)
@@ -564,3 +564,30 @@ func mapFindInsensitive[T any](m map[string]T, key string) (T, bool) {
564564
var got T
565565
return got, false
566566
}
567+
568+
// getTagsFilterMap returns a map used to filter tags.
569+
// Each user-configured tag key is normalized to lowercase and added to the map for case-insensitive lookup.
570+
func getTagsFilterMap(appendTagsAsAttributes []string) (tagsFilterMap map[string]struct{}) {
571+
tagsFilterMap = make(map[string]struct{}, len(appendTagsAsAttributes))
572+
for _, v := range appendTagsAsAttributes {
573+
tagsFilterMap[strings.ToLower(v)] = struct{}{}
574+
}
575+
return tagsFilterMap
576+
}
577+
578+
// filterResourceTags filter out resource tags according to configured tag list (append_tags_as_attributes)
579+
func filterResourceTags(tagFilterList map[string]struct{}, resourceTags map[string]*string) map[string]*string {
580+
if _, includeAll := tagFilterList["*"]; includeAll {
581+
return resourceTags
582+
}
583+
584+
// wildcard not found. include only configured tags
585+
includedTags := make(map[string]*string, len(resourceTags))
586+
for tagName, value := range resourceTags {
587+
if _, ok := tagFilterList[strings.ToLower(tagName)]; ok {
588+
includedTags[tagName] = value
589+
}
590+
}
591+
592+
return includedTags
593+
}

receiver/azuremonitorreceiver/scraper_batch.go

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,8 @@ func (s *azureBatchScraper) getResourcesAndTypes(ctx context.Context, subscripti
233233
Filter: &filter,
234234
}
235235

236+
tagsFilterMap := getTagsFilterMap(s.cfg.AppendTagsAsAttributes)
237+
236238
resourceTypes := map[string]*azureType{}
237239
pager := clientResources.NewListPager(opts)
238240

@@ -256,7 +258,7 @@ func (s *azureBatchScraper) getResourcesAndTypes(ctx context.Context, subscripti
256258
}
257259
s.resources[subscriptionID][*resource.ID] = &azureResource{
258260
attributes: attributes,
259-
tags: resource.Tags,
261+
tags: filterResourceTags(tagsFilterMap, resource.Tags),
260262
resourceType: resource.Type,
261263
}
262264
if resourceTypes[*resource.Type] == nil {
@@ -449,11 +451,9 @@ func (s *azureBatchScraper) getBatchMetricsValues(ctx context.Context, subscript
449451
name := metadataPrefix + *value.Name.Value
450452
attributes[name] = value.Value
451453
}
452-
if s.cfg.AppendTagsAsAttributes {
453-
for tagName, value := range res.tags {
454-
name := tagPrefix + tagName
455-
attributes[name] = value
456-
}
454+
for tagName, value := range res.tags {
455+
name := tagPrefix + tagName
456+
attributes[name] = value
457457
}
458458
attributes["timegrain"] = &compositeKey.timeGrain
459459
for i := len(timeseriesElement.Data) - 1; i >= 0; i-- { // reverse for loop because newest timestamp is at the end of the slice

receiver/azuremonitorreceiver/scraper_batch_test.go

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -219,10 +219,21 @@ func TestAzureScraperBatchScrape(t *testing.T) {
219219
}
220220
cfg := createDefaultTestConfig()
221221
cfg.MaximumNumberOfMetricsInACall = 2
222+
cfg.AppendTagsAsAttributes = []string{}
222223
cfg.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
223224

225+
cfgTagsSelective := createDefaultTestConfig()
226+
cfgTagsSelective.AppendTagsAsAttributes = []string{"tagName1"}
227+
cfgTagsSelective.MaximumNumberOfMetricsInACall = 2
228+
cfgTagsSelective.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
229+
230+
cfgTagsCaseInsensitive := createDefaultTestConfig()
231+
cfgTagsCaseInsensitive.AppendTagsAsAttributes = []string{"TAGNAME1"}
232+
cfgTagsCaseInsensitive.MaximumNumberOfMetricsInACall = 2
233+
cfgTagsCaseInsensitive.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
234+
224235
cfgTagsEnabled := createDefaultTestConfig()
225-
cfgTagsEnabled.AppendTagsAsAttributes = true
236+
cfgTagsEnabled.AppendTagsAsAttributes = []string{"*"}
226237
cfgTagsEnabled.MaximumNumberOfMetricsInACall = 2
227238

228239
tests := []struct {
@@ -249,6 +260,24 @@ func TestAzureScraperBatchScrape(t *testing.T) {
249260
ctx: context.Background(),
250261
},
251262
},
263+
{
264+
name: "metrics_selective_tags",
265+
fields: fields{
266+
cfg: cfgTagsSelective,
267+
},
268+
args: args{
269+
ctx: context.Background(),
270+
},
271+
},
272+
{
273+
name: "metrics_selective_tags",
274+
fields: fields{
275+
cfg: cfgTagsCaseInsensitive,
276+
},
277+
args: args{
278+
ctx: context.Background(),
279+
},
280+
},
252281
}
253282

254283
for _, tt := range tests {
@@ -258,7 +287,7 @@ func TestAzureScraperBatchScrape(t *testing.T) {
258287
optionsResolver := newMockClientOptionsResolver(
259288
getSubscriptionByIDMockData(),
260289
getSubscriptionsMockData(),
261-
getResourcesMockData(tt.fields.cfg.AppendTagsAsAttributes),
290+
getResourcesMockData(),
262291
getMetricsDefinitionsMockData(),
263292
nil,
264293
getMetricsQueryResponseMockData(),

receiver/azuremonitorreceiver/scraper_test.go

Lines changed: 41 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -94,17 +94,29 @@ func TestAzureScraperScrape(t *testing.T) {
9494
}
9595
cfg := createDefaultTestConfig()
9696
cfg.MaximumNumberOfMetricsInACall = 2
97+
cfg.AppendTagsAsAttributes = []string{}
9798
cfg.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
9899

99100
cfgTagsEnabled := createDefaultTestConfig()
100-
cfgTagsEnabled.AppendTagsAsAttributes = true
101+
cfgTagsEnabled.AppendTagsAsAttributes = []string{"*"}
101102
cfgTagsEnabled.MaximumNumberOfMetricsInACall = 2
102103
cfgTagsEnabled.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
103104

105+
cfgTagsSelective := createDefaultTestConfig()
106+
cfgTagsSelective.AppendTagsAsAttributes = []string{"tagName1"}
107+
cfgTagsSelective.MaximumNumberOfMetricsInACall = 2
108+
cfgTagsSelective.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
109+
104110
cfgSubNameAttr := createDefaultTestConfig()
111+
cfgSubNameAttr.AppendTagsAsAttributes = []string{}
105112
cfgSubNameAttr.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
106113
cfgSubNameAttr.MetricsBuilderConfig.ResourceAttributes.AzuremonitorSubscription.Enabled = true
107114

115+
cfgTagsCaseInsensitive := createDefaultTestConfig()
116+
cfgTagsCaseInsensitive.AppendTagsAsAttributes = []string{"TAGNAME1"}
117+
cfgTagsCaseInsensitive.MaximumNumberOfMetricsInACall = 2
118+
cfgTagsCaseInsensitive.SubscriptionIDs = []string{"subscriptionId1", "subscriptionId3"}
119+
108120
tests := []struct {
109121
name string
110122
fields fields
@@ -129,6 +141,15 @@ func TestAzureScraperScrape(t *testing.T) {
129141
ctx: context.Background(),
130142
},
131143
},
144+
{
145+
name: "metrics_selective_tags",
146+
fields: fields{
147+
cfg: cfgTagsSelective,
148+
},
149+
args: args{
150+
ctx: context.Background(),
151+
},
152+
},
132153
{
133154
name: "metrics_subname_golden",
134155
fields: fields{
@@ -138,6 +159,15 @@ func TestAzureScraperScrape(t *testing.T) {
138159
ctx: context.Background(),
139160
},
140161
},
162+
{
163+
name: "metrics_selective_tags",
164+
fields: fields{
165+
cfg: cfgTagsCaseInsensitive,
166+
},
167+
args: args{
168+
ctx: context.Background(),
169+
},
170+
},
141171
}
142172

143173
for _, tt := range tests {
@@ -147,7 +177,7 @@ func TestAzureScraperScrape(t *testing.T) {
147177
optionsResolver := newMockClientOptionsResolver(
148178
getSubscriptionByIDMockData(),
149179
getSubscriptionsMockData(),
150-
getResourcesMockData(tt.fields.cfg.AppendTagsAsAttributes),
180+
getResourcesMockData(),
151181
getMetricsDefinitionsMockData(),
152182
getMetricsValuesMockData(),
153183
nil,
@@ -460,7 +490,7 @@ func getNominalTestScraper() *azureScraper {
460490
optionsResolver := newMockClientOptionsResolver(
461491
getSubscriptionByIDMockData(),
462492
getSubscriptionsMockData(),
463-
getResourcesMockData(false),
493+
getResourcesMockData(),
464494
getMetricsDefinitionsMockData(),
465495
getMetricsValuesMockData(),
466496
nil,
@@ -570,7 +600,7 @@ func getTimeMock() timeNowIface {
570600
return &timeMock{time: time.Now()}
571601
}
572602

573-
func getResourcesMockData(tags bool) map[string][]armresources.ClientListResponse {
603+
func getResourcesMockData() map[string][]armresources.ClientListResponse {
574604
id1, id2, id3, id4,
575605
location1, name1, type1 := "/subscriptions/subscriptionId1/resourceGroups/group1/resourceId1",
576606
"/subscriptions/subscriptionId1/resourceGroups/group1/resourceId2",
@@ -584,10 +614,14 @@ func getResourcesMockData(tags bool) map[string][]armresources.ClientListRespons
584614
Name: &name1,
585615
Type: &type1,
586616
}
587-
if tags {
588-
tagName1, tagValue1 := "tagName1", "tagValue1"
589-
resourceID1.Tags = map[string]*string{tagName1: &tagValue1}
617+
618+
tagName1, tagValue1 := "tagName1", "tagValue1"
619+
tagName2, tagValue2 := "tagName2", "tagValue2"
620+
resourceID1.Tags = map[string]*string{
621+
tagName1: &tagValue1,
622+
tagName2: &tagValue2,
590623
}
624+
591625
return map[string][]armresources.ClientListResponse{
592626
"subscriptionId1": {
593627
{

receiver/azuremonitorreceiver/testdata/config.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,15 @@ azuremonitor/max_resources_per_batch_negative_value:
4848
credentials: does-not-matter
4949
use_batch_api: true
5050
maximum_resources_per_batch: -1
51+
52+
azuremonitor/append_tags_all:
53+
subscription_ids:
54+
- test
55+
credentials: default_credentials
56+
append_tags_as_attributes: ["*"]
57+
58+
azuremonitor/append_tags_specific:
59+
subscription_ids:
60+
- test
61+
credentials: default_credentials
62+
append_tags_as_attributes: ["service", "environment"]

0 commit comments

Comments
 (0)