Skip to content

Commit 7154bf1

Browse files
zmoogcursoragent
zmoog
and
cursoragent
authored
[extension/azure_encoding] Add support for Administrative, Alert, Autoscale, Policy, Security, ServiceHealth, and ResourceHealth log categories. (open-telemetry#45699)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description This PR brings over the remaining Activity log categories from `translator/azurelogs`, aligning this component with the changes made in open-telemetry#44871. New log categories: - Administrative - Alert - Autoscale - Policy - Security - ServiceHealth - ResourceHealth <!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. #### Link to tracking issue Fixes n/a --> <!--Describe what testing was performed and which tests were added.--> #### Testing We added test cases for each category, leveraging the same log events found in translator/azurelogs for consistency. <!--Describe the documentation added.--> #### Documentation <!--Please delete paragraphs that you did not use before submitting.--> Added a section covering Identity (including its major subfields) along with dedicated sections for each new log category. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent b0c41a1 commit 7154bf1

27 files changed

Lines changed: 2039 additions & 115 deletions

‎.chloggen/zmoog_azureencodingextension_port-activitylogs-from-translator.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. receiver/filelog)
7+
component: extension/azure_encoding
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Add support for Administrative, Alert, Autoscale, Policy, Security, ServiceHealth, and ResourceHealth log categories.
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [45699]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: [user]

‎extension/encoding/azureencodingextension/internal/unmarshaler/logs/README.md‎

Lines changed: 166 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,66 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or
4141
| `durationMs` | `azure.operation.duration` | Log Attribute |
4242
| `callerIpAddress` | `network.peer.address` | Log Attribute |
4343
| `correlationId` | `azure.correlation_id` | Log Attribute |
44-
| `identity` | `azure.identity` | Log Attribute |
44+
| `identity` | see [Identity Field](#identity-field) below | Log Attribute |
4545
| `Level` | `log.SeverityNumber` | Log |
4646
| `properties` | see mapping for each Category below | mixed |
4747

48+
### Identity Field
49+
50+
The `identity` field has different structures across Azure log categories, so identity parsing is handled per-category:
51+
52+
- **Activity Logs**: Specific known fields are extracted into flat, semantically meaningful attributes (see below)
53+
- **Storage Logs**: Stored as a nested map under `azure.identity` (different structure with authorization as an array)
54+
- **Unknown/Generic categories**: Stored as a nested map under `azure.identity`
55+
56+
Only known useful fields are extracted to minimize the risk of accidentally including sensitive data.
57+
58+
#### Activity Log Identity
59+
60+
Activity Logs contain caller identity information with JWT claims from Azure AD/Entra ID tokens and authorization details.
61+
62+
##### Authorization Fields
63+
64+
| Azure identity Field | OpenTelemetry | OpenTelemetry Scope |
65+
|---------------------|---------------|---------------------|
66+
| `identity.authorization.scope` | `azure.identity.authorization.scope` | Log Attribute |
67+
| `identity.authorization.action` | `azure.identity.authorization.action` | Log Attribute |
68+
| `identity.authorization.evidence.role` | `azure.identity.authorization.evidence.role` | Log Attribute |
69+
| `identity.authorization.evidence.roleAssignmentScope` | `azure.identity.authorization.evidence.role.assignment.scope` | Log Attribute |
70+
| `identity.authorization.evidence.roleAssignmentId` | `azure.identity.authorization.evidence.role.assignment.id` | Log Attribute |
71+
| `identity.authorization.evidence.roleDefinitionId` | `azure.identity.authorization.evidence.role.definition.id` | Log Attribute |
72+
| `identity.authorization.evidence.principalId` | `azure.identity.authorization.evidence.principal.id` | Log Attribute |
73+
| `identity.authorization.evidence.principalType` | `azure.identity.authorization.evidence.principal.type` | Log Attribute |
74+
75+
##### Claims Fields
76+
77+
Unix timestamps (`exp`, `nbf`, `iat`) are converted to RFC3339 format.
78+
79+
| Azure identity.claims Field | OpenTelemetry | OpenTelemetry Scope |
80+
|----------------------------|---------------|---------------------|
81+
| `iss` | `azure.identity.issuer` | Log Attribute |
82+
| `sub` | `azure.identity.subject` | Log Attribute |
83+
| `aud` | `azure.identity.audience` | Log Attribute |
84+
| `exp` | `azure.identity.not_after` | Log Attribute |
85+
| `nbf` | `azure.identity.not_before` | Log Attribute |
86+
| `iat` | `azure.identity.created` | Log Attribute |
87+
| `http://schemas.microsoft.com/identity/claims/scope` | `azure.identity.scope` | Log Attribute |
88+
| `idtyp` | `azure.identity.type` | Log Attribute |
89+
| `appid` | `azure.identity.application.id` | Log Attribute |
90+
| `http://schemas.microsoft.com/claims/authnmethodsreferences` | `azure.identity.auth.methods.references` | Log Attribute |
91+
| `http://schemas.microsoft.com/identity/claims/identityprovider` | `azure.identity.provider` | Log Attribute |
92+
| `http://schemas.microsoft.com/identity/claims/objectidentifier` | `azure.identity.identifier.object` | Log Attribute |
93+
| `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier` | `user.id` | Log Attribute |
94+
| `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress` | `user.email` | Log Attribute |
95+
96+
#### Storage Log Identity
97+
98+
Storage Logs have a different identity structure containing authorization decisions as an array, token information, and requester details. The entire identity object is stored as a nested map under `azure.identity`.
99+
100+
#### Unknown/Generic Categories
101+
102+
For log categories where the identity structure is not known, the entire identity object is stored as a nested map under `azure.identity` to preserve all data.
103+
48104
## Application Gateway
49105

50106
### Application Gateway Access Logs (both v1 and v2)
@@ -510,18 +566,6 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or
510566
| `roleInstance` | `service.instance.id` | Resource Attribute |
511567
| - | `faas.invoked_provider`=`azure` | Log Attribute |
512568

513-
## Recommendation Logs
514-
515-
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
516-
|-------------------------------|---------------------------------------|---------------------|
517-
| `recommendationSchemaVersion` | `azure.recommendation.schema_version` | Log Attribute |
518-
| `recommendationCategory` | `azure.recommendation.category` | Log Attribute |
519-
| `recommendationImpact` | `azure.recommendation.impact` | Log Attribute |
520-
| `recommendationName` | `azure.recommendation.name` | Log Attribute |
521-
| `recommendationResourceLink` | `azure.recommendation.link` | Log Attribute |
522-
| `recommendationType` | `azure.recommendation.type` | Log Attribute |
523-
| `recommendationRisk` | `azure.recommendation.risk` | Log Attribute |
524-
525569
## Storage Blob Logs (StorageRead, StorageWrite, StorageDelete)
526570

527571
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
@@ -543,3 +587,112 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or
543587
| `tlsVersion` | `tls.protocol.name` + `tls.protocol.version`. If unparsable - `tls.protocol.original` | Log Attribute |
544588
| `objectKey` | `azure.storage.object.key` | Log Attribute |
545589
| `sourceAccessTier` | `azure.storage.source.access_tier` | Log Attribute |
590+
591+
## Activity Logs
592+
593+
Activity Logs are a type of Azure platform log that provides insight into subscription-level events. The following categories are supported:
594+
595+
### Administrative
596+
597+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
598+
|---------------------------|-------------------------------------|---------------------|
599+
| `entity` | `azure.administrative.entity` | Log Attribute |
600+
| `message` | `azure.administrative.message` | Log Attribute |
601+
| `hierarchy` | `azure.administrative.hierarchy` | Log Attribute |
602+
603+
### Alert
604+
605+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
606+
|---------------------------|-------------------------------------|---------------------|
607+
| `webHookUri` | `azure.alert.webhook.uri` | Log Attribute |
608+
| `RuleUri` | `azure.alert.rule.uri` | Log Attribute |
609+
| `RuleName` | `azure.alert.rule.name` | Log Attribute |
610+
| `RuleDescription` | `azure.alert.rule.description` | Log Attribute |
611+
| `Threshold` | `azure.alert.threshold` | Log Attribute |
612+
| `WindowSizeInMinutes` | `azure.alert.window_size_minutes` | Log Attribute |
613+
| `Aggregation` | `azure.alert.aggregation` | Log Attribute |
614+
| `Operator` | `azure.alert.operator` | Log Attribute |
615+
| `MetricName` | `azure.alert.metric.name` | Log Attribute |
616+
| `MetricUnit` | `azure.alert.metric.unit` | Log Attribute |
617+
618+
### Autoscale
619+
620+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
621+
|---------------------------|---------------------------------------------|---------------------|
622+
| `Description` | `azure.autoscale.description` | Log Attribute |
623+
| `ResourceName` | `azure.autoscale.resource.name` | Log Attribute |
624+
| `OldInstancesCount` | `azure.autoscale.instances.previous_count` | Log Attribute |
625+
| `NewInstancesCount` | `azure.autoscale.instances.count` | Log Attribute |
626+
| `LastScaleActionTime` | `azure.autoscale.resource.last_scale` | Log Attribute |
627+
628+
### Security
629+
630+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
631+
|---------------------------|-------------------------------------|---------------------|
632+
| `commandLine` | `process.command_line` | Log Attribute |
633+
| `processName` | `process.executable.path` | Log Attribute |
634+
| `userName` | `process.owner` | Log Attribute |
635+
| `UserSID` | `enduser.id` | Log Attribute |
636+
| `processId` | `process.pid` | Log Attribute |
637+
| `parentProcess id` | `process.parent_pid` | Log Attribute |
638+
| `accountLogonId` | `azure.security.account_logon_id` | Log Attribute |
639+
| `domainName` | `azure.security.domain_name` | Log Attribute |
640+
| `ActionTaken` | `azure.security.action_taken` | Log Attribute |
641+
| `Severity` | `azure.security.severity` | Log Attribute |
642+
643+
### Policy
644+
645+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
646+
|---------------------------|-------------------------------------|---------------------|
647+
| `isComplianceCheck` | `azure.policy.compliance_check` | Log Attribute |
648+
| `resourceLocation` | `azure.location` | Log Attribute |
649+
| `ancestors` | `azure.policy.ancestors` | Log Attribute |
650+
| `hierarchy` | `azure.policy.hierarchy` | Log Attribute |
651+
| `policies` | `azure.policy.policies` (parsed as structured array) | Log Attribute |
652+
653+
### Recommendation
654+
655+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
656+
|-------------------------------|---------------------------------------|---------------------|
657+
| `recommendationSchemaVersion` | `azure.recommendation.schema_version` | Log Attribute |
658+
| `recommendationCategory` | `azure.recommendation.category` | Log Attribute |
659+
| `recommendationImpact` | `azure.recommendation.impact` | Log Attribute |
660+
| `recommendationName` | `azure.recommendation.name` | Log Attribute |
661+
| `recommendationResourceLink` | `azure.recommendation.link` | Log Attribute |
662+
| `recommendationType` | `azure.recommendation.type` | Log Attribute |
663+
| `recommendationRisk` | `azure.recommendation.risk` | Log Attribute |
664+
665+
### Service Health
666+
667+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
668+
|---------------------------|---------------------------------------------|---------------------|
669+
| `title` | `azure.servicehealth.title` | Log Attribute |
670+
| `service` | `azure.servicehealth.service` | Log Attribute |
671+
| `region` | `azure.servicehealth.region` | Log Attribute |
672+
| `communication` | `azure.servicehealth.communication.body` | Log Attribute |
673+
| `communicationId` | `azure.servicehealth.communication.id` | Log Attribute |
674+
| `incidentType` | `azure.servicehealth.incident.type` | Log Attribute |
675+
| `trackingId` | `azure.servicehealth.tracking.id` | Log Attribute |
676+
| `impactStartTime` | `azure.servicehealth.impact.start` | Log Attribute |
677+
| `impactMitigationTime` | `azure.servicehealth.impact.mitigation` | Log Attribute |
678+
| `impactedServices` | `azure.servicehealth.impact.services` (parsed as structured array) | Log Attribute |
679+
| `impactType` | `azure.servicehealth.impact.type` | Log Attribute |
680+
| `impactCategory` | `azure.servicehealth.impact.category` | Log Attribute |
681+
| `defaultLanguageTitle` | `azure.servicehealth.default_language.title` | Log Attribute |
682+
| `defaultLanguageContent` | `azure.servicehealth.default_language.content` | Log Attribute |
683+
| `stage` | `azure.servicehealth.state` | Log Attribute |
684+
| `maintenanceId` | `azure.servicehealth.maintenance.id` | Log Attribute |
685+
| `maintenanceType` | `azure.servicehealth.maintenance.type` | Log Attribute |
686+
| `isHIR` | `azure.servicehealth.is_hir` | Log Attribute |
687+
| `IsSynthetic` | `azure.servicehealth.is_synthetic` | Log Attribute |
688+
689+
### Resource Health
690+
691+
| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope |
692+
|---------------------------|---------------------------------------------|---------------------|
693+
| `title` | `azure.resourcehealth.title` | Log Attribute |
694+
| `details` | `azure.resourcehealth.details` | Log Attribute |
695+
| `currentHealthStatus` | `azure.resourcehealth.state` | Log Attribute |
696+
| `previousHealthStatus` | `azure.resourcehealth.previous_state` | Log Attribute |
697+
| `type` | `azure.resourcehealth.type` | Log Attribute |
698+
| `cause` | `azure.resourcehealth.cause` | Log Attribute |

0 commit comments

Comments
 (0)