diff --git a/.chloggen/zmoog_azureencodingextension_port-activitylogs-from-translator.yaml b/.chloggen/zmoog_azureencodingextension_port-activitylogs-from-translator.yaml new file mode 100644 index 0000000000000..475d4ab3df941 --- /dev/null +++ b/.chloggen/zmoog_azureencodingextension_port-activitylogs-from-translator.yaml @@ -0,0 +1,27 @@ +# Use this changelog template to create an entry for release notes. + +# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' +change_type: enhancement + +# The name of the component, or a single word describing the area of concern, (e.g. receiver/filelog) +component: extension/azure_encoding + +# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). +note: Add support for Administrative, Alert, Autoscale, Policy, Security, ServiceHealth, and ResourceHealth log categories. + +# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. +issues: [45699] + +# (Optional) One or more lines of additional information to render under the primary note. +# These lines will be padded with 2 spaces and then inserted directly into the document. +# Use pipe (|) for multiline entries. +subtext: + +# If your change doesn't affect end users or the exported elements of any package, +# you should instead start your pull request title with [chore] or use the "Skip Changelog" label. +# Optional: The change log or logs in which this entry should be included. +# e.g. '[user]' or '[user, api]' +# Include 'user' if the change is relevant to end users. +# Include 'api' if there is a change to a library API. +# Default: '[user]' +change_logs: [user] diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/README.md b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/README.md index 090c3737be21b..dc8f72958c89e 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/README.md +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/README.md @@ -41,10 +41,50 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or | `durationMs` | `azure.operation.duration` | Log Attribute | | `callerIpAddress` | `network.peer.address` | Log Attribute | | `correlationId` | `azure.correlation_id` | Log Attribute | -| `identity` | `azure.identity` | Log Attribute | +| `identity` | see [Identity Field](#identity-field) below | Log Attribute | | `Level` | `log.SeverityNumber` | Log | | `properties` | see mapping for each Category below | mixed | +### Identity Field + +The `identity` field contains authorization and claims information. Rather than storing it as a nested structure, +specific fields are extracted into flat, semantically meaningful attributes. Only known useful fields are extracted +to minimize the risk of accidentally including sensitive data. + +#### Authorization Fields + +| Azure identity Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------|---------------|---------------------| +| `identity.authorization.scope` | `azure.identity.authorization.scope` | Log Attribute | +| `identity.authorization.action` | `azure.identity.authorization.action` | Log Attribute | +| `identity.authorization.evidence.role` | `azure.identity.authorization.evidence.role` | Log Attribute | +| `identity.authorization.evidence.roleAssignmentScope` | `azure.identity.authorization.evidence.role.assignment.scope` | Log Attribute | +| `identity.authorization.evidence.roleAssignmentId` | `azure.identity.authorization.evidence.role.assignment.id` | Log Attribute | +| `identity.authorization.evidence.roleDefinitionId` | `azure.identity.authorization.evidence.role.definition.id` | Log Attribute | +| `identity.authorization.evidence.principalId` | `azure.identity.authorization.evidence.principal.id` | Log Attribute | +| `identity.authorization.evidence.principalType` | `azure.identity.authorization.evidence.principal.type` | Log Attribute | + +#### Claims Fields + +Unix timestamps (`exp`, `nbf`, `iat`) are converted to RFC3339 format. + +| Azure identity.claims Field | OpenTelemetry | OpenTelemetry Scope | +|----------------------------|---------------|---------------------| +| `iss` | `azure.identity.issuer` | Log Attribute | +| `sub` | `azure.identity.subject` | Log Attribute | +| `aud` | `azure.identity.audience` | Log Attribute | +| `exp` | `azure.identity.not_after` | Log Attribute | +| `nbf` | `azure.identity.not_before` | Log Attribute | +| `iat` | `azure.identity.created` | Log Attribute | +| `http://schemas.microsoft.com/identity/claims/scope` | `azure.identity.scope` | Log Attribute | +| `idtyp` | `azure.identity.type` | Log Attribute | +| `appid` | `azure.identity.application.id` | Log Attribute | +| `http://schemas.microsoft.com/claims/authnmethodsreferences` | `azure.identity.auth.methods.references` | Log Attribute | +| `http://schemas.microsoft.com/identity/claims/identityprovider` | `azure.identity.provider` | Log Attribute | +| `http://schemas.microsoft.com/identity/claims/objectidentifier` | `azure.identity.identifier.object` | Log Attribute | +| `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier` | `user.name` | Log Attribute | +| `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress` | `user.email` | Log Attribute | + ## Application Gateway ### Application Gateway Access Logs (both v1 and v2) @@ -510,18 +550,6 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or | `roleInstance` | `service.instance.id` | Resource Attribute | | - | `faas.invoked_provider`=`azure` | Log Attribute | -## Recommendation Logs - -| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | -|-------------------------------|---------------------------------------|---------------------| -| `recommendationSchemaVersion` | `azure.recommendation.schema_version` | Log Attribute | -| `recommendationCategory` | `azure.recommendation.category` | Log Attribute | -| `recommendationImpact` | `azure.recommendation.impact` | Log Attribute | -| `recommendationName` | `azure.recommendation.name` | Log Attribute | -| `recommendationResourceLink` | `azure.recommendation.link` | Log Attribute | -| `recommendationType` | `azure.recommendation.type` | Log Attribute | -| `recommendationRisk` | `azure.recommendation.risk` | Log Attribute | - ## Storage Blob Logs (StorageRead, StorageWrite, StorageDelete) | Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | @@ -543,3 +571,112 @@ in OpenTelemetry Collector pipeline (for example, using `transformprocessor`) or | `tlsVersion` | `tls.protocol.name` + `tls.protocol.version`. If unparsable - `tls.protocol.original` | Log Attribute | | `objectKey` | `azure.storage.object.key` | Log Attribute | | `sourceAccessTier` | `azure.storage.source.access_tier` | Log Attribute | + +## Activity Logs + +Activity Logs are a type of Azure platform log that provides insight into subscription-level events. The following categories are supported: + +### Administrative + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|-------------------------------------|---------------------| +| `entity` | `azure.administrative.entity` | Log Attribute | +| `message` | `azure.administrative.message` | Log Attribute | +| `hierarchy` | `azure.administrative.hierarchy` | Log Attribute | + +### Alert + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|-------------------------------------|---------------------| +| `webHookUri` | `azure.alert.webhook.uri` | Log Attribute | +| `RuleUri` | `azure.alert.rule.uri` | Log Attribute | +| `RuleName` | `azure.alert.rule.name` | Log Attribute | +| `RuleDescription` | `azure.alert.rule.description` | Log Attribute | +| `Threshold` | `azure.alert.threshold` | Log Attribute | +| `WindowSizeInMinutes` | `azure.alert.window_size_minutes` | Log Attribute | +| `Aggregation` | `azure.alert.aggregation` | Log Attribute | +| `Operator` | `azure.alert.operator` | Log Attribute | +| `MetricName` | `azure.alert.metric.name` | Log Attribute | +| `MetricUnit` | `azure.alert.metric.unit` | Log Attribute | + +### Autoscale + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|---------------------------------------------|---------------------| +| `Description` | `azure.autoscale.description` | Log Attribute | +| `ResourceName` | `azure.autoscale.resource.name` | Log Attribute | +| `OldInstancesCount` | `azure.autoscale.instances.previous_count` | Log Attribute | +| `NewInstancesCount` | `azure.autoscale.instances.count` | Log Attribute | +| `LastScaleActionTime` | `azure.autoscale.resource.last_scale` | Log Attribute | + +### Security + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|-------------------------------------|---------------------| +| `commandLine` | `process.command_line` | Log Attribute | +| `processName` | `process.executable.path` | Log Attribute | +| `userName` | `process.owner` | Log Attribute | +| `UserSID` | `enduser.id` | Log Attribute | +| `processId` | `process.pid` | Log Attribute | +| `parentProcess id` | `process.parent_pid` | Log Attribute | +| `accountLogonId` | `azure.security.account_logon_id` | Log Attribute | +| `domainName` | `azure.security.domain_name` | Log Attribute | +| `ActionTaken` | `azure.security.action_taken` | Log Attribute | +| `Severity` | `azure.security.severity` | Log Attribute | + +### Policy + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|-------------------------------------|---------------------| +| `isComplianceCheck` | `azure.policy.compliance_check` | Log Attribute | +| `resourceLocation` | `azure.location` | Log Attribute | +| `ancestors` | `azure.policy.ancestors` | Log Attribute | +| `hierarchy` | `azure.policy.hierarchy` | Log Attribute | +| `policies` | `azure.policy.policies` (parsed as structured array) | Log Attribute | + +### Recommendation + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|-------------------------------|---------------------------------------|---------------------| +| `recommendationSchemaVersion` | `azure.recommendation.schema_version` | Log Attribute | +| `recommendationCategory` | `azure.recommendation.category` | Log Attribute | +| `recommendationImpact` | `azure.recommendation.impact` | Log Attribute | +| `recommendationName` | `azure.recommendation.name` | Log Attribute | +| `recommendationResourceLink` | `azure.recommendation.link` | Log Attribute | +| `recommendationType` | `azure.recommendation.type` | Log Attribute | +| `recommendationRisk` | `azure.recommendation.risk` | Log Attribute | + +### Service Health + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|---------------------------------------------|---------------------| +| `title` | `azure.servicehealth.title` | Log Attribute | +| `service` | `azure.servicehealth.service` | Log Attribute | +| `region` | `azure.servicehealth.region` | Log Attribute | +| `communication` | `azure.servicehealth.communication.body` | Log Attribute | +| `communicationId` | `azure.servicehealth.communication.id` | Log Attribute | +| `incidentType` | `azure.servicehealth.incident.type` | Log Attribute | +| `trackingId` | `azure.servicehealth.tracking.id` | Log Attribute | +| `impactStartTime` | `azure.servicehealth.impact.start` | Log Attribute | +| `impactMitigationTime` | `azure.servicehealth.impact.mitigation` | Log Attribute | +| `impactedServices` | `azure.servicehealth.impact.services` (parsed as structured array) | Log Attribute | +| `impactType` | `azure.servicehealth.impact.type` | Log Attribute | +| `impactCategory` | `azure.servicehealth.impact.category` | Log Attribute | +| `defaultLanguageTitle` | `azure.servicehealth.default_language.title` | Log Attribute | +| `defaultLanguageContent` | `azure.servicehealth.default_language.content` | Log Attribute | +| `stage` | `azure.servicehealth.state` | Log Attribute | +| `maintenanceId` | `azure.servicehealth.maintenance.id` | Log Attribute | +| `maintenanceType` | `azure.servicehealth.maintenance.type` | Log Attribute | +| `isHIR` | `azure.servicehealth.is_hir` | Log Attribute | +| `IsSynthetic` | `azure.servicehealth.is_synthetic` | Log Attribute | + +### Resource Health + +| Azure "properties" Field | OpenTelemetry | OpenTelemetry Scope | +|---------------------------|---------------------------------------------|---------------------| +| `title` | `azure.resourcehealth.title` | Log Attribute | +| `details` | `azure.resourcehealth.details` | Log Attribute | +| `currentHealthStatus` | `azure.resourcehealth.state` | Log Attribute | +| `previousHealthStatus` | `azure.resourcehealth.previous_state` | Log Attribute | +| `type` | `azure.resourcehealth.type` | Log Attribute | +| `cause` | `azure.resourcehealth.cause` | Log Attribute | diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category.go b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category.go index 7b217746a50f3..7f32e4be0c2fa 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category.go +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category.go @@ -7,7 +7,9 @@ import ( "encoding/json" "errors" "fmt" + "time" + gojson "github.com/goccy/go-json" jsoniter "github.com/json-iterator/go" "go.opentelemetry.io/collector/pdata/pcommon" "go.opentelemetry.io/collector/pdata/plog" @@ -42,10 +44,17 @@ const ( categoryFrontDoorHealthProbeLog = "FrontDoorHealthProbeLog" categoryFrontdoorWebApplicationFirewallLog = "FrontDoorWebApplicationFirewallLog" categoryFunctionAppLogs = "FunctionAppLogs" - categoryRecommendation = "Recommendation" categoryStorageRead = "StorageRead" categoryStorageWrite = "StorageWrite" categoryStorageDelete = "StorageDelete" + categoryAdministrative = "Administrative" + categoryAlert = "Alert" + categoryAutoscale = "Autoscale" + categorySecurity = "Security" + categoryPolicy = "Policy" + categoryRecommendation = "Recommendation" + categoryResourceHealth = "ResourceHealth" + categoryServiceHealth = "ServiceHealth" ) // Non-SemConv attributes that are used for common Azure Log Record fields @@ -62,10 +71,6 @@ const ( // from `durationMs` field in Azure Log Record attributeAzureOperationDuration = "azure.operation.duration" - // OpenTelemetry attribute name for Azure Identity, - // from `identity` field in Azure Log Record - attributeAzureIdentity = "azure.identity" - // OpenTelemetry attribute name for Azure Log Record properties, // from `properties` field in Azure Log Record // Used when we cannot map parse "properties" field or @@ -105,6 +110,80 @@ const ( attributeAzureRequestDuration = "azure.request.duration" ) +// Constants for Identity > claims +const ( + identityClaimIssuer = "iss" + identityClaimSubject = "sub" + identityClaimAudience = "aud" + identityClaimExpires = "exp" + identityClaimNotBefore = "nbf" + identityClaimIssuedAt = "iat" + + identityClaimScope = "http://schemas.microsoft.com/identity/claims/scope" + identityClaimType = "idtyp" + identityClaimApplicationID = "appid" + identityClaimAuthMethodsReferences = "http://schemas.microsoft.com/claims/authnmethodsreferences" + identityClaimProvider = "http://schemas.microsoft.com/identity/claims/identityprovider" + identityClaimIdentifierObject = "http://schemas.microsoft.com/identity/claims/objectidentifier" + identityClaimIdentifierName = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier" + identityClaimEmailAddress = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" +) + +// Identity specific attributes +const ( + attributeIdentityAuthorizationScope = "azure.identity.authorization.scope" + attributeIdentityAuthorizationAction = "azure.identity.authorization.action" + // Identity > authorization > evidence + attributeIdentityAuthorizationEvidenceRole = "azure.identity.authorization.evidence.role" + attributeIdentityAuthorizationEvidenceRoleAssignmentScope = "azure.identity.authorization.evidence.role.assignment.scope" + attributeIdentityAuthorizationEvidenceRoleAssignmentID = "azure.identity.authorization.evidence.role.assignment.id" + attributeIdentityAuthorizationEvidenceRoleDefinitionID = "azure.identity.authorization.evidence.role.definition.id" + attributeIdentityAuthorizationEvidencePrincipalID = "azure.identity.authorization.evidence.principal.id" + attributeIdentityAuthorizationEvidencePrincipalType = "azure.identity.authorization.evidence.principal.type" + // Identity > claims (standard JWT claims) + attributeIdentityClaimsAudience = "azure.identity.audience" + attributeIdentityClaimsIssuer = "azure.identity.issuer" + attributeIdentityClaimsSubject = "azure.identity.subject" + attributeIdentityClaimsNotAfter = "azure.identity.not_after" + attributeIdentityClaimsNotBefore = "azure.identity.not_before" + attributeIdentityClaimsCreated = "azure.identity.created" + // Identity > claims (Azure specific claims) + attributeIdentityClaimsScope = "azure.identity.scope" + attributeIdentityClaimsType = "azure.identity.type" + attributeIdentityClaimsApplicationID = "azure.identity.application.id" + attributeIdentityClaimsAuthMethodsReferences = "azure.identity.auth.methods.references" + attributeIdentityClaimsIdentifierObject = "azure.identity.identifier.object" + attributeIdentityClaimsIdentifierName = "user.name" + attributeIdentityClaimsProvider = "azure.identity.provider" +) + +// evidence describes role assignment evidence in identity authorization +type evidence struct { + Role string `json:"role"` + RoleAssignmentScope string `json:"roleAssignmentScope"` + RoleAssignmentID string `json:"roleAssignmentId"` + RoleDefinitionID string `json:"roleDefinitionId"` + PrincipalID string `json:"principalId"` + PrincipalType string `json:"principalType"` +} + +// authorization describes identity authorization details +type authorization struct { + Scope string `json:"scope"` + Action string `json:"action"` + Evidence *evidence `json:"evidence"` +} + +// identity describes the identity of the user or application that performed the operation +// described by the log event. +type identity struct { + // Claims usually contains the JWT token used by Active Directory + // to authenticate the user or application to perform this + // operation in Resource Manager. + Claims map[string]string `json:"claims"` + Authorization *authorization `json:"authorization"` +} + var errNoTimestamp = errors.New("no valid time fields are set on Log record ('time' or 'timestamp')") // azureLogRecord is a common interface for all category-specific structures @@ -133,8 +212,8 @@ type azureLogRecordBase struct { DurationMs *json.Number `json:"durationMs"` // int CallerIPAddress *string `json:"callerIpAddress"` CorrelationID *string `json:"correlationId"` - Identity *map[string]any `json:"identity"` - Level *string `json:"level"` + Identity json.RawMessage `json:"identity"` + Level *json.Number `json:"level"` Location string `json:"location"` } @@ -180,7 +259,7 @@ func (r *azureLogRecordBase) GetLevel() (plog.SeverityNumber, string, bool) { severity := asSeverity(*r.Level) // Saving original log.Level text, // not the internal OpenTelemetry SeverityNumber -> SeverityText mapping - return severity, *r.Level, true + return severity, r.Level.String(), true } // PutCommonAttributes puts already parsed common attributes into provided Attributes Map/Body @@ -195,8 +274,98 @@ func (r *azureLogRecordBase) PutCommonAttributes(attrs pcommon.Map, _ pcommon.Va unmarshaler.AttrPutStrPtrIf(attrs, string(conventions.NetworkPeerAddressKey), r.CallerIPAddress) unmarshaler.AttrPutStrPtrIf(attrs, attributeAzureCorrelationID, r.CorrelationID) unmarshaler.AttrPutIntNumberPtrIf(attrs, attributeAzureOperationDuration, r.DurationMs) - if r.Identity != nil { - unmarshaler.AttrPutMapIf(attrs, attributeAzureIdentity, *r.Identity) + addIdentityAttributes(r.Identity, attrs) +} + +// addIdentityAttributes extracts identity details +// +// The `identity` field is part of the Top-level common schema for +// resource logs and it's also in use in the activity logs. +// +// We're applying the strategy to only pick the identity elements +// that we know are useful. This approach also minimizes the risk +// of accidentally including sensitive data. +func addIdentityAttributes(identityJSON json.RawMessage, attrs pcommon.Map) { + if len(identityJSON) == 0 { + return + } + + var id identity + if err := gojson.Unmarshal(identityJSON, &id); err != nil { + return + } + + // Authorization + // ------------------------------------------------------------ + + if id.Authorization != nil { + attrs.PutStr(attributeIdentityAuthorizationScope, id.Authorization.Scope) + attrs.PutStr(attributeIdentityAuthorizationAction, id.Authorization.Action) + + if id.Authorization.Evidence != nil { + attrs.PutStr(attributeIdentityAuthorizationEvidenceRole, id.Authorization.Evidence.Role) + attrs.PutStr(attributeIdentityAuthorizationEvidenceRoleAssignmentScope, id.Authorization.Evidence.RoleAssignmentScope) + attrs.PutStr(attributeIdentityAuthorizationEvidenceRoleAssignmentID, id.Authorization.Evidence.RoleAssignmentID) + attrs.PutStr(attributeIdentityAuthorizationEvidenceRoleDefinitionID, id.Authorization.Evidence.RoleDefinitionID) + attrs.PutStr(attributeIdentityAuthorizationEvidencePrincipalID, id.Authorization.Evidence.PrincipalID) + attrs.PutStr(attributeIdentityAuthorizationEvidencePrincipalType, id.Authorization.Evidence.PrincipalType) + } + } + + // Claims + // ------------------------------------------------------------ + + // Extract known claims details we want to include in the + // log record. + // Extract common claim fields + + if iss := id.Claims[identityClaimIssuer]; iss != "" { + attrs.PutStr(attributeIdentityClaimsIssuer, iss) + } + if sub := id.Claims[identityClaimSubject]; sub != "" { + attrs.PutStr(attributeIdentityClaimsSubject, sub) + } + if aud := id.Claims[identityClaimAudience]; aud != "" { + attrs.PutStr(attributeIdentityClaimsAudience, aud) + } + if exp := id.Claims[identityClaimExpires]; exp != "" { + if expTime, err := parseUnixTimestamp(exp); err == nil { + attrs.PutStr(attributeIdentityClaimsNotAfter, expTime.Format(time.RFC3339)) + } + } + if nbf := id.Claims[identityClaimNotBefore]; nbf != "" { + if nbfTime, err := parseUnixTimestamp(nbf); err == nil { + attrs.PutStr(attributeIdentityClaimsNotBefore, nbfTime.Format(time.RFC3339)) + } + } + if iat := id.Claims[identityClaimIssuedAt]; iat != "" { + if iatTime, err := parseUnixTimestamp(iat); err == nil { + attrs.PutStr(attributeIdentityClaimsCreated, iatTime.Format(time.RFC3339)) + } + } + if scope := id.Claims[identityClaimScope]; scope != "" { + attrs.PutStr(attributeIdentityClaimsScope, scope) + } + if idtyp := id.Claims[identityClaimType]; idtyp != "" { + attrs.PutStr(attributeIdentityClaimsType, idtyp) + } + if appid := id.Claims[identityClaimApplicationID]; appid != "" { + attrs.PutStr(attributeIdentityClaimsApplicationID, appid) + } + if authmethods := id.Claims[identityClaimAuthMethodsReferences]; authmethods != "" { + attrs.PutStr(attributeIdentityClaimsAuthMethodsReferences, authmethods) + } + if provider := id.Claims[identityClaimProvider]; provider != "" { + attrs.PutStr(attributeIdentityClaimsProvider, provider) + } + if object := id.Claims[identityClaimIdentifierObject]; object != "" { + attrs.PutStr(attributeIdentityClaimsIdentifierObject, object) + } + if name := id.Claims[identityClaimIdentifierName]; name != "" { + attrs.PutStr(attributeIdentityClaimsIdentifierName, name) + } + if email := id.Claims[identityClaimEmailAddress]; email != "" { + attrs.PutStr(string(conventions.UserEmailKey), email) } } @@ -316,12 +485,26 @@ func processLogRecord(logCategory string, record []byte) (azureLogRecord, error) parsed = new(frontDoorWAFLog) case categoryFunctionAppLogs: parsed = new(azureFunctionAppLog) - case categoryRecommendation: - parsed = new(azureRecommendationLog) // StorageRead, StorageWrite, StorageDelete share the same properties, // called StorageBlobLogs, see https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/storagebloblogs case categoryStorageRead, categoryStorageWrite, categoryStorageDelete: parsed = new(azureStorageBlobLog) + case categoryAdministrative: + parsed = new(azureAdministrativeLog) + case categoryAlert: + parsed = new(azureAlertLog) + case categoryAutoscale: + parsed = new(azureAutoscaleLog) + case categorySecurity: + parsed = new(azureSecurityLog) + case categoryPolicy: + parsed = new(azurePolicyLog) + case categoryServiceHealth: + parsed = new(azureServiceHealthLog) + case categoryRecommendation: + parsed = new(azureRecommendationLog) + case categoryResourceHealth: + parsed = new(azureResourceHealthLog) default: parsed = new(azureLogRecordGeneric) } diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category_activitylogs.go b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category_activitylogs.go new file mode 100644 index 0000000000000..f9e60aa84cf4e --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/category_activitylogs.go @@ -0,0 +1,443 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package logs // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension/internal/unmarshaler/logs" + +import ( + "strconv" + + gojson "github.com/goccy/go-json" + "go.opentelemetry.io/collector/pdata/pcommon" + conventions "go.opentelemetry.io/otel/semconv/v1.38.0" + + "github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension/internal/unmarshaler" +) + +// ------------------------------------------------------------ +// Activity Log - Administrative category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Administrative activity logs +const ( + attributeAzureAdministrativeEntity = "azure.administrative.entity" + attributeAzureAdministrativeMessage = "azure.administrative.message" + attributeAzureAdministrativeHierarchy = "azure.administrative.hierarchy" +) + +// azureAdministrativeLog represents an Administrative activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#administrative-category +type azureAdministrativeLog struct { + azureLogRecordBase + + Properties struct { + Entity string `json:"entity"` + Message string `json:"message"` + Hierarchy string `json:"hierarchy"` + } `json:"properties"` +} + +func (r *azureAdministrativeLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + unmarshaler.AttrPutStrIf(attrs, attributeAzureAdministrativeEntity, r.Properties.Entity) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAdministrativeMessage, r.Properties.Message) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAdministrativeHierarchy, r.Properties.Hierarchy) + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Alert category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Alert activity logs +const ( + attributeAzureAlertWebhookURI = "azure.alert.webhook.uri" + attributeAzureAlertRuleURI = "azure.alert.rule.uri" + attributeAzureAlertRuleName = "azure.alert.rule.name" + attributeAzureAlertRuleDescription = "azure.alert.rule.description" + attributeAzureAlertThreshold = "azure.alert.threshold" + attributeAzureAlertWindowSize = "azure.alert.window_size_minutes" + attributeAzureAlertAggregation = "azure.alert.aggregation" + attributeAzureAlertOperator = "azure.alert.operator" + attributeAzureAlertMetricName = "azure.alert.metric.name" + attributeAzureAlertMetricUnit = "azure.alert.metric.unit" +) + +// azureAlertLog represents an Alert activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#alert-category +type azureAlertLog struct { + azureLogRecordBase + + Properties struct { + WebHookURI string `json:"webHookUri"` + RuleURI string `json:"RuleUri"` + RuleName string `json:"RuleName"` + RuleDescription string `json:"RuleDescription"` + Threshold string `json:"Threshold"` + WindowSizeInMinutes string `json:"WindowSizeInMinutes"` + Aggregation string `json:"Aggregation"` + Operator string `json:"Operator"` + MetricName string `json:"MetricName"` + MetricUnit string `json:"MetricUnit"` + } `json:"properties"` +} + +func (r *azureAlertLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertWebhookURI, r.Properties.WebHookURI) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertRuleURI, r.Properties.RuleURI) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertRuleName, r.Properties.RuleName) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertRuleDescription, r.Properties.RuleDescription) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertThreshold, r.Properties.Threshold) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertWindowSize, r.Properties.WindowSizeInMinutes) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertAggregation, r.Properties.Aggregation) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertOperator, r.Properties.Operator) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertMetricName, r.Properties.MetricName) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAlertMetricUnit, r.Properties.MetricUnit) + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Autoscale category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Autoscale activity logs +const ( + attributeAzureAutoscaleDescription = "azure.autoscale.description" + attributeAzureAutoscaleResourceName = "azure.autoscale.resource.name" + attributeAzureAutoscaleOldInstances = "azure.autoscale.instances.previous_count" + attributeAzureAutoscaleNewInstances = "azure.autoscale.instances.count" + attributeAzureAutoscaleLastScaleAction = "azure.autoscale.resource.last_scale" +) + +// azureAutoscaleLog represents an Autoscale activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#autoscale-category +type azureAutoscaleLog struct { + azureLogRecordBase + + Properties struct { + Description string `json:"Description"` + ResourceName string `json:"ResourceName"` + OldInstancesCount string `json:"OldInstancesCount"` + NewInstancesCount string `json:"NewInstancesCount"` + LastScaleActionTime string `json:"LastScaleActionTime"` + } `json:"properties"` +} + +func (r *azureAutoscaleLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + unmarshaler.AttrPutStrIf(attrs, attributeAzureAutoscaleDescription, r.Properties.Description) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAutoscaleResourceName, r.Properties.ResourceName) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAutoscaleOldInstances, r.Properties.OldInstancesCount) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAutoscaleNewInstances, r.Properties.NewInstancesCount) + unmarshaler.AttrPutStrIf(attrs, attributeAzureAutoscaleLastScaleAction, r.Properties.LastScaleActionTime) + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Security category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Security activity logs +const ( + attributeAzureSecurityAccountLogonID = "azure.security.account_logon_id" + attributeAzureSecurityDomainName = "azure.security.domain_name" + attributeAzureSecurityActionTaken = "azure.security.action_taken" + attributeAzureSecuritySeverity = "azure.security.severity" +) + +// azureSecurityLog represents a Security activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#security-category +type azureSecurityLog struct { + azureLogRecordBase + + Properties struct { + AccountLogonID string `json:"accountLogonId"` + CommandLine string `json:"commandLine"` + DomainName string `json:"domainName"` + ParentProcess string `json:"parentProcess"` + ParentProcessID string `json:"parentProcess id"` + ProcessID string `json:"processId"` + ProcessName string `json:"processName"` + UserName string `json:"userName"` + UserSID string `json:"UserSID"` + ActionTaken string `json:"ActionTaken"` + Severity string `json:"Severity"` + } `json:"properties"` +} + +func (r *azureSecurityLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + // Map to OTel process semantic conventions + unmarshaler.AttrPutStrIf(attrs, string(conventions.ProcessCommandLineKey), r.Properties.CommandLine) + unmarshaler.AttrPutStrIf(attrs, string(conventions.ProcessExecutablePathKey), r.Properties.ProcessName) + unmarshaler.AttrPutStrIf(attrs, string(conventions.ProcessOwnerKey), r.Properties.UserName) + unmarshaler.AttrPutStrIf(attrs, string(conventions.EnduserIDKey), r.Properties.UserSID) + + // Parse and set process.pid if present + if r.Properties.ProcessID != "" { + if pid, err := strconv.ParseInt(r.Properties.ProcessID, 10, 64); err == nil { + attrs.PutInt(string(conventions.ProcessPIDKey), pid) + } + } + + // Parse and set process.parent_pid if present + if r.Properties.ParentProcessID != "" { + if ppid, err := strconv.ParseInt(r.Properties.ParentProcessID, 10, 64); err == nil { + attrs.PutInt(string(conventions.ProcessParentPIDKey), ppid) + } + } + + // Azure-specific fields that don't have OTel equivalents + unmarshaler.AttrPutStrIf(attrs, attributeAzureSecurityAccountLogonID, r.Properties.AccountLogonID) + unmarshaler.AttrPutStrIf(attrs, attributeAzureSecurityDomainName, r.Properties.DomainName) + unmarshaler.AttrPutStrIf(attrs, attributeAzureSecurityActionTaken, r.Properties.ActionTaken) + unmarshaler.AttrPutStrIf(attrs, attributeAzureSecuritySeverity, r.Properties.Severity) + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Policy category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Policy activity logs +const ( + attributeAzurePolicyIsComplianceCheck = "azure.policy.compliance_check" + attributeAzureLocation = "azure.location" + attributeAzurePolicyAncestors = "azure.policy.ancestors" + attributeAzurePolicyHierarchy = "azure.policy.hierarchy" + attributeAzurePolicyPolicies = "azure.policy.policies" +) + +// policyElement represents a single policy in the policies array +type policyElement struct { + DefinitionID string `json:"policyDefinitionId"` + SetDefinitionID string `json:"policySetDefinitionId"` + ReferenceID string `json:"policyDefinitionReferenceId"` + SetDefinitionName string `json:"policySetDefinitionName"` + SetDefinitionDisplayName string `json:"policySetDefinitionDisplayName"` + SetDefinitionVersion string `json:"policySetDefinitionVersion"` + DefinitionName string `json:"policyDefinitionName"` + DefinitionDisplayName string `json:"policyDefinitionDisplayName"` + DefinitionVersion string `json:"policyDefinitionVersion"` + DefinitionEffect string `json:"policyDefinitionEffect"` + AssignmentID string `json:"policyAssignmentId"` + AssignmentName string `json:"policyAssignmentName"` + AssignmentDisplayName string `json:"policyAssignmentDisplayName"` + AssignmentScope string `json:"policyAssignmentScope"` + ExemptionIDs []string `json:"policyExemptionIds"` + AssignmentIDs []string `json:"policyAssignmentIds"` +} + +// azurePolicyLog represents a Policy activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#policy-category +type azurePolicyLog struct { + azureLogRecordBase + + Properties struct { + IsComplianceCheck string `json:"isComplianceCheck"` + ResourceLocation string `json:"resourceLocation"` + Ancestors string `json:"ancestors"` + Policies string `json:"policies"` + Hierarchy string `json:"hierarchy"` + } `json:"properties"` +} + +func (r *azurePolicyLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + // Parse isComplianceCheck as boolean + if r.Properties.IsComplianceCheck != "" { + isComplianceCheck := r.Properties.IsComplianceCheck == "True" || r.Properties.IsComplianceCheck == "true" + attrs.PutBool(attributeAzurePolicyIsComplianceCheck, isComplianceCheck) + } + + unmarshaler.AttrPutStrIf(attrs, attributeAzureLocation, r.Properties.ResourceLocation) + unmarshaler.AttrPutStrIf(attrs, attributeAzurePolicyAncestors, r.Properties.Ancestors) + unmarshaler.AttrPutStrIf(attrs, attributeAzurePolicyHierarchy, r.Properties.Hierarchy) + + // Parse the embedded policies JSON string + if r.Properties.Policies != "" { + var policies []policyElement + if err := gojson.Unmarshal([]byte(r.Properties.Policies), &policies); err == nil && len(policies) > 0 { + policiesSlice := attrs.PutEmptySlice(attributeAzurePolicyPolicies) + for i := range policies { + policyMap := policiesSlice.AppendEmpty().SetEmptyMap() + policyMap.PutStr("definition.id", policies[i].DefinitionID) + policyMap.PutStr("definition.name", policies[i].DefinitionName) + policyMap.PutStr("definition.display_name", policies[i].DefinitionDisplayName) + policyMap.PutStr("definition.version", policies[i].DefinitionVersion) + policyMap.PutStr("definition.effect", policies[i].DefinitionEffect) + policyMap.PutStr("definition.reference_id", policies[i].ReferenceID) + policyMap.PutStr("set_definition.id", policies[i].SetDefinitionID) + policyMap.PutStr("set_definition.name", policies[i].SetDefinitionName) + policyMap.PutStr("set_definition.display_name", policies[i].SetDefinitionDisplayName) + policyMap.PutStr("set_definition.version", policies[i].SetDefinitionVersion) + policyMap.PutStr("assignment.id", policies[i].AssignmentID) + policyMap.PutStr("assignment.name", policies[i].AssignmentName) + policyMap.PutStr("assignment.display_name", policies[i].AssignmentDisplayName) + policyMap.PutStr("assignment.scope", policies[i].AssignmentScope) + } + } + } + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Service Health category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Service Health activity logs +const ( + attributeAzureServiceHealthTitle = "azure.servicehealth.title" + attributeAzureServiceHealthService = "azure.servicehealth.service" + attributeAzureServiceHealthRegion = "azure.servicehealth.region" + attributeAzureServiceHealthCommunicationID = "azure.servicehealth.communication.id" + attributeAzureServiceHealthCommunicationBody = "azure.servicehealth.communication.body" + attributeAzureServiceHealthIncidentType = "azure.servicehealth.incident.type" + attributeAzureServiceHealthTrackingID = "azure.servicehealth.tracking.id" + attributeAzureServiceHealthImpactStartTime = "azure.servicehealth.impact.start" + attributeAzureServiceHealthImpactMitigationTime = "azure.servicehealth.impact.mitigation" + attributeAzureServiceHealthImpactedServices = "azure.servicehealth.impact.services" + attributeAzureServiceHealthImpactType = "azure.servicehealth.impact.type" + attributeAzureServiceHealthImpactCategory = "azure.servicehealth.impact.category" + attributeAzureServiceHealthDefaultLanguageTitle = "azure.servicehealth.default_language.title" + attributeAzureServiceHealthDefaultLanguageContent = "azure.servicehealth.default_language.content" + attributeAzureServiceHealthState = "azure.servicehealth.state" + attributeAzureServiceHealthMaintenanceID = "azure.servicehealth.maintenance.id" + attributeAzureServiceHealthMaintenanceType = "azure.servicehealth.maintenance.type" + attributeAzureServiceHealthIsHIR = "azure.servicehealth.is_hir" + attributeAzureServiceHealthIsSynthetic = "azure.servicehealth.is_synthetic" +) + +// impactedService represents an impacted service in Service Health logs +type impactedService struct { + Name string `json:"ServiceName"` + ID string `json:"ServiceId"` + GUID string `json:"ServiceGuid"` + Regions []struct { + Name string `json:"RegionName"` + ID string `json:"RegionId"` + } `json:"ImpactedRegions"` +} + +// azureServiceHealthLog represents a Service Health activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#service-health-category +type azureServiceHealthLog struct { + azureLogRecordBase + + Properties struct { + Title string `json:"title"` + Service string `json:"service"` + Region string `json:"region"` + CommunicationText string `json:"communication"` + CommunicationID string `json:"communicationId"` + IncidentType string `json:"incidentType"` + TrackingID string `json:"trackingId"` + ImpactStartTime string `json:"impactStartTime"` + ImpactMitigationTime string `json:"impactMitigationTime"` + ImpactedServices string `json:"impactedServices"` + DefaultLanguageTitle string `json:"defaultLanguageTitle"` + DefaultLanguageContent string `json:"defaultLanguageContent"` + Stage string `json:"stage"` + MaintenanceID string `json:"maintenanceId"` + MaintenanceType string `json:"maintenanceType"` + IsHIR bool `json:"isHIR"` + IsSynthetic string `json:"IsSynthetic"` + ImpactType string `json:"impactType"` + ImpactCategory string `json:"impactCategory"` + } `json:"properties"` +} + +func (r *azureServiceHealthLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthTitle, r.Properties.Title) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthService, r.Properties.Service) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthRegion, r.Properties.Region) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthCommunicationBody, r.Properties.CommunicationText) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthIncidentType, r.Properties.IncidentType) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthTrackingID, r.Properties.TrackingID) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthImpactStartTime, r.Properties.ImpactStartTime) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthImpactMitigationTime, r.Properties.ImpactMitigationTime) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthDefaultLanguageTitle, r.Properties.DefaultLanguageTitle) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthDefaultLanguageContent, r.Properties.DefaultLanguageContent) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthState, r.Properties.Stage) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthCommunicationID, r.Properties.CommunicationID) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthMaintenanceID, r.Properties.MaintenanceID) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthMaintenanceType, r.Properties.MaintenanceType) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthImpactType, r.Properties.ImpactType) + unmarshaler.AttrPutStrIf(attrs, attributeAzureServiceHealthImpactCategory, r.Properties.ImpactCategory) + + // Handle isHIR - only set if true + if r.Properties.IsHIR { + attrs.PutBool(attributeAzureServiceHealthIsHIR, r.Properties.IsHIR) + } + + // Parse isSynthetic as boolean + if r.Properties.IsSynthetic != "" { + isSynthetic := r.Properties.IsSynthetic == "True" || r.Properties.IsSynthetic == "true" + attrs.PutBool(attributeAzureServiceHealthIsSynthetic, isSynthetic) + } + + // Parse the embedded impactedServices JSON string + if r.Properties.ImpactedServices != "" { + var impactedServices []impactedService + if err := gojson.Unmarshal([]byte(r.Properties.ImpactedServices), &impactedServices); err == nil && len(impactedServices) > 0 { + impactedServicesSlice := attrs.PutEmptySlice(attributeAzureServiceHealthImpactedServices) + for _, s := range impactedServices { + impactedServiceMap := impactedServicesSlice.AppendEmpty().SetEmptyMap() + impactedServiceMap.PutStr("name", s.Name) + impactedServiceMap.PutStr("id", s.ID) + impactedServiceMap.PutStr("guid", s.GUID) + + if len(s.Regions) > 0 { + regionsSlice := impactedServiceMap.PutEmptySlice("regions") + for _, region := range s.Regions { + regionMap := regionsSlice.AppendEmpty().SetEmptyMap() + regionMap.PutStr("name", region.Name) + regionMap.PutStr("id", region.ID) + } + } + } + } + } + + return nil +} + +// ------------------------------------------------------------ +// Activity Log - Resource Health category +// ------------------------------------------------------------ + +// Non-SemConv attributes for Resource Health activity logs +const ( + attributeAzureResourceHealthTitle = "azure.resourcehealth.title" + attributeAzureResourceHealthDetails = "azure.resourcehealth.details" + attributeAzureResourceHealthCurrentHealthStatus = "azure.resourcehealth.state" + attributeAzureResourceHealthPreviousHealthStatus = "azure.resourcehealth.previous_state" + attributeAzureResourceHealthType = "azure.resourcehealth.type" + attributeAzureResourceHealthCause = "azure.resourcehealth.cause" +) + +// azureResourceHealthLog represents a Resource Health activity log +// See: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log-schema#resource-health-category +type azureResourceHealthLog struct { + azureLogRecordBase + + Properties struct { + Title string `json:"title"` + Details string `json:"details"` + CurrentHealthStatus string `json:"currentHealthStatus"` + PreviousHealthStatus string `json:"previousHealthStatus"` + Type string `json:"type"` + Cause string `json:"cause"` + } `json:"properties"` +} + +func (r *azureResourceHealthLog) PutProperties(attrs pcommon.Map, _ pcommon.Value) error { + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthTitle, r.Properties.Title) + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthDetails, r.Properties.Details) + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthCurrentHealthStatus, r.Properties.CurrentHealthStatus) + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthPreviousHealthStatus, r.Properties.PreviousHealthStatus) + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthType, r.Properties.Type) + unmarshaler.AttrPutStrIf(attrs, attributeAzureResourceHealthCause, r.Properties.Cause) + + return nil +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers.go b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers.go index 4457402f56f82..baae024d81acf 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers.go +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers.go @@ -7,6 +7,8 @@ import ( "bytes" "crypto/tls" "encoding/json" + "strconv" + "time" "go.opentelemetry.io/collector/pdata/pcommon" "go.opentelemetry.io/collector/pdata/plog" @@ -47,8 +49,8 @@ var ( // According to the documentation, the level Must be one of: // `Informational`, `Warning`, `Error` or `Critical`. // see https://learn.microsoft.com/en-us/azure/azure-monitor/platform/resource-logs-schema -func asSeverity(input string) plog.SeverityNumber { - switch input { +func asSeverity(number json.Number) plog.SeverityNumber { + switch number.String() { case "Informational": return plog.SeverityNumberInfo case "Warning": @@ -213,3 +215,12 @@ func convertStringToJSONNumber(s string) json.Number { return json.Number(s) } + +// parseUnixTimestamp parses a Unix timestamp string (seconds since epoch) and returns a time.Time +func parseUnixTimestamp(s string) (time.Time, error) { + ts, err := strconv.ParseInt(s, 10, 64) + if err != nil { + return time.Time{}, err + } + return time.Unix(ts, 0).UTC(), nil +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers_test.go b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers_test.go index 91b64675312e2..7a4c13d3ef022 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers_test.go +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/helpers_test.go @@ -27,7 +27,7 @@ func TestAsSeverity(t *testing.T) { for input, expected := range tests { t.Run(input, func(t *testing.T) { - assert.Equal(t, expected, asSeverity(input)) + assert.Equal(t, expected, asSeverity(json.Number(input))) }) } } diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log.json new file mode 100644 index 0000000000000..9627e072b2cb9 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log.json @@ -0,0 +1,76 @@ +{ + "records": [ + { + "RoleLocation": "France South", + "Stamp": "FDWeb", + "ReleaseVersion": "6.2025.14.4+0123456.release_2025w14", + "time": "2025-04-15T10:16:32.9873441Z", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/PROVIDERS/MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/EXAMPLE-COLLECT-SAMPLE-LOGS", + "operationName": "MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/WRITE", + "category": "Administrative", + "resultType": "Start", + "resultSignature": "Started.", + "durationMs": "0", + "callerIpAddress": "203.0.113.10", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-111111111111", + "identity": { + "authorization": { + "scope": "/subscriptions/11111111-1111-1111-1111-111111111111/providers/microsoft.insights/diagnosticSettings/example-collect-sample-logs", + "action": "microsoft.insights/diagnosticSettings/write", + "evidence": { + "role": "Owner", + "roleAssignmentScope": "/subscriptions/11111111-1111-1111-1111-111111111111", + "roleAssignmentId": "33333333333333333333333333333333", + "roleDefinitionId": "44444444444444444444444444444444", + "principalId": "55555555555555555555555555555555", + "principalType": "User" + } + }, + "claims": { + "aud": "https://management.core.windows.net/", + "iss": "https://sts.windows.net/22222222-2222-2222-2222-222222222222/", + "iat": "1744711621", + "nbf": "1744711621", + "exp": "1744717084", + "http://schemas.microsoft.com/claims/authnclassreference": "1", + "aio": "AaQAW/8ZAAAAZgO/4jIRs/N/KB2dU/OBCRIKqhlJptIDhdwKznsOAvG9l6XBKLVbb3gcOhXy9f7l7PPh72wUbG+ej/8K+eWceVLG+R554s0NPw8+L9LbF/FkzqSnEd3FGiBccK4bL1UbJ0B9ddpT60CbBOgu/xF9JTzYPu3Mtl7em3l+o4oybByk52SljldSN5LizwAmpC0b/rho3D6WMskK7fBotCV3XQ==", + "altsecid": "5::10032001EEC259BA", + "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd,mfa", + "appid": "66666666-6666-6666-6666-666666666666", + "appidacr": "0", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "user@example.com", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "Doe", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "John", + "groups": "77777777-7777-7777-7777-777777777777,88888888-8888-8888-8888-888888888888,99999999-9999-9999-9999-999999999999", + "http://schemas.microsoft.com/identity/claims/identityprovider": "https://sts.windows.net/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/", + "idtyp": "user", + "ipaddr": "203.0.113.10", + "name": "John Doe", + "http://schemas.microsoft.com/identity/claims/objectidentifier": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", + "puid": "100320024FCDB44E", + "rh": "1.AUgAW2hAqn1BZEa07I92QHGa20ZIf3kAutdPukPawfj2MBMLAR1IAA.", + "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation", + "sid": "cccccccc-cccc-cccc-cccc-cccccccccccc", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "8lEWzt6Ed5a3RZMV7p9x6FI31xksIsZsmh09g0Lbdq0", + "http://schemas.microsoft.com/identity/claims/tenantid": "22222222-2222-2222-2222-222222222222", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "user@example.com", + "uti": "s4OAwbVGQ0CX8sNFq4tAAA", + "ver": "1.0", + "wids": "dddddddd-dddd-dddd-dddd-dddddddddddd,eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee,ffffffff-ffff-ffff-ffff-ffffffffffff", + "xms_edov": "true", + "xms_idrel": "5 26", + "xms_tcdt": "1391159646" + } + }, + "level": "Informational", + "properties": { + "requestbody": "{\"id\":\"/subscriptions/11111111-1111-1111-1111-111111111111/providers/microsoft.insights/diagnosticSettings/example-collect-sample-logs\",\"name\":\"example-collect-sample-logs\",\"properties\":{\"logs\":[{\"category\":\"Administrative\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"Security\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"ServiceHealth\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"Alert\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"Recommendation\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"Policy\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"Autoscale\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}},{\"category\":\"ResourceHealth\",\"categoryGroup\":null,\"enabled\":true,\"retentionPolicy\":{\"days\":0,\"enabled\":false}}],\"metrics\":[],\"storageAccountId\":\"/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.Storage/storageAccounts/examplestorageacct\",\"eventHubAuthorizationRuleId\":\"/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.EventHub/namespaces/exampleeventhubns/authorizationrules/RootManageSharedAccessKey\",\"eventHubName\":\"logs\"}}", + "eventCategory": "Administrative", + "entity": "/subscriptions/11111111-1111-1111-1111-111111111111/providers/microsoft.insights/diagnosticSettings/example-collect-sample-logs", + "message": "microsoft.insights/diagnosticSettings/write", + "hierarchy": "22222222-2222-2222-2222-222222222222/11111111-1111-1111-1111-111111111111" + }, + "tenantId": "22222222-2222-2222-2222-222222222222" + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log_expected.yaml new file mode 100644 index 0000000000000..cc62ac12b9214 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/administrative/administrative-log_expected.yaml @@ -0,0 +1,118 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/PROVIDERS/MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/EXAMPLE-COLLECT-SAMPLE-LOGS + - key: azure.tenant.id + value: + stringValue: 22222222-2222-2222-2222-222222222222 + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: Administrative + - key: azure.operation.name + value: + stringValue: MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/WRITE + - key: azure.result.type + value: + stringValue: Start + - key: azure.result.signature + value: + stringValue: Started. + - key: network.peer.address + value: + stringValue: 203.0.113.10 + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-111111111111 + - key: azure.operation.duration + value: + intValue: "0" + - key: azure.identity.authorization.scope + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/providers/microsoft.insights/diagnosticSettings/example-collect-sample-logs + - key: azure.identity.authorization.action + value: + stringValue: microsoft.insights/diagnosticSettings/write + - key: azure.identity.authorization.evidence.role + value: + stringValue: Owner + - key: azure.identity.authorization.evidence.role.assignment.scope + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111 + - key: azure.identity.authorization.evidence.role.assignment.id + value: + stringValue: "33333333333333333333333333333333" + - key: azure.identity.authorization.evidence.role.definition.id + value: + stringValue: "44444444444444444444444444444444" + - key: azure.identity.authorization.evidence.principal.id + value: + stringValue: "55555555555555555555555555555555" + - key: azure.identity.authorization.evidence.principal.type + value: + stringValue: User + - key: azure.identity.issuer + value: + stringValue: https://sts.windows.net/22222222-2222-2222-2222-222222222222/ + - key: azure.identity.audience + value: + stringValue: https://management.core.windows.net/ + - key: azure.identity.not_after + value: + stringValue: "2025-04-15T11:38:04Z" + - key: azure.identity.not_before + value: + stringValue: "2025-04-15T10:07:01Z" + - key: azure.identity.created + value: + stringValue: "2025-04-15T10:07:01Z" + - key: azure.identity.scope + value: + stringValue: user_impersonation + - key: azure.identity.type + value: + stringValue: user + - key: azure.identity.application.id + value: + stringValue: 66666666-6666-6666-6666-666666666666 + - key: azure.identity.auth.methods.references + value: + stringValue: pwd,mfa + - key: azure.identity.provider + value: + stringValue: https://sts.windows.net/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/ + - key: azure.identity.identifier.object + value: + stringValue: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb + - key: user.name + value: + stringValue: 8lEWzt6Ed5a3RZMV7p9x6FI31xksIsZsmh09g0Lbdq0 + - key: user.email + value: + stringValue: user@example.com + - key: azure.administrative.entity + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/providers/microsoft.insights/diagnosticSettings/example-collect-sample-logs + - key: azure.administrative.message + value: + stringValue: microsoft.insights/diagnosticSettings/write + - key: azure.administrative.hierarchy + value: + stringValue: 22222222-2222-2222-2222-222222222222/11111111-1111-1111-1111-111111111111 + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1744712192987344100" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log.json new file mode 100644 index 0000000000000..7191c2ac08b4d --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log.json @@ -0,0 +1,32 @@ +{ + "records": [ + { + "Level": 5, + "category": "Alert", + "location": "global", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-RESOURCE-GROUP/PROVIDERS/MICROSOFT.CLASSICCOMPUTE/DOMAINNAMES/EXAMPLE-RESOURCE-GROUP/SLOTS/PRODUCTION/ROLES/EVENT.BACKGROUNDJOBSWORKER.RAZZLE", + "time": "2017-07-21T09:24:13.522192Z", + "level": "Informational", + "correlationId": "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/microsoft.insights/alertrules/examplealert/incidents/L3N1YnNjcmlwdGlvbnMvZGY2MDJjOWMtN2FhMC00MDdkLWE2ZmItZWIyMGM4YmQxMTkyL3Jlc291cmNlR3JvdXBzL0NzbUV2ZW50RE9HRk9PRC1XZXN0VVMvcHJvdmlkZXJzL21pY3Jvc29mdC5pbnNpZ2h0cy9hbGVydHJ1bGVzL2V4YW1wbGVhbGVydDA2MzYzNjIyNTg1MzUyMjE5MjA=", + "operationName": "MICROSOFT.INSIGHTS/ALERTRULES/RESOLVED/ACTION", + "resultType": "Resolved", + "properties": { + "RuleUri": "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/microsoft.insights/alertrules/examplealert", + "RuleName": "examplealert", + "RuleDescription": "", + "Threshold": "100000", + "WindowSizeInMinutes": "5", + "Aggregation": "Average", + "Operator": "LessThan", + "MetricName": "Disk read", + "MetricUnit": "Count" + }, + "tenantId": "22222222-2222-2222-2222-222222222222", + "identity": { + "claims": { + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/spn": "Microsoft.Insights/alertRules" + } + } + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log_expected.yaml new file mode 100644 index 0000000000000..99f45d59c18dd --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/alert/alert-log_expected.yaml @@ -0,0 +1,64 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-RESOURCE-GROUP/PROVIDERS/MICROSOFT.CLASSICCOMPUTE/DOMAINNAMES/EXAMPLE-RESOURCE-GROUP/SLOTS/PRODUCTION/ROLES/EVENT.BACKGROUNDJOBSWORKER.RAZZLE + - key: cloud.region + value: + stringValue: global + - key: azure.tenant.id + value: + stringValue: 22222222-2222-2222-2222-222222222222 + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: Alert + - key: azure.operation.name + value: + stringValue: MICROSOFT.INSIGHTS/ALERTRULES/RESOLVED/ACTION + - key: azure.result.type + value: + stringValue: Resolved + - key: azure.correlation_id + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/microsoft.insights/alertrules/examplealert/incidents/L3N1YnNjcmlwdGlvbnMvZGY2MDJjOWMtN2FhMC00MDdkLWE2ZmItZWIyMGM4YmQxMTkyL3Jlc291cmNlR3JvdXBzL0NzbUV2ZW50RE9HRk9PRC1XZXN0VVMvcHJvdmlkZXJzL21pY3Jvc29mdC5pbnNpZ2h0cy9hbGVydHJ1bGVzL2V4YW1wbGVhbGVydDA2MzYzNjIyNTg1MzUyMjE5MjA= + - key: azure.alert.rule.uri + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/microsoft.insights/alertrules/examplealert + - key: azure.alert.rule.name + value: + stringValue: examplealert + - key: azure.alert.threshold + value: + stringValue: "100000" + - key: azure.alert.window_size_minutes + value: + stringValue: "5" + - key: azure.alert.aggregation + value: + stringValue: Average + - key: azure.alert.operator + value: + stringValue: LessThan + - key: azure.alert.metric.name + value: + stringValue: Disk read + - key: azure.alert.metric.unit + value: + stringValue: Count + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1500629053522192000" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log.json new file mode 100644 index 0000000000000..5a74318999da8 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log.json @@ -0,0 +1,28 @@ +{ + "records": [ + { + "Level": 5, + "category": "Autoscale", + "location": "global", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-RESOURCE-GROUP/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/EXAMPLE-RESOURCE-GROUP-PRODUCTION-EXAMPLE-RESOURCE-EXAMPLE-RESOURCE-GROUP", + "time": "2017-07-21T01:00:51.8681572Z", + "level": "Informational", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-222222222222", + "operationName": "MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/SCALEDOWN/ACTION", + "resultType": "Succeeded", + "properties": { + "Description": "The autoscale engine attempting to scale resource '/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.ClassicCompute/domainNames/example-resource-group/slots/Production/roles/exampleResource' from 3 instances count to 2 instances count.", + "ResourceName": "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.ClassicCompute/domainNames/example-resource-group/slots/Production/roles/exampleResource", + "OldInstancesCount": "3", + "NewInstancesCount": "2", + "LastScaleActionTime": "Fri, 21 Jul 2017 01:00:51 GMT" + }, + "tenantId": "22222222-2222-2222-2222-222222222222", + "identity": { + "claims": { + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/spn": "Microsoft.Insights/autoscaleSettings" + } + } + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log_expected.yaml new file mode 100644 index 0000000000000..1bcaa21c29578 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/autoscale/autoscale-log_expected.yaml @@ -0,0 +1,55 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-RESOURCE-GROUP/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/EXAMPLE-RESOURCE-GROUP-PRODUCTION-EXAMPLE-RESOURCE-EXAMPLE-RESOURCE-GROUP + - key: cloud.region + value: + stringValue: global + - key: azure.tenant.id + value: + stringValue: 22222222-2222-2222-2222-222222222222 + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: Autoscale + - key: azure.operation.name + value: + stringValue: MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/SCALEDOWN/ACTION + - key: azure.result.type + value: + stringValue: Succeeded + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-222222222222 + - key: azure.autoscale.description + value: + stringValue: The autoscale engine attempting to scale resource '/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.ClassicCompute/domainNames/example-resource-group/slots/Production/roles/exampleResource' from 3 instances count to 2 instances count. + - key: azure.autoscale.resource.name + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/example-resource-group/providers/Microsoft.ClassicCompute/domainNames/example-resource-group/slots/Production/roles/exampleResource + - key: azure.autoscale.instances.previous_count + value: + stringValue: "3" + - key: azure.autoscale.instances.count + value: + stringValue: "2" + - key: azure.autoscale.resource.last_scale + value: + stringValue: Fri, 21 Jul 2017 01:00:51 GMT + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1500598851868157200" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/general/maximum_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/general/maximum_expected.yaml index 50a2898eae144..a490f0d5f6895 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/general/maximum_expected.yaml +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/general/maximum_expected.yaml @@ -46,17 +46,6 @@ resourceLogs: - key: azure.operation.duration value: intValue: "1234" - - key: azure.identity - value: - kvlistValue: - values: - - key: claim - value: - kvlistValue: - values: - - key: oid - value: - stringValue: 607964b6-41a5-4e24-a5db-db7aab3b9b34 - key: string value: stringValue: string @@ -123,17 +112,6 @@ resourceLogs: - key: azure.operation.duration value: intValue: "4321" - - key: azure.identity - value: - kvlistValue: - values: - - key: claim - value: - kvlistValue: - values: - - key: oid - value: - stringValue: 96317703-2132-4a8d-a5d7-e18d2f486783 - key: string value: stringValue: string @@ -178,17 +156,6 @@ resourceLogs: - key: azure.operation.duration value: intValue: "321" - - key: azure.identity - value: - kvlistValue: - values: - - key: claim - value: - kvlistValue: - values: - - key: oid - value: - stringValue: 4ae807da-39d9-4327-b5b4-0ab685a57f9a - key: string value: stringValue: string diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log.json new file mode 100644 index 0000000000000..00b4ead2a80fc --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log.json @@ -0,0 +1,80 @@ +{ + "records": [ + { + "RoleLocation": "Switzerland West", + "Stamp": "FDWorker", + "ReleaseVersion": "6.2025.14.4+0123456.release_2025w14", + "time": "2025-04-23T11:02:06.6966319Z", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/CONTOSO-RESOURCES/PROVIDERS/MICROSOFT.WEB/SITES/CONTOSO-WEB-APP", + "operationName": "MICROSOFT.AUTHORIZATION/POLICIES/AUDIT/ACTION", + "category": "Policy", + "resultType": "Success", + "resultSignature": "Succeeded.", + "durationMs": "0", + "callerIpAddress": "203.0.113.50", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", + "identity": { + "authorization": { + "scope": "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/contoso-resources/providers/Microsoft.Web/sites/contoso-web-app", + "action": "Microsoft.Web/sites/write", + "evidence": { + "role": "Owner", + "roleAssignmentScope": "/subscriptions/11111111-1111-1111-1111-111111111111", + "roleAssignmentId": "22222222222222222222222222222222", + "roleDefinitionId": "33333333333333333333333333333333", + "principalId": "44444444444444444444444444444444", + "principalType": "User" + } + }, + "claims": { + "aud": "https://management.core.windows.net/", + "iss": "https://sts.windows.net/55555555-5555-5555-5555-555555555555/", + "iat": "1745405200", + "nbf": "1745405200", + "exp": "1745409118", + "http://schemas.microsoft.com/claims/authnclassreference": "1", + "aio": "AbCdEfGhIjKlMnOpQrStUvWxYz1234567890AbCdEfGhIjKlMnOpQrStUvWxYz", + "altsecid": "5::10032001XXXXXXXX", + "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd,mfa", + "appid": "66666666-6666-6666-6666-666666666666", + "appidacr": "0", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "john.doe@contoso.com", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "Doe", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "John", + "groups": "77777777-7777-7777-7777-777777777777,88888888-8888-8888-8888-888888888888", + "http://schemas.microsoft.com/identity/claims/identityprovider": "https://sts.windows.net/99999999-9999-9999-9999-999999999999/", + "idtyp": "user", + "ipaddr": "203.0.113.50", + "name": "John Doe", + "http://schemas.microsoft.com/identity/claims/objectidentifier": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", + "puid": "1003200000000000", + "rh": "1.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.", + "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation", + "sid": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxX", + "http://schemas.microsoft.com/identity/claims/tenantid": "55555555-5555-5555-5555-555555555555", + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "john.doe@contoso.com", + "uti": "XXXXXXXXXXXXXXXXXXXX", + "ver": "1.0", + "wids": "cccccccc-cccc-cccc-cccc-cccccccccccc,dddddddd-dddd-dddd-dddd-dddddddddddd", + "xms_edov": "true", + "xms_idrel": "5 2", + "xms_tcdt": "1391159646", + "correlationid": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" + } + }, + "level": "Warning", + "properties": { + "isComplianceCheck": "False", + "resourceLocation": "eastus2", + "ancestors": "55555555-5555-5555-5555-555555555555", + "policies": "[{\"policyDefinitionId\":\"/providers/Microsoft.Authorization/policyDefinitions/2b9ad585-36bc-4615-b300-fd4435808332\",\"policySetDefinitionId\":\"/providers/Microsoft.Authorization/policySetDefinitions/1f3afdf9-d0c9-4c3d-847f-89da613e70a8\",\"policyDefinitionReferenceId\":\"managedIdentityShouldBeUsedInYourWebAppMonitoringEffect\",\"policySetDefinitionName\":\"1f3afdf9-d0c9-4c3d-847f-89da613e70a8\",\"policySetDefinitionDisplayName\":\"Microsoft cloud security benchmark\",\"policySetDefinitionVersion\":\"57.49.0\",\"policyDefinitionName\":\"2b9ad585-36bc-4615-b300-fd4435808332\",\"policyDefinitionDisplayName\":\"App Service apps should use managed identity\",\"policyDefinitionVersion\":\"3.0.0\",\"policyDefinitionEffect\":\"AuditIfNotExists\",\"policyAssignmentId\":\"/subscriptions/11111111-1111-1111-1111-111111111111/providers/Microsoft.Authorization/policyAssignments/SecurityCenterBuiltIn\",\"policyAssignmentName\":\"SecurityCenterBuiltIn\",\"policyAssignmentDisplayName\":\"ASC Default (subscription: 11111111-1111-1111-1111-111111111111)\",\"policyAssignmentScope\":\"/subscriptions/11111111-1111-1111-1111-111111111111\",\"policyExemptionIds\":[],\"policyEnrollmentIds\":[]}]", + "eventCategory": "Policy", + "entity": "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/contoso-resources/providers/Microsoft.Web/sites/contoso-web-app", + "message": "Microsoft.Authorization/policies/audit/action", + "hierarchy": "" + }, + "tenantId": "55555555-5555-5555-5555-555555555555" + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log_expected.yaml new file mode 100644 index 0000000000000..f6cf11bbd95c4 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/policy/policy-log_expected.yaml @@ -0,0 +1,166 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/CONTOSO-RESOURCES/PROVIDERS/MICROSOFT.WEB/SITES/CONTOSO-WEB-APP + - key: azure.tenant.id + value: + stringValue: 55555555-5555-5555-5555-555555555555 + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: Policy + - key: azure.operation.name + value: + stringValue: MICROSOFT.AUTHORIZATION/POLICIES/AUDIT/ACTION + - key: azure.result.type + value: + stringValue: Success + - key: azure.result.signature + value: + stringValue: Succeeded. + - key: network.peer.address + value: + stringValue: 203.0.113.50 + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee + - key: azure.operation.duration + value: + intValue: "0" + - key: azure.identity.authorization.scope + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/contoso-resources/providers/Microsoft.Web/sites/contoso-web-app + - key: azure.identity.authorization.action + value: + stringValue: Microsoft.Web/sites/write + - key: azure.identity.authorization.evidence.role + value: + stringValue: Owner + - key: azure.identity.authorization.evidence.role.assignment.scope + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111 + - key: azure.identity.authorization.evidence.role.assignment.id + value: + stringValue: "22222222222222222222222222222222" + - key: azure.identity.authorization.evidence.role.definition.id + value: + stringValue: "33333333333333333333333333333333" + - key: azure.identity.authorization.evidence.principal.id + value: + stringValue: "44444444444444444444444444444444" + - key: azure.identity.authorization.evidence.principal.type + value: + stringValue: User + - key: azure.identity.issuer + value: + stringValue: https://sts.windows.net/55555555-5555-5555-5555-555555555555/ + - key: azure.identity.audience + value: + stringValue: https://management.core.windows.net/ + - key: azure.identity.not_after + value: + stringValue: "2025-04-23T11:51:58Z" + - key: azure.identity.not_before + value: + stringValue: "2025-04-23T10:46:40Z" + - key: azure.identity.created + value: + stringValue: "2025-04-23T10:46:40Z" + - key: azure.identity.scope + value: + stringValue: user_impersonation + - key: azure.identity.type + value: + stringValue: user + - key: azure.identity.application.id + value: + stringValue: 66666666-6666-6666-6666-666666666666 + - key: azure.identity.auth.methods.references + value: + stringValue: pwd,mfa + - key: azure.identity.provider + value: + stringValue: https://sts.windows.net/99999999-9999-9999-9999-999999999999/ + - key: azure.identity.identifier.object + value: + stringValue: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa + - key: user.name + value: + stringValue: XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxX + - key: user.email + value: + stringValue: john.doe@contoso.com + - key: azure.policy.compliance_check + value: + boolValue: false + - key: azure.location + value: + stringValue: eastus2 + - key: azure.policy.ancestors + value: + stringValue: 55555555-5555-5555-5555-555555555555 + - key: azure.policy.policies + value: + arrayValue: + values: + - kvlistValue: + values: + - key: definition.id + value: + stringValue: /providers/Microsoft.Authorization/policyDefinitions/2b9ad585-36bc-4615-b300-fd4435808332 + - key: definition.name + value: + stringValue: 2b9ad585-36bc-4615-b300-fd4435808332 + - key: definition.display_name + value: + stringValue: App Service apps should use managed identity + - key: definition.version + value: + stringValue: 3.0.0 + - key: definition.effect + value: + stringValue: AuditIfNotExists + - key: definition.reference_id + value: + stringValue: managedIdentityShouldBeUsedInYourWebAppMonitoringEffect + - key: set_definition.id + value: + stringValue: /providers/Microsoft.Authorization/policySetDefinitions/1f3afdf9-d0c9-4c3d-847f-89da613e70a8 + - key: set_definition.name + value: + stringValue: 1f3afdf9-d0c9-4c3d-847f-89da613e70a8 + - key: set_definition.display_name + value: + stringValue: Microsoft cloud security benchmark + - key: set_definition.version + value: + stringValue: 57.49.0 + - key: assignment.id + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111/providers/Microsoft.Authorization/policyAssignments/SecurityCenterBuiltIn + - key: assignment.name + value: + stringValue: SecurityCenterBuiltIn + - key: assignment.display_name + value: + stringValue: 'ASC Default (subscription: 11111111-1111-1111-1111-111111111111)' + - key: assignment.scope + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111 + body: {} + severityNumber: 13 + severityText: Warning + timeUnixNano: "1745406126696631900" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log.json index 36fba59d968ee..8c5cdc6127377 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log.json +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log.json @@ -1,10 +1,10 @@ { "records": [ { - "correlationId": "91e76397-5361-49bc-b814-a346a1761bb3", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-333333333333", "time": "2025-04-24T14:11:46.4216690Z", - "eventDataId": "d2ecfbc7-151b-48fd-ab16-4db80d087e7b", - "resourceId": "/SUBSCRIPTIONS/441fcba2-0b42-4269-b8b8-52f7c1cfb5d7/RESOURCEGROUPS/FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/FRONTDOOR-PROFILE", + "eventDataId": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/EXAMPLE-FRONTDOOR-PROFILE", "operationName": "Microsoft.Advisor/recommendations/available/action", "resultType": "Active", "properties": { @@ -12,7 +12,7 @@ "recommendationCategory": "HighAvailability", "recommendationImpact": "High", "recommendationName": "Consider having at least two origins", - "recommendationResourceLink": "https://portal.azure.com/#blade/Microsoft_Azure_Expert/RecommendationListBlade/source/ActivityLog/recommendationTypeId/589ab0b0-1362-44fd-8551-0e7847767600/resourceId/%2Fsubscriptions%2F441fcba2-0b42-4269-b8b8-52f7c1cfb5d7%2Fresourcegroups%2Fjohn-doe-frontdoor%2Fproviders%2FMicrosoft.Cdn%2Fprofiles%2Fjohn-doe-frontdoor-profile", + "recommendationResourceLink": "https://portal.azure.com/#blade/Microsoft_Azure_Expert/RecommendationListBlade/source/ActivityLog/recommendationTypeId/589ab0b0-1362-44fd-8551-0e7847767600/resourceId/%2Fsubscriptions%2F11111111-1111-1111-1111-111111111111%2Fresourcegroups%2Fexample-frontdoor%2Fproviders%2FMicrosoft.Cdn%2Fprofiles%2Fexample-frontdoor-profile", "recommendationType": "589ab0b0-1362-44fd-8551-0e7847767600" }, "location": "global", diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log_expected.yaml index 4b803a849640d..78a3afd7ed832 100644 --- a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log_expected.yaml +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/recommendation/recommendation-log_expected.yaml @@ -9,7 +9,7 @@ resourceLogs: stringValue: azure.resource.log - key: cloud.resource_id value: - stringValue: /SUBSCRIPTIONS/441fcba2-0b42-4269-b8b8-52f7c1cfb5d7/RESOURCEGROUPS/FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/FRONTDOOR-PROFILE + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/EXAMPLE-FRONTDOOR-PROFILE - key: cloud.region value: stringValue: global @@ -39,21 +39,13 @@ resourceLogs: stringValue: 0.0.0.0 - key: azure.correlation_id value: - stringValue: 91e76397-5361-49bc-b814-a346a1761bb3 + stringValue: aaaaaaaa-bbbb-cccc-dddd-333333333333 - key: azure.operation.duration value: intValue: "10" - - key: azure.identity - value: - kvlistValue: - values: - - key: claims - value: - kvlistValue: - values: - - key: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress - value: - stringValue: Microsoft.Advisor + - key: user.email + value: + stringValue: Microsoft.Advisor - key: azure.recommendation.category value: stringValue: HighAvailability @@ -71,7 +63,7 @@ resourceLogs: stringValue: "1.0" - key: azure.recommendation.link value: - stringValue: https://portal.azure.com/#blade/Microsoft_Azure_Expert/RecommendationListBlade/source/ActivityLog/recommendationTypeId/589ab0b0-1362-44fd-8551-0e7847767600/resourceId/%2Fsubscriptions%2F441fcba2-0b42-4269-b8b8-52f7c1cfb5d7%2Fresourcegroups%2Fjohn-doe-frontdoor%2Fproviders%2FMicrosoft.Cdn%2Fprofiles%2Fjohn-doe-frontdoor-profile + stringValue: https://portal.azure.com/#blade/Microsoft_Azure_Expert/RecommendationListBlade/source/ActivityLog/recommendationTypeId/589ab0b0-1362-44fd-8551-0e7847767600/resourceId/%2Fsubscriptions%2F11111111-1111-1111-1111-111111111111%2Fresourcegroups%2Fexample-frontdoor%2Fproviders%2FMicrosoft.Cdn%2Fprofiles%2Fexample-frontdoor-profile body: {} severityNumber: 9 severityText: Informational diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log.json new file mode 100644 index 0000000000000..c389d933a2b08 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log.json @@ -0,0 +1,22 @@ +{ + "records": [ + { + "time": "2025-04-24T12:49:14.6241035Z", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-444444444444", + "level": "Informational", + "resultType": "Active", + "operationName": "Microsoft.Resourcehealth/healthevent/Activated/action", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/EXAMPLE-FRONTDOOR-PROFILE", + "properties": { + "title": "Unknown", + "details": "Unknown", + "currentHealthStatus": "Unavailable", + "previousHealthStatus": "Available", + "type": "Downtime", + "cause": "Unknown" + }, + "category": "ResourceHealth", + "location": "global" + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log_expected.yaml new file mode 100644 index 0000000000000..a0b42c61c70d1 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/resourcehealth/resourcehealth-log_expected.yaml @@ -0,0 +1,55 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/RESOURCEGROUPS/EXAMPLE-FRONTDOOR/PROVIDERS/MICROSOFT.CDN/PROFILES/EXAMPLE-FRONTDOOR-PROFILE + - key: cloud.region + value: + stringValue: global + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: ResourceHealth + - key: azure.operation.name + value: + stringValue: Microsoft.Resourcehealth/healthevent/Activated/action + - key: azure.result.type + value: + stringValue: Active + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-444444444444 + - key: azure.resourcehealth.title + value: + stringValue: Unknown + - key: azure.resourcehealth.details + value: + stringValue: Unknown + - key: azure.resourcehealth.state + value: + stringValue: Unavailable + - key: azure.resourcehealth.previous_state + value: + stringValue: Available + - key: azure.resourcehealth.type + value: + stringValue: Downtime + - key: azure.resourcehealth.cause + value: + stringValue: Unknown + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1745498954624103500" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log.json new file mode 100644 index 0000000000000..7411938b9febd --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log.json @@ -0,0 +1,30 @@ +{ + "records": [ + { + "Level": 5, + "category": "Security", + "location": "centralus", + "resourceId": "/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/PROVIDERS/MICROSOFT.SECURITY/LOCATIONS/CENTRALUS/ALERTS/2518939942613820660_A48F8653-3FC6-4166-9F19-914F030A13D3", + "time": "2017-10-18T06:02:18.6179339Z", + "level": "Informational", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-555555555555", + "operationName": "MICROSOFT.SECURITY/LOCATIONS/ALERTS/ACTIVATE/ACTION", + "resultType": "Active", + "properties": { + "accountLogonId": "0x2r4", + "commandLine": "c:\\mydirectory\\doubleetension.pdf.exe", + "domainName": "hpc", + "parentProcess": "unknown", + "parentProcess id": "0", + "processId": "6988", + "processName": "c:\\mydirectory\\doubleetension.pdf.exe", + "userName": "exampleUser", + "UserSID": "S-3-2-12", + "ActionTaken": "Detected", + "Severity": "High", + "eventCategory": "Security" + }, + "tenantId": "22222222-2222-2222-2222-222222222222" + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log_expected.yaml new file mode 100644 index 0000000000000..0425b7d5ee9b0 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/security/security-log_expected.yaml @@ -0,0 +1,70 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/PROVIDERS/MICROSOFT.SECURITY/LOCATIONS/CENTRALUS/ALERTS/2518939942613820660_A48F8653-3FC6-4166-9F19-914F030A13D3 + - key: cloud.region + value: + stringValue: centralus + - key: azure.tenant.id + value: + stringValue: 22222222-2222-2222-2222-222222222222 + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: Security + - key: azure.operation.name + value: + stringValue: MICROSOFT.SECURITY/LOCATIONS/ALERTS/ACTIVATE/ACTION + - key: azure.result.type + value: + stringValue: Active + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-555555555555 + - key: process.command_line + value: + stringValue: c:\mydirectory\doubleetension.pdf.exe + - key: process.executable.path + value: + stringValue: c:\mydirectory\doubleetension.pdf.exe + - key: process.owner + value: + stringValue: exampleUser + - key: enduser.id + value: + stringValue: S-3-2-12 + - key: process.pid + value: + intValue: "6988" + - key: process.parent_pid + value: + intValue: "0" + - key: azure.security.account_logon_id + value: + stringValue: 0x2r4 + - key: azure.security.domain_name + value: + stringValue: hpc + - key: azure.security.action_taken + value: + stringValue: Detected + - key: azure.security.severity + value: + stringValue: High + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1508306538617933900" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log.json b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log.json new file mode 100644 index 0000000000000..51a1165bb1329 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log.json @@ -0,0 +1,52 @@ +{ + "records": [ + { + "Level": 5, + "category": "ServiceHealth", + "location": "global", + "resourceId": "/subscriptions/11111111-1111-1111-1111-111111111111", + "resultType": "Resolved", + "time": "2025-04-23T15:01:23.3361261Z", + "level": "Informational", + "correlationId": "aaaaaaaa-bbbb-cccc-dddd-666666666666", + "operationName": "Microsoft.ServiceHealth/maintenance/action", + "resultDescription": "Resolved: End of Routine Planned Maintenance for App Service in East US 2", + "properties": { + "title": "End of Routine Planned Maintenance for App Service in East US 2", + "service": "App Service", + "region": "East US 2", + "communication": "This notification is for previously scheduled maintenance to your Azure App Service instance(s) in East US 2. The maintenance has now completed. During this time, your Azure App Service application(s) may have seen a brief drop in connectivity for a few seconds. Please see frequently asked questions for maintenance notifications: Routine maintenance for App Service. For maximum availability and minimal disruption to your service, follow our list of best practices outlined here: The Ultimate Guide to Running Healthy Apps in the Cloud. Feel free to refer to your Azure portal for more information, FAQs regarding routine maintenance notifications, and for any additional questions, please contact support. Release Notes Release - Azure/AppService
To view your resource list in the region, click here. The resource list with maintenance status is now also available within Impacted Resources tab of this event from Service Health.", + "incidentType": "Maintenance", + "trackingId": "1_6N-3XG", + "impactStartTime": "2025-04-22T18:04:08.4705934Z", + "impactMitigationTime": "2025-05-06T18:04:08.4705934Z", + "impactedServices": "[{\"ImpactedRegions\":[{\"RegionName\":\"East US 2\",\"RegionId\":\"eastus2\"}],\"ServiceName\":\"App Service\",\"ServiceId\":\"appservice\",\"ServiceGuid\":\"75a8fc46-5105-4973-9aff-d070081824b4\"}]", + "impactedServicesTableRows": "\r\nApp Service\r\nEast US 2
\r\n\r\n", + "defaultLanguageTitle": "End of Routine Planned Maintenance for App Service in East US 2", + "defaultLanguageContent": "This notification is for previously scheduled maintenance to your Azure App Service instance(s) in East US 2. The maintenance has now completed. During this time, your Azure App Service application(s) may have seen a brief drop in connectivity for a few seconds. Please see frequently asked questions for maintenance notifications: Routine maintenance for App Service. For maximum availability and minimal disruption to your service, follow our list of best practices outlined here: The Ultimate Guide to Running Healthy Apps in the Cloud. Feel free to refer to your Azure portal for more information, FAQs regarding routine maintenance notifications, and for any additional questions, please contact support. Release Notes Release - Azure/AppService
To view your resource list in the region, click here. The resource list with maintenance status is now also available within Impacted Resources tab of this event from Service Health.", + "stage": "Complete", + "communicationId": "11000201860740", + "maintenanceId": "202504221704/bn1", + "maintenanceType": "", + "isHIR": false, + "IsSynthetic": "False", + "impactType": "SubscriptionList", + "emailTemplateId": "tem_jCfXGkcx9fMD8p4hq6XVdGv9", + "emailTemplateFullVersion": "ver_YmZvWy6k7MwmcYj3KYg8MloA", + "emailTemplateLocale": "en-US", + "smsText": "Resolved:End of Routine Planned Maintenance for App Service in East US 2", + "impactCategory": "Other", + "communicationRouteType": "Normal", + "version": "0.1.1", + "argQuery": "", + "oldRate": 0.0, + "newRate": 0.0 + }, + "identity": { + "claims": { + "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "AcmClient@microsoft.com" + } + } + } + ] +} diff --git a/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log_expected.yaml b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log_expected.yaml new file mode 100644 index 0000000000000..b5ad5f4afcf69 --- /dev/null +++ b/extension/encoding/azureencodingextension/internal/unmarshaler/logs/testdata/servicehealth/servicehealth-log_expected.yaml @@ -0,0 +1,118 @@ +resourceLogs: + - resource: + attributes: + - key: cloud.provider + value: + stringValue: azure + - key: cloudevents.event_source + value: + stringValue: azure.resource.log + - key: cloud.resource_id + value: + stringValue: /subscriptions/11111111-1111-1111-1111-111111111111 + - key: cloud.region + value: + stringValue: global + scopeLogs: + - logRecords: + - attributes: + - key: azure.category + value: + stringValue: ServiceHealth + - key: azure.operation.name + value: + stringValue: Microsoft.ServiceHealth/maintenance/action + - key: azure.result.type + value: + stringValue: Resolved + - key: azure.result.description + value: + stringValue: 'Resolved: End of Routine Planned Maintenance for App Service in East US 2' + - key: azure.correlation_id + value: + stringValue: aaaaaaaa-bbbb-cccc-dddd-666666666666 + - key: user.email + value: + stringValue: AcmClient@microsoft.com + - key: azure.servicehealth.title + value: + stringValue: End of Routine Planned Maintenance for App Service in East US 2 + - key: azure.servicehealth.service + value: + stringValue: App Service + - key: azure.servicehealth.region + value: + stringValue: East US 2 + - key: azure.servicehealth.communication.body + value: + stringValue: 'This notification is for previously scheduled maintenance to your Azure App Service instance(s) in East US 2. The maintenance has now completed. During this time, your Azure App Service application(s) may have seen a brief drop in connectivity for a few seconds. Please see frequently asked questions for maintenance notifications: Routine maintenance for App Service. For maximum availability and minimal disruption to your service, follow our list of best practices outlined here: The Ultimate Guide to Running Healthy Apps in the Cloud. Feel free to refer to your Azure portal for more information, FAQs regarding routine maintenance notifications, and for any additional questions, please contact support. Release Notes Release - Azure/AppService
To view your resource list in the region, click here. The resource list with maintenance status is now also available within Impacted Resources tab of this event from Service Health.' + - key: azure.servicehealth.incident.type + value: + stringValue: Maintenance + - key: azure.servicehealth.tracking.id + value: + stringValue: 1_6N-3XG + - key: azure.servicehealth.impact.start + value: + stringValue: "2025-04-22T18:04:08.4705934Z" + - key: azure.servicehealth.impact.mitigation + value: + stringValue: "2025-05-06T18:04:08.4705934Z" + - key: azure.servicehealth.default_language.title + value: + stringValue: End of Routine Planned Maintenance for App Service in East US 2 + - key: azure.servicehealth.default_language.content + value: + stringValue: 'This notification is for previously scheduled maintenance to your Azure App Service instance(s) in East US 2. The maintenance has now completed. During this time, your Azure App Service application(s) may have seen a brief drop in connectivity for a few seconds. Please see frequently asked questions for maintenance notifications: Routine maintenance for App Service. For maximum availability and minimal disruption to your service, follow our list of best practices outlined here: The Ultimate Guide to Running Healthy Apps in the Cloud. Feel free to refer to your Azure portal for more information, FAQs regarding routine maintenance notifications, and for any additional questions, please contact support. Release Notes Release - Azure/AppService
To view your resource list in the region, click here. The resource list with maintenance status is now also available within Impacted Resources tab of this event from Service Health.' + - key: azure.servicehealth.state + value: + stringValue: Complete + - key: azure.servicehealth.communication.id + value: + stringValue: "11000201860740" + - key: azure.servicehealth.maintenance.id + value: + stringValue: 202504221704/bn1 + - key: azure.servicehealth.impact.type + value: + stringValue: SubscriptionList + - key: azure.servicehealth.impact.category + value: + stringValue: Other + - key: azure.servicehealth.is_synthetic + value: + boolValue: false + - key: azure.servicehealth.impact.services + value: + arrayValue: + values: + - kvlistValue: + values: + - key: name + value: + stringValue: App Service + - key: id + value: + stringValue: appservice + - key: guid + value: + stringValue: 75a8fc46-5105-4973-9aff-d070081824b4 + - key: regions + value: + arrayValue: + values: + - kvlistValue: + values: + - key: name + value: + stringValue: East US 2 + - key: id + value: + stringValue: eastus2 + body: {} + severityNumber: 9 + severityText: Informational + timeUnixNano: "1745420483336126100" + scope: + name: github.com/open-telemetry/opentelemetry-collector-contrib/extension/encoding/azureencodingextension + version: test-version