Skip to content

New Cloud Platform Monitoring: Linking and enriching logs records #169

@ajoga

Description

@ajoga

Is your feature request related to a problem? Please describe.
When shipping ALB/S3 Access logs/Classic LB access logs to Dynatrace using this solution, Dynatrace doesn't link them to the resource that generated them.
The logs don't get the topology information and aren't linked to the entity that generated them.
They also don't get the AWS tags the resource that generated them has.

Describe the solution you'd like
For Cloudwatch logs, Dynatrace suggests to push the logs via AWS Firehose to a dedicated endpoint. With the AWS Firehose adding HTTP headers for aws.arn and aws.resource.type enables the logs to be enriched with the AWS tags and dt.security_context the entity with the same ARN has.
We'd like to also have s3-shipped logs be enriched with the AWS tags & dt.security_context field set upon ingestion.

Describe alternatives you've considered
For a classic load balancer access log, I tried adding a log_forwarding_rules that add the annotations aws.arn and aws.resource.type as suggested for Cloudwatch logs that are pushed via a dedicated firehose, but this solution doesn't use the Firehose endpoint (/api/v2/logs/ingest/aws_firehose) so it doesn't work.
I understand this solution uses /api/v2/logs/ingest, the documentation says that "All attributes should preferably map to semantic attributes for Dynatrace to interpret them correctly." and list them in the same page. I've set the forwarding rule to pass all of the attributes that relate to aws (except aws.log_group & aws.log_stream as they're not applicable) with no success (aws.account.id, aws.arn, aws.region, aws.resource.id, aws.resource.type, aws.service).

Additional context

I've opened a support ticket, reference 608830. I'm don't think I can disclose its outcome, but maybe someone from Dynatrace can.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions