Skip to content

loki.process should support extracting data from strucutured metadata #4717

@baurmatt

Description

@baurmatt

Background

When passing log streams to loki.process, it's currently not possible to use existing structured metadata. Being able to work with existing structured metadata is crucial for transform those log entries.

Use case

We send log data to Alloy via loki.source.api. Received log entries contain a structured metadata field called pod. We need to transform it into a log label called instance.

Proposal

Add a new stage stage.structured_metadata_unpack which unpacks existing structured metadata into extracted values. This is done by 1:1 copying all data over. Optionally one could implement exclude/include parameters for this stage to limit the structured metadata to certain keys(/values).

Tip

React with 👍 if this issue is important to you.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Likely Accept

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions