Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions solutions/observability/streams/management/extract.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Streams supports the following processors:
- [**Date**](./extract/date.md): Converts date strings into timestamps, with options for timezone, locale, and output formatting.
- [**Dissect**](./extract/dissect.md): Extracts fields from structured log messages using defined delimiters instead of patterns, making it faster than Grok and ideal for consistently formatted logs.
- [**Drop**](./extract/drop.md): Drops the document without raising any errors. This is useful to prevent the document from getting indexed based on a condition.
- [**Enrich**](./extract/enrich.md): Adds data from an enrich policy to incoming documents, such as geographic coordinates from an IP address or account details from a user ID.
- [**Grok**](./extract/grok.md): Extracts fields from unstructured log messages using predefined or custom patterns, supports multiple match attempts in sequence, and can automatically generate patterns with an [LLM connector](/explore-analyze/ai-features/llm-guides/llm-connectors.md).
- [**Join**](./extract/join.md): Concatenates the values of multiple fields with a delimiter.
- [**Lowercase**](./extract/lowercase.md): Converts a string field to lowercase.
Expand Down
47 changes: 47 additions & 0 deletions solutions/observability/streams/management/extract/enrich.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
applies_to:
serverless: ga
stack: ga 9.4+
products:
- id: observability
- id: elasticsearch
- id: kibana
- id: cloud-serverless
- id: cloud-hosted
- id: cloud-enterprise
- id: cloud-kubernetes
- id: elastic-stack
---
Comment thread
mdbirnstiehl marked this conversation as resolved.

# Enrich processor [streams-enrich-processor]

The **Enrich** processor adds data from an existing [enrich policy](elasticsearch://reference/enrich-processor/enrich-processor.md) to incoming documents during processing. Use it to look up and append supplemental data, such as geographic coordinates from an IP address or account details from a user ID, without modifying the original source.

Before using the enrich processor, you must have at least one enrich policy configured in {{es}}. Refer to [Enrich your data](/manage-data/ingest/transform-enrich/data-enrichment.md) for setup instructions.

To enrich documents:

1. Select **Create** → **Create processor**.
1. Select **Enrich** from the **Processor** menu.
1. Select an **Enrich policy** from the list of available policies.
1. Set **Target field** to the field where the enriched data is stored.

This functionality uses the {{es}} [Enrich processor](elasticsearch://reference/enrich-processor/enrich-processor.md) internally, but you configure it in Streamlang. Streamlang doesn't always have 1:1 parity with the ingest processor options and behavior. Refer to [Processor limitations and inconsistencies](../extract.md#streams-processor-inconsistencies).

## YAML reference [streams-enrich-yaml-reference]

In [YAML mode](../extract.md#streams-editing-yaml-mode), configure the enrich processor using the following parameters. For the complete Streamlang syntax, refer to the [Streamlang reference](../streamlang.md).

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `policy_name` | string | Yes | The name of the enrich policy to use. |
| `to` | string | Yes | Target field for the enriched data. |
| `override` | boolean | No | When `true`, overwrite pre-existing non-null field values. Defaults to `true`. |
| `ignore_missing` | boolean | No | When `true`, skip this processor if the source field is missing. |

```yaml
- action: enrich
policy_name: ip_location
to: attributes.geo
override: true
```
25 changes: 13 additions & 12 deletions solutions/observability/streams/management/streamlang.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,24 +57,25 @@ The following table lists all available processors. Refer to the individual proc
| Action | Description |
| --- | --- |
| [`append`](./extract/append.md) | Adds values to an array field, or creates the field as an array if it doesn't exist. |
| [`concat`](./extract/concat.md) | Concatenates a mix of field values and literal strings into a single field. |
| [`convert`](./extract/convert.md) | Converts a field value to a different data type. |
| [`concat`](./extract/concat.md) | {applies_to}`stack: ga 9.4+` Concatenates a mix of field values and literal strings into a single field. |
| [`convert`](./extract/convert.md) | {applies_to}`stack: ga 9.3+` Converts a field value to a different data type. |
| [`date`](./extract/date.md) | Parses date strings into timestamps. |
| [`dissect`](./extract/dissect.md) | Parses structured text using delimiter-based patterns. |
| [`drop_document`](./extract/drop.md) | Prevents indexing of a document from based on a condition. |
| [`drop_document`](./extract/drop.md) | {applies_to}`stack: ga 9.3+` Prevents indexing of a document from based on a condition. |
| [`enrich`](./extract/enrich.md) | {applies_to}`stack: ga 9.4+` Adds data from an enrich policy to incoming documents. |
| [`grok`](./extract/grok.md) | Parses unstructured text using predefined or custom patterns. |
| [`join`](./extract/join.md) | Concatenates the values of multiple fields with a delimiter. |
| [`lowercase`](./extract/lowercase.md) | Converts a string field to lowercase. |
| [`math`](./extract/math.md) | Evaluates an arithmetic expression and stores the result. |
| [`network_direction`](./extract/network-direction.md) | Determines network traffic direction based on source and destination IP addresses. |
| [`redact`](./extract/redact.md) | Redacts sensitive data in a string field by matching patterns. |
| [`remove`](./extract/remove.md) | Removes a field from the document. |
| [`join`](./extract/join.md) | {applies_to}`stack: ga 9.4+` Concatenates the values of multiple fields with a delimiter. |
| [`lowercase`](./extract/lowercase.md) | {applies_to}`stack: ga 9.4+` Converts a string field to lowercase. |
| [`math`](./extract/math.md) | {applies_to}`stack: ga 9.3+` Evaluates an arithmetic expression and stores the result. |
| [`network_direction`](./extract/network-direction.md) | {applies_to}`stack: ga 9.4+` Determines network traffic direction based on source and destination IP addresses. |
| [`redact`](./extract/redact.md) | {applies_to}`stack: ga 9.4+` Redacts sensitive data in a string field by matching patterns. |
| [`remove`](./extract/remove.md) | {applies_to}`stack: ga 9.3+` Removes a field from the document. |
| [`remove_by_prefix`](./extract/remove.md#streams-remove-by-prefix-processor) | Removes a field and all nested fields matching a prefix. |
| [`rename`](./extract/rename.md) | Moves a field's value to a new field name and removes the original. |
| [`replace`](./extract/replace.md) | Replaces portions of a string field that match a regular expression. |
| [`replace`](./extract/replace.md) | {applies_to}`stack: ga 9.3+` Replaces portions of a string field that match a regular expression. |
| [`set`](./extract/set.md) | Assigns a value to a field, creating the field if it doesn't exist. |
| [`trim`](./extract/trim.md) | Removes leading and trailing whitespace from a string field. |
| [`uppercase`](./extract/uppercase.md) | Converts a string field to uppercase. |
| [`trim`](./extract/trim.md) | {applies_to}`stack: ga 9.4+` Removes leading and trailing whitespace from a string field. |
| [`uppercase`](./extract/uppercase.md) | {applies_to}`stack: ga 9.4+` Converts a string field to uppercase. |

## Conditions [streams-streamlang-conditions]

Expand Down
1 change: 1 addition & 0 deletions solutions/toc.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
project: "Solutions and use cases"
toc:
- file: index.md
Expand Down Expand Up @@ -465,6 +465,7 @@
- file: observability/streams/management/extract/date.md
- file: observability/streams/management/extract/dissect.md
- file: observability/streams/management/extract/drop.md
- file: observability/streams/management/extract/enrich.md
- file: observability/streams/management/extract/grok.md
- file: observability/streams/management/extract/join.md
- file: observability/streams/management/extract/lowercase.md
Expand Down
Loading