Skip to content

Commit

Permalink
Merge pull request #74 from axoflow/some-filterx-fixes
Browse files Browse the repository at this point in the history
Some filterx fixes
  • Loading branch information
fekete-robert authored Oct 22, 2024
2 parents a09338f + e9dbcae commit d7c50d4
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 32 deletions.
14 changes: 7 additions & 7 deletions content/filterx/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ filterx {
The same is true for complex objects, like JSON, for example:

```shell
js = json_object({
js = json({
"key": "value",
"second-key": "another-value"
});
Expand Down Expand Up @@ -145,7 +145,7 @@ Variables can have the following types. All of these types have a matching funct
- `dict`
- `double`
- `int`
- [`json, json_object`]({{< relref "/filterx/function-reference.md#json" >}}) and [`json_array`]({{< relref "/filterx/function-reference.md#json-array" >}}) for JSON or JSON-like objects. The `json` type is an alias for the `json_object` type.
- [`json`]({{< relref "/filterx/function-reference.md#json" >}}) and [`json_array`]({{< relref "/filterx/function-reference.md#json-array" >}}) for JSON or JSON-like objects.
- `list`
- `null`
- `otel_array`
Expand Down Expand Up @@ -254,7 +254,7 @@ my_list = []; # Creates an empty list (which defaults to a JSON list)
my_array = {}; # Creates an empty dictionary (which defaults to a JSON object)

my_list2 = json_array(); # Creates an empty JSON list
my_array2 = json_object(); # Creates an empty JSON object. json() is an alias for json_object()
my_array2 = json(); # Creates an empty JSON object.
```

You can add elements to lists and dictionaries like this:
Expand Down Expand Up @@ -287,7 +287,7 @@ ${MESSAGE} = list;

In all three cases, the value of `${MESSAGE}` is the same JSON array: `["first_element", "second_element", "third_element"]`.

You can define JSON objects using the `json_object` type, for example:
You can define JSON objects using the `json()` type, for example:

```shell
js1 = json();
Expand All @@ -300,14 +300,14 @@ js1 += {
}
};

js2 = json_object({"key": "value"})
js2 = json({"key": "value"})
```

Naturally, you can assign values from other variables to an object, for example:

```shell
js = json_array(["foo", "bar", "baz"]);
${MESSAGE} = json_object({
${MESSAGE} = json({
"key": "value",
"list": list
});
Expand All @@ -316,7 +316,7 @@ ${MESSAGE} = json_object({
or

```shell
js = json_object({
js = json({
"key": ${MY-NAME-VALUE-PAIR},
"key-from-expression": isset(${HOST}) ? ${HOST} : "default-hostname",
"list": list
Expand Down
44 changes: 22 additions & 22 deletions content/filterx/function-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Functions have arguments that can be either mandatory or optional.

## cache_json_file {#cache-json-file}

Load the contents of an external JSON file in an efficient manner. You can use this to lookup contextual information. (Basically, this is a FilterX-specific implementation of the [`add-contextual-data() functionality`]({{< relref "/chapter-enrich-data/data-enrichment-add-contextual-data/_index.md" >}}).)
Load the contents of an external JSON file in an efficient manner. You can use this function to lookup contextual information. (Basically, this is a FilterX-specific implementation of the [`add-contextual-data() functionality`]({{< relref "/chapter-enrich-data/data-enrichment-add-contextual-data/_index.md" >}}).)

Usage: `cache_json_file("/path/to/file.json")`

Expand Down Expand Up @@ -61,8 +61,8 @@ date = datetime("1701350398.123000+01:00");

Usually, you use the [strptime](#strptime) FilterX function to create datetime values. Alternatively, you can cast an integer, double, string, or isodate variable into datetime with the `datetime()` FilterX function. Note that:

- When casting from an integer, the integer is the number of microseconds elapsed since the UNIX epoch (January 1, 1970 12:00:00 AM).
- When casting from a double, the double is the number of seconds elapsed since the UNIX epoch (January 1, 1970 12:00:00 AM). (The part before the floating points is the seconds, the part after the floating point is the microseconds.)
- When casting from an integer, the integer is the number of microseconds elapsed since the UNIX epoch (00:00:00 UTC on 1 January 1970).
- When casting from a double, the double is the number of seconds elapsed since the UNIX epoch (00:00:00 UTC on 1 January 1970). (The part before the floating points is the seconds, the part after the floating point is the microseconds.)
- When casting from a string, the string (for example, `1701350398.123000+01:00`) is interpreted as: `<the number of seconds elapsed since the UNIX epoch>.<microseconds>+<timezone relative to UTC (GMT +00:00)>`

## flatten
Expand Down Expand Up @@ -90,7 +90,7 @@ Only the input is mandatory, other arguments are optional. Note that the delimit

By default, the delimiter is the comma (`delimiter=","`), the `columns` and `default_value` are empty.

If the `columns` option is set, {{< product >}} checks that the number of entries in the input data matches the number of columns. If there are fewer entries, it adds the `default_value` to the missing entries.
If the `columns` option is set, {{< product >}} checks that the number of fields or entries in the input data matches the number of columns. If there are fewer items, it adds the `default_value` to the missing entries.

## format_kv {#format-kv}

Expand Down Expand Up @@ -137,18 +137,20 @@ Usage: `istype(object, "type_str")`
For example:

```shell
istype({"key": "value"}, "json_object"); # True
obj = json();
istype(obj, "json_object"); # True

istype(${PID}, "string");
istype(my-local-json-object.mylist, "json_array");
```

If the object doesn't exist, `istype()` returns with an error, causing the FilterX statement to become false, and logs an error message to the `internal()` source of {{< product >}}.

## json, json_object {#json}
## json {#json}

Cast a value into a JSON object. `json_object()` is an alias for `json()`.
Cast a value into a JSON object.

Usage: `json(<string or expression to cast as json>)`
Usage: `json(<string or expression to cast to json>)`

For example:

Expand All @@ -160,7 +162,7 @@ js = json({"key": "value"});

Cast a value into a JSON array.

Usage: `json_array(<string or expression to cast as json array>)`
Usage: `json_array(<string or expression to cast to json array>)`

For example:

Expand All @@ -176,7 +178,7 @@ Usage: `len(object)`

## lower

Converts a string into lowercase characters.
Converts all characters of a string lowercase characters.

Usage: `lower(string)`

Expand All @@ -202,19 +204,19 @@ Creates an [OpenTelemetry scope object]({{< relref "/filterx/filterx-otel/_index

## parse_csv {#parse-csv}

Separate a comma-separated or similar string.
Split a comma-separated or similar string.

Usage: `parse_csv(msg_str [columns=json_array, delimiter=string, string_delimiters=json_array, dialect=string, strip_whitespace=boolean, greedy=boolean])`

For details, see {{% xref "/filterx/filterx-parsing/csv/_index.md" %}}.

## parse_kv {#parse-kv}

Separate a string consisting of whitespace or comma-separated `key=value` pairs (for example, WELF-formatted messages).
Split a string consisting of whitespace or comma-separated `key=value` pairs (for example, WELF-formatted messages).

Usage: `parse_kv(msg, value_separator="=", pair_separator=", ", stray_words_key="stray_words")`

The `value_separator` must be a single-character string. The `pair_separator` must be a string.
The `value_separator` must be a single character. The `pair_separator` can consist of multiple characters.

For details, see {{% xref "/filterx/filterx-parsing/key-value-parser/_index.md" %}}.

Expand Down Expand Up @@ -255,7 +257,7 @@ ${MY-LIST}.named = regexp_search("first-word second-part third", /(?<one>first-w

### Mixed match groups

If you use mixed (some named, some unnamed) groups, the output is a dictionary, where {{< product >}} automatically assigns a key to the unnamed groups. For example:
If you use mixed (some named, some unnamed) groups in your regular expression, the output is a dictionary, where {{< product >}} automatically assigns a key to the unnamed groups. For example:

```shell
${MY-LIST} = json(); # Creates an empty JSON object
Expand Down Expand Up @@ -286,19 +288,17 @@ regexp_subst(${MESSAGE}, "IP", "IP-Address", global=true);

{{< include-headless "chunk/filterx-regexp-notes.md" >}}

For a case sensitive search, use the `ignorecase=true` option.

### Options

You can use the following flags with the `regexp_subst` function:

- `global=true`:

Replace every occurrence of the search string.
Replace every match of the regular expression, not only the first one.

- `ignorecase=true`:

Do case insensitive search.
Do case insensitive match.

- `jit=true`:

Expand All @@ -310,7 +310,7 @@ You can use the following flags with the `regexp_subst` function:

## string

Cast a value into a string. Note currently {{< product >}} evaluates strings and executes [template functions]({{< relref "/filterx/_index.md#template-functions" >}}) and template expressions. In the future, template evaluation will be moved to a separate FilterX function.
Cast a value into a string. Note that currently {{< product >}} evaluates strings and executes [template functions]({{< relref "/filterx/_index.md#template-functions" >}}) and template expressions within the strings. In the future, template evaluation will be moved to a separate FilterX function.

Usage: `string(<string or expression to cast>)`

Expand Down Expand Up @@ -340,11 +340,11 @@ If none of the format strings match, `strptime` returns the null value and logs

{{% /alert %}}

You can use the following elements in the format string:
You can use the following format codes in the format string:

{{< include-headless "chunk/date-string-format.md" >}}

The [`isodate`](#isodate) FilterX function is a specialized version of `strptime` that accepts only a fixed format.
The [`isodate`](#isodate) FilterX function is a specialized variant of `strptime`, that accepts only a fixed format.

## unset

Expand Down Expand Up @@ -389,7 +389,7 @@ Usage: `unset_empties(object, recursive=true)`

## upper

Converts a string into uppercase characters.
Converts all characters of a string uppercase characters.

Usage: `upper(string)`

Expand Down
5 changes: 2 additions & 3 deletions content/headless/chunk/date-string-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,10 @@
{{% alert title="Warning" color="warning" %}}
When using the %z and %Z format elements, consider that while %z strictly expects a specified timezone, and triggers a warning if the timezone is not specified, %Z does not trigger a warning if the timezone is not specified.
When using the `%z` and `%Z` format codes, consider that while `%z` strictly expects a specified timezone, and triggers a warning if the timezone is missing, `%Z` does not trigger a warning if the timezone is not specified.
For further information about the %z and %Z format elements, see the 'DESCRIPTION' section on the [srtptime(3) - NetBSD Manual Pages](https://man.netbsd.org/NetBSD-7.0/i386/strptime.3).
For further information about the `%z` and `%Z` format codes, see the 'DESCRIPTION' section on the [srtptime(3) - NetBSD Manual Pages](https://man.netbsd.org/NetBSD-7.0/i386/strptime.3).
{{% /alert %}}
For example, for the date `01/Jan/2016:13:05:05 PST` use the following format string: `"%d/%b/%Y:%H:%M:%S %Z"`

0 comments on commit d7c50d4

Please sign in to comment.