axosyslog-4.19.0

4.19.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• dict_to_pairs() FilterX function: Added a new function to convert dicts to list of pairs

  Example usage:

  dict = {
      "value_1": "foo",
      "value_2": "bar",
      "value_3": ["baz", "bax"],
  };
  
  list = dict_to_pairs(dict, "key", "value");
  # Becomes:
  # [
  #   {"key":"value_1","value":"foo"},
  #   {"key":"value_2","value":"bar"},
  #   {"key":"value_3","value":["baz","bax"]}
  # ]
  

  (#810)

Bugfixes

• syslogng_output_unreachable metric: fix marking destinations unreachable during reload
  (#818)

• transport(proxied-tcp): Fix a HAProxy protocol v2 parsing issue that
  caused a failed assertion. This essentially triggers a crash with a SIGABRT
  whenever a "LOCAL" command was sent in the HAProxy header without address
  information.
  (#814)

• filterx: fix parse_csv function crash if coulumns specified non-existent variable
  (#819)

• opentelemetry() source: fix various crashes during startup/reload
  (#822)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, László Várady,
Szilard Parrag, Tamás Kosztyu, shifter

axosyslog-4.18.1

4.18.1

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Bugfixes

• strftime() FilterX function: Fixed %Z formatting for some rare cases

  America/Caracas (-04:30) time offset will now be correctly formatted.
  (#811)

• disk-buffer(): fix getting stuck under rare circumstances
  (#813)

• disk-buffer(): do not allow flow-control misconfiguration
  The flow-control-window_size() (formerly mem-buf-length()) option is now deprecated and no longer has any
  effect.
  (#813)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, Hofi, László Várady,
Szilard Parrag, Tamás Kosztyu, shifter

axosyslog-4.18.0

4.18.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• http(): Added support for templated headers()

  In case of batching the templates in headers() will be calculated
  from the first message. Make sure to use worker-partition-key() to
  group similar messages together.

  Literal dollar signs ($) used in headers() must be escaped like $$.
  (#794)

• FilterX: unary + and - operators

  Useful for dynamic string slicing, for example:

  str[..-tempvar]
  

  (#788)

• FilterX parse_csv(): add quote_pairs parameter

  For example:

  filterx {
    str = "sarga,[bogre],'gorbe'";
    $MSG = parse_csv(str, quote_pairs=["[]", "'"]);
  };
  

  (#804)

Bugfixes

• FilterX in operator: fix crash when left- or right-hand side operand evaluation fails
  (#798)

• in FilterX operator: Fixed finding message_values in arrays.
  (#791)

• python(): LogTemplate::format() now returns a bytes object

  In the Python bindings, LogMessage is not UTF-8 or Unicode–safe by default.
  This means developers must explicitly call decode() on message fields and handle any decoding errors themselves.

  Since LogTemplate operates on LogMessage fields, this behavior also applies to it.

  Breaking change:
  When using templates, you now need to decode the result manually. For example:
  format().decode("utf-8", errors="backslashreplace")
  (#799)

• in FilterX operator: Fixed possible memory corruption regarding unreferencing an operand.
  (#792)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, Hofi, László Várady,
Szilard Parrag, Tamás Kosztyu, shifter

axosyslog-charts-0.18.0

AxoSyslog for Kubernetes

axosyslog-4.17.0

4.17.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• FilterX dpath(): new function to set a potentially non-existing path within a dict

  For example,

  dpath(dict.path.to["complex.n-v"].create) = {...};
  

  (#746)

• FilterX string slices: support negative indexing

  For example,

  filterx {
    str = "example";
    str[..-2] == "examp";
    str[-3..] == "ple";
  };
  

  (#780)

• parallelize(): add batch-size() option and other perf improvements

  The batch-size() option specifies, for each input thread, how many consecutive
  messages should be processed by a single parallelize() worker.

  This ensures that this many messages preserve their order on the output side
  and also improves parallelize() performance. A value around 100 is recommended.
  (#757)

• clickhouse-destination(): new json-var() directive

  The new json-var() option accepts either a JSON template or a variable containing a JSON string, and sends it to the ClickHouse server in Protobuf/JSON mixed mode (JSONEachRow format). In this mode, type validation is performed by the ClickHouse server itself, so no Protobuf schema is required for communication.

  This option is mutually exclusive with proto-var(), server-side-schema(), schema(), and protobuf-schema() directives.

  example:

     destination {
        clickhouse (
        ...
        json-var("$json_data");
  or
        json-var(json("{\"ingest_time\":1755248921000000000, \"body\": \"test template\"}"))
        ...
        };
     };
  

  (#761)

• FilterX parse_kv(): add stray_words_append_to_value flag

  For example,

  filterx {
    $MSG = parse_kv($MSG, value_separator="=", pair_separator=" ", stray_words_append_to_value=true);
  };
  
  input: a=b b=c d e f=g
  output: {"a":"b","b":"c d e","f":"g"}
  

  (#770)

Bugfixes

• filterx: fix startswith()/endswith()/includes() functions early free
  (#772)

• /string/: fix escaping

  Strings between / characters are now treated literally. The only exception is \/, which can be used to represent a / character.

  This syntax allows you to construct regular expression patterns without worrying about double escaping.

  Note: Because of the simplified escaping rules, you cannot represent strings that end with a single \ character,
  but such strings would not be valid regular expression patterns anyway.
  (#776)

• program() destination: Fix invalid access of freed log-writer cfg.
  (#779)

• ebpf(): acquire CAP_BPF before loading eBPF programs

  Previously, when AxoSyslog was compiled with Linux capabilities enabled,
  the ebpf() module was unable to load programs.
  (#768)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, Ben Ireland,
László Várady, Szilard Parrag, Tamás Kosztyu, shifter

axosyslog-4.16.0

4.16.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• New $PROTO_NAME macro: add a $PROTO_NAME macro that expands to "tcp" or
  "udp" depending on the transport used by syslog-ng.
  (#724)

Bugfixes

• Fix memory leaks during configuration parsing
  (#755)

• grpc based destinations: Fixed a race condition around syslogng_output_grpc_requests_total metrics
  (#754)

• $PEER_PORT: fix the value for the PEER_PORT macro, as it was incorrectly
  reversing the digits in the port value, port 514 would become port 415.
  (#724)

• google-pubsub(), logscale(), openobserver(), splunk(), and other batching destinations: fix slowdown

  The default value of batch-timeout() is now 0.
  This prevents artificial slowdowns in destinations when flow control is enabled
  and the log-iw-size() option of sources is set to a value lower than batch-lines().

  If you enable batch-timeout(), you can further improve batching performance,
  but you must also adjust the log-iw-size() option of your sources accordingly:

  log-iw-size / max-connections >= batch-lines * workers
  (#753)

FilterX features

• str_strip(), str_lstrip(), str_rstrip(): new string transformation functions to remove leading/trailing whitespaces from a string
  (#745)

FilterX bugfixes

• in operator: fix crash, that happened, when the list was declared as an operand

  Example:

  if ("test" in ["foo", "bar", "test"]) {
    $MSG = "YES";
  };
  

  (#759)

• parse_cef(), parse_leef(): Renamed some parsed header fields.

  These are breaking changes

  The motivation behind the renaming is that these names were too
  generic, and there is a chance to find them in the extensions.

  parse_cef():

  • version -> cef_version
  • name -> event_name

  parse_leef():

  • version -> leef_version
  • vendor -> vendor_name
  • delimiter -> leef_delimiter
    (#748)

Other changes

• The base of the AxoSyslog container image is updated to Alpine 3.22
  (#758)
• mongodb(): support libmongoc v2

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, László Várady,
Tamás Kosztyu

axosyslog-charts-0.17.0

AxoSyslog for Kubernetes

axosyslog-4.15.0

4.15.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• http: Added templating support to body-prefix()

  In case of batching the templates in body-prefix() will be calculated
  from the first message. Make sure to use worker-partition-key() to
  group similar messages together.

  Literal dollar signs ($) used in body-prefix() must be escaped like $$.

  Example usage:

  http(
    ...
    body-prefix('{"log_type": "$log_type", "entries": [')
    body('"$MSG"')
    delimiter(",")
    body-suffix("]}")
  
    batch-lines(1000)
    batch-timeout(1000)
    worker-partition-key("$log_type")
  );
  

  (#731)

• cloud-auth(): Added scope() option for gcp(service-account())

  Can be used for authentications using scope instead of audiance.
  For more info, see: https://google.aip.dev/auth/4111#scope-vs-audience

  Example usage:

  http(
    ...
    cloud-auth(
      gcp(
        service-account(
          key("/path/to/secret.json")
          scope("https://www.googleapis.com/auth/example-scope")
        )
      )
    )
  );
  

  (#738)

• affile: Add ability to refine the wildcard-file() filename-pattern() option with exclude-pattern(), to exclude some matching files. For example, match all *.log but exclude *.?.log.
  (#719)

• bigquery(), google-pubsub-grpc(): add service-account-key option to ADC auth mode

  Example usage:

  destination {
          google-pubsub-grpc(
              project("test")
              topic("test")
              auth(adc(service-account-key("absolute path to file")))
         );
  };
  

  Note: File path must be the absolute path.
  (#732)

Bugfixes

• gRPC based destinations: Gracefully stop if field name is not valid.
  (#739)

• clickhouse(): Fixed setting UINT8 protobuf type.
  (#739)

• disk-buffer() metrics: fix showing used buffers as both active and abandoned
  (#726)

FilterX features

• Add str_replace() function

  For example:

  filterx {
      dal = "érik a szőlő, hajlik a vessző";
      str_replace(dal, "a", "egy") == "érik egy szőlő, hegyjlik egy vessző";
  
      dal = "érik a szőlő, hajlik a vessző";
      str_replace(dal, "a", "egy", 1) == "érik egy szőlő, hajlik a vessző";
  };
  

  (#725)

• Support string slicing

  For example:

  filterx {
      str = "example";
      idx = 3;
      str[idx..5] == "mp";
      str[..idx] == "exa";
      str[idx..] == "mple";
  };
  

  (#720)

• Null and error-safe dict elements

  For example, the following fields won't be set:

  $MSG = {
      "nullidontwant":?? null,
      "erroridontwant":?? nonexistingvar,
  };
  

  (#736)

FilterX bugfixes

• format_xml(): Fixed an occasionally occurring crash

  In case the input was not a dict, a crash could occour during logging the error.
  (#730)

• parse_windows_eventlog_xml(): Fixed a Data misparsing

  "<Data Name="key" />" is now parsed correctly, identical to "<Data Name="key"></Data>"
  (#722)

• format_xml(), format_windows_eventlog_xml(): Fixed escaping in element values

  Example:

  <b> -> &lt;b&gt;
  

  (#743)

[1] syslog-ng is a trademark of One Identity

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, László Várady,
Ross Williams, Szilard Parrag, Tamás Kosztyu

axosyslog-charts-0.16.0

AxoSyslog for Kubernetes

axosyslog-4.14.0

4.14.0

AxoSyslog is binary-compatible with syslog-ng [1] and serves as a drop-in replacement.

We provide cloud-ready container images and Helm charts.

Packages are available in our APT and RPM repositories (Ubuntu, Debian, AlmaLinux, Fedora).

Check out the AxoSyslog documentation for all the details.

Features

• cisco-parser(): Added support for Cisco Nexus NXOS 9.3 syslog format
  (#713)

• loggen: Added --client-port option to set the outbound (client) port
  (#709)

FilterX features

• format_xml() and format_windows_eventlog_xml(): new functions added

  Example usage:

  $MSG = format_xml({"a":{"b":"foo"}});
  

  (#684)

FilterX bugfixes

• metrics_labels(): Fixed a bug where update_metrics() did not omit null values.
  (#711)

Other changes

• syslog-ng-ctl stats prometheus: show orphan metrics

  Stale counters will be shown in order not to lose information, for example,
  when messages are sent using short-lived connections and metrics are scraped in
  minute intervals.

  We recommend using syslog-ng-ctl stats --remove-orphans during each configuration reload,
  but only after the values of those metrics have been scraped by all scrapers.
  (#715)

[1] syslog-ng is a trademark of One Identity.

Discord

For a bit more interactive discussion, join our Discord server:

Credits

AxoSyslog is developed as a community project, and as such it relies
on volunteers, to do the work necessary to produce AxoSyslog.

Reporting bugs, testing changes, writing code or simply providing
feedback is an important contribution, so please if you are a user
of AxoSyslog, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Attila Szakacs, Balazs Scheidler, Hofi, László Várady,
Szilard Parrag, Tamás Kosztyu, shifter, Shiraz McClennon