1- 4.25 .0
1+ 4.26 .0
22======
33
44AxoSyslog is binary-compatible with syslog-ng [ 1] and serves as a drop-in replacement.
@@ -9,128 +9,172 @@ Packages are available in our [APT](https://github.com/axoflow/axosyslog/#deb-pa
99
1010Check out the [ AxoSyslog documentation] ( https://axoflow.com/docs/axosyslog-core/ ) for all the details.
1111
12+ ## Highlights
1213
13- ## Features
14+ < Fill this block manually from the blocks below >
1415
15- * ` syslog() ` / ` network() ` : HAProxy protocol v2 support over UDP
16+ ## Features
1617
17- When ` transport(proxied-udp) ` is configured, the original client address and port are available as ` ${SOURCEIP} ` ,
18- ` ${SOURCEPORT} ` , ` ${DESTIP} ` , and ` ${DESTPORT} ` .
19- ([ #987 ] ( https://github.com/axoflow/axosyslog/pull/987 ) )
18+ * ` syslog() ` , ` network() ` sources: add ` syslogng_input_transport_errors_total ` metric
2019
21- * FilterX: Added new function: ` uuid() ` that generates a random UUID v4 string .
22- ([ #1018 ] ( https://github.com/axoflow/axosyslog/pull/1018 ) )
20+ It shows ` invalid-frame-header ` or ` tls-handshake ` related transport errors .
21+ ([ #1026 ] ( https://github.com/axoflow/axosyslog/pull/1026 ) )
2322
24- * FilterX: Added various crypto hash digest functions.
23+ * Add ` trusted-fingerprints() ` option: this new option will allow you to trust
24+ X.509 certificates based on their fingerprints. The new option deprecates
25+ ` trusted-keys() ` , the major difference being that ` trusted-fingerprints() `
26+ will accept X.509 certificates as valid, even if the normal X.509 validation
27+ fails, whereas ` trusted-keys() ` needed both the X.509 verification pass and
28+ the fingerprint checking to succeed. This feature also adds the capability
29+ to use a fingerprinting method other than SHA1, which is not considered safe
30+ anymore.
2531
26- These functions compute the hash of a string or bytes and return the result as a hex string:
27- * ` md5() `
28- * ` sha1() `
29- * ` sha256() `
30- * ` sha512() `
32+ Here's an example syntax:
3133
32- The generic ` digest(input, alg="sha256") ` function accepts an optional algorithm
33- name and returns the raw hash as a bytes object.
34- ([ #1019 ] ( https://github.com/axoflow/axosyslog/pull/1019 ) )
34+ tls(
35+ ...
36+ trusted-fingerprints("SHA1:0C:EF:34:4D:0B:74:AE:03:72:9A:4E:68:AF:90:59:A9:EF:35:1F:AA",
37+ "SHA512:15:B3:C5:96:48:5B:F6:20:C3:86:47:78:99:E1:2B:F2:C4:A6:93:AE:E8:0A:B3:F7:78:39:66:B4:EF:4F:A5:47:2A:E0:4A:93:06:46:72:C0:15:6A:FC:59:10:25:37:60:E3:84:E9:EC:90:30:12:F5:27:EA:22:1F:55:9B:3B:97")
38+ )
3539
36- * FilterX: Added ` utf8_validate() ` and ` utf8_sanitize() ` string functions.
40+ NOTE: the fingerprinting method is the word before the first colon. The
41+ naming of the fingerprinting methods should match OpenSSL's supported digest
42+ algorithms.
3743
38- * ` utf8_validate() ` checks whether the string contains valid UTF-8 sequences and returns a boolean
39- * ` utf8_sanitize() ` replaces invalid byte sequences with their ` \xNN ` escaped representation
40- ([ #1019 ] ( https://github.com/axoflow/axosyslog/pull/1019 ) )
44+ As of OpenSSL 3.0.10, the following digest algorithms are supported:
4145
42- * FilterX: Added various encoding/decoding functions.
46+ Message Digest commands (see the `dgst' command for more details)
47+ blake2b512 blake2s256 md4 md5
48+ rmd160 sha1 sha224 sha256
49+ sha3-224 sha3-256 sha3-384 sha3-512
50+ sha384 sha512 sha512-224 sha512-256
51+ shake128 shake256 sm3
4352
44- * ` base64_encode() ` /` base64_decode() ` (bytes <-> string)
45- * ` urlencode() ` /` urldecode() ` (string <-> string)
46- * ` hex_encode() ` /` hex_decode() ` (bytes <-> string)
47- ([ #1019 ] ( https://github.com/axoflow/axosyslog/pull/1019 ) )
53+ To find out the fingerprint for a certificate, you can use this command:
4854
49- * New FilterX types ` subnet() ` and ` ip() ` : these new types encapsulate an
50- IPv4/IPv6 subnet (in CIDR notation) or a single IP address. Both types takes
51- their string representation and will return an ERROR if the format cannot be
52- parsed.
55+ $ openssl x509 -sha512 -in <certificate file in pem > -fingerprint
56+ ([ #137 ] ( https://github.com/axoflow/axosyslog/pull/137 ) )
5357
54- Example configuration:
58+ * ` afuser ` : add escaping() option to usertty() output
5559
56- a = subnet("192.168.0.0/24");
60+ The usertty() destination now supports an escaping() option, using the same
61+ template escaping behavior as templates.
62+ ([ #1117 ] ( https://github.com/axoflow/axosyslog/pull/1117 ) )
5763
58- "192.168.0.5" in a;
59- "192.168.1.5" in a or true;
60- ip("192.168.0.11") in a;
64+ * ` switch ` : add ranged case support for FilterX ` switch `
6165
62- a6 = subnet("DEAD:BEEF::1/64");
63-
64- "DEAD:BEEF::2" in a6;
65- "DEAD:BABE::1" in a6 or true;
66- ip("DEAD:BEEF::00ac") in a6;
67- ([ #1021 ] ( https://github.com/axoflow/axosyslog/pull/1021 ) )
66+ Example usage:
67+ ```
68+ selector = 5;
69+ switch (selector) {
70+ case 1..4:
71+ result = "below";
72+ break;
73+ case 5:
74+ result = "exact";
75+ break;
76+ default:
77+ result = "above";
78+ break;
79+ };
80+ ```
81+ ([#1093](https://github.com/axoflow/axosyslog/pull/1093))
6882
69- * FilterX ` glob_match() ` function: this function will match a filename against
70- a single-, or a list of glob-style patterns.
83+ * `parallelize()`: add `syslogng_parallelized_batch_size` and
84+ `syslogng_parallelized_input_batch_size` histograms to the prometheus style
85+ stats output, at stats(level(4)).
86+ ([#1077](https://github.com/axoflow/axosyslog/pull/1077))
7187
72- Example:
88+ * `arrow-flight()`: Added a new destination for [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html)
7389
74- glob_match(filename, "*.zip");
75- glob_match(filename, ["*.zip", "*.7z"]);
76- ([ #1039 ] ( https://github.com/axoflow/axosyslog/pull/1039 ) )
90+ Options:
91+ * `url()`: Flight endpoint
92+ * `path()`: descriptor path, templatable so a single destination can route to many tables
93+ * `schema()`: one `"name" TYPE => template` entry per column
7794
78- * FilterX ` cache_json_file ` : add deafult_value parameter to FilterX function
95+ Column types supported in `schema()`:
96+ * `STRING`
97+ * `INT64` (alias `INTEGER`)
98+ * `DOUBLE` (alias `FLOAT64`)
99+ * `BOOL` (alias `BOOLEAN`)
100+ * `TIMESTAMP`
101+ * `MAP(STRING, STRING)`
79102
80- If the file is not present, or an error occurs when reading it, the default_value will be used, if provided.
103+ Example configuration:
81104
82- Example:
83105 ```
84- cache_json_file("./test.json", default_value={"key": "value"});
106+ destination d_arrow {
107+ arrow-flight(
108+ url("grpc://flight.example.com:8815")
109+ path("events.${HOST}")
110+ schema(
111+ "ts" TIMESTAMP => "$UNIXTIME"
112+ "host" STRING => "$HOST"
113+ "program" STRING => "$PROGRAM"
114+ "severity" INT64 => "$LEVEL_NUM"
115+ "msg" STRING => "$MSG"
116+ )
117+ batch-lines(1000)
118+ batch-bytes(1048576)
119+ batch-timeout(5000)
120+ workers(4)
121+ worker-partition-key("${HOST}")
122+ );
123+ };
85124 ```
86- ([#1034 ](https://github.com/axoflow/axosyslog/pull/1034 ))
125+ ([#1069 ](https://github.com/axoflow/axosyslog/pull/1069 ))
87126
88127
89128## Bugfixes
90129
91- * `disk-buffer()`: keep the queue alive during reload
92-
93- Keeping the disk-buffer alive on reload fixes a bug, where a full disk-buffer can
94- grow infinitely by reloading. It can also cause significant reload speedup.
95- ([#1030](https://github.com/axoflow/axosyslog/pull/1030))
96-
97- * `disk-buffer()`: fix message ordering issue when a message batch failed to be delivered
98- ([#1005](https://github.com/axoflow/axosyslog/pull/1005))
130+ * `logmsg`: fix crash when iterating name-value registry concurrently
99131
100- * CR (`\r`) characters are now removed from line endings, empty UDP datagrams are dropped
101- ([#1000](https://github.com/axoflow/axosyslog/pull/1000))
132+ The hash table mapping value names to handles was iterated without locking, which could crash AxoSyslog
133+ when another thread registered new name-value pairs at the same time. This happened for example when the
134+ Python `LogMessage.keys()` method ran while a `kv-parser()` was processing messages on a parallel path.
135+ ([#1122](https://github.com/axoflow/axosyslog/pull/1122))
102136
103- * FilterX `parse_xml() `: fix crash in case of invalid XML
104- ([#1041 ](https://github.com/axoflow/axosyslog/pull/1041 ))
137+ * `logproto `: fix crash with `transport(auto)` sources across a config reload
138+ ([#1124 ](https://github.com/axoflow/axosyslog/pull/1124 ))
105139
106- * FilterX : fix crash when using the `+` operator on dicts
107- ([#1040 ](https://github.com/axoflow/axosyslog/pull/1040 ))
140+ * `afsql` : fix segfault after database error
141+ ([#1114 ](https://github.com/axoflow/axosyslog/pull/1114 ))
108142
109- * FilterX: fix error handling of the `=??` operator
110- ([#1053](https://github.com/axoflow/axosyslog/pull/1053))
143+ * `java/hdfs`: fix unreleased lock in `send()` when file open fails
111144
145+ If `getHdfsFile()` returned `null`, the lock acquired at the start of
146+ `send()` was never released, causing a permanent deadlock on all
147+ subsequent calls.
148+ ([#1108](https://github.com/axoflow/axosyslog/pull/1108))
112149
113- ## Other changes
150+ * `correlation`: fix radix parser end-of-input handling
114151
115- * `opentelemetry()`, `axosyslog-otlp`, `loki()`, `google-pubsub`, `clickhouse`, `bigquery` : improve performance by using gRPC arenas for allocation
116- ([#1015](https://github.com/axoflow/axosyslog/pull/1015))
152+ Fixes two related radix matcher edge cases at end of input.
117153
118- * New metrics: `syslogng_input_transport_errors_total` for syslog framing and TLS errors
119- ([#1026](https://github.com/axoflow/axosyslog/pull/1026))
154+ Parser scans now stop before '\0' to avoid reading past end-of-input and to keep captured lengths correct.
155+ Parser-node traversal now continues with empty remaining input, so OPTIONALSET children can still match.
156+ ([#1110](https://github.com/axoflow/axosyslog/pull/1110))
120157
121- * `http()`: error reporting improvements of batched sending
122- ([#1001](https://github.com/axoflow/axosyslog/pull/1001))
123158
124- * Network sources: `transport(auto)` detections became more robust
125- ([#1013](https://github.com/axoflow/axosyslog/pull/1013))
159+ ## Other changes
126160
127- * `syslog-ng --interactive`: new debugger commands - step, continue, follow, trace
128- ([#340](https://github.com/axoflow/axosyslog/pull/340))
161+ * `parallelize()`: improve throughput by avoiding batches that are too large.
162+ Also set the default batch-size() parameter to 100, both in case of
163+ partition based and round robin batching.
164+ ([#1078](https://github.com/axoflow/axosyslog/pull/1078))
129165
130- * FilterX performance optimizations
166+ * Ubuntu 26.04 and Fedora 44 packages
167+ ([#1068](https://github.com/axoflow/axosyslog/pull/1068))
131168
132- * Contribution Guide: added section on how to contribute AI-assisted code
133- ([#1043](https://github.com/axoflow/axosyslog/pull/1043))
169+ * Decouple `repr()` and `string()`: previously the repr() of an object was
170+ typically the same as str, did not include type-related hints in its output.
171+ This makes repr() usage less useful, especially as we are adding more types
172+ to filterx. Starting with this version, repr() includes a format similar to
173+ Python's repr. This is an incompatible change for uses where repr() was
174+ directly used in an output, but normally its intended has always been the
175+ debug log of filterx. If your use-case relies on the current repr() format,
176+ you should explicitly cast your object to `string()` instead.
177+ ([#1033](https://github.com/axoflow/axosyslog/pull/1033))
134178
135179
136180
@@ -153,5 +197,6 @@ of AxoSyslog, contribute.
153197
154198We would like to thank the following people for their contribution:
155199
156- Andras Mitzki, Attila Szakacs, Balazs Scheidler, László Várady,
157- Szilard Parrag, Tamás Kosztyu
200+ Andras Mitzki, Attila Szakacs-Bertok, Balazs Scheidler,
201+ Balint Ferencz, Bence Csati, Hofi, László Várady, Szilard Parrag,
202+ Tamás Kosztyu, engzaz
0 commit comments