Merge branch 'master' into severity_fix

_data/external_links.yml

@@ -608,6 +608,11 @@ gh-es:
   url: https://github.com/syslog-ng/syslog-ng/tree/master/scl/elasticsearch
   title: [ "Elasticsearch configuration snippet on GitHub" ]
 
+gh-es-ds:
+  id: gh-es-ds
+  url: https://github.com/syslog-ng/syslog-ng/blob/master/scl/elasticsearch/elastic-datastream.conf
+  title: [ "Elasticsearch datastream configuration snippet on GitHub" ]
+
 gh-fortigate:
   id: gh-fortigate
   url: https://github.com/syslog-ng/syslog-ng/blob/master/scl/fortigate/fortigate.conf
@@ -796,6 +801,11 @@ ebpf:
   url: https://ebpf.io/
   title: [ "Extended Berkeley Packet Filter" ]
 
+elastic-ds:
+  id: elastic-ds
+  url: https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html
+  title: [ "Elasticsearch data streams" ]
+
 gcp-ser:
   id: gcp-ser
   url: https://cloud.google.com/iam/docs/service-account-overview

doc/_admin-guide/060_Sources/220_unix-stream_unix-dgram/README.md

@@ -4,10 +4,9 @@ short_title: unix-stream, unix-dgram
 id: adm-src-unix
 description: >-
     The unix-stream() and unix-dgram() drivers open an AF_UNIX socket and
-    start listening on it for messages. The unix-stream() driver is
-    primarily used on Linux and uses SOCK_STREAM semantics (connection
-    oriented, no messages are lost), while unix-dgram() is used on BSDs and
-    uses SOCK_DGRAM semantics: this may result in lost local messages if
+    start listening on it for messages. On Linux both the unix-stream() and unix-dgram() drivers are used and are always reliable. The unix-stream() driver uses SOCK_STREAM semantics (connection
+    oriented, no messages are lost),
+    while unix-dgram() is used on BSDs and uses SOCK_DGRAM semantics: this may result in lost local messages if
     the system is overloaded.
 ---
 

doc/_admin-guide/070_Destinations/035_elasticsearch-datastream/README.md

@@ -0,0 +1,31 @@
+---
+title: 'elasticsearch-datastream: Elasticsearch data streams'
+short_title: elasticsearch-datastream
+id: adm-dest-es-datastream
+description: >-
+    From {{ site.product.short_name }} 4.8 and later versions, you  can send messages and metrics to Elasticsearch data streams to store your log and metrics data as time series data.
+---
+
+**Declaration**
+
+```config
+destination d_elastic_data_stream {
+  elasticsearch-datastream(
+    url("https://elastic-endpoint:9200/my-data-stream/_bulk")
+    user("elastic")
+    password("ba253DOn434Tc0pY22OI")
+  );
+};
+```
+
+This driver is a reusable configuration snippet configured to send log messages using the http() driver using a template. You can find the Elasticsearch datastream configuration snippet on GitHub.
+
+## Prerequisites
+
+* An account for Elasticsearch datastreams with a username and a password.
+
+## Options
+
+Elasticsearch datastream is an HTTP based driver, hence it utilizes the HTTP destination options.
+
+> *Copyright © 2024 Axoflow*

doc/_admin-guide/070_Destinations/225_Amazon-s3/000_amazon_s3_options.md

@@ -7,22 +7,22 @@ description: >-
 
 The following options are specific to the s3 destination.
 
-### access-key()
+## access-key()
 
 |Type:|   string|
 |Default:|           N/A|
 
 *Description:* The `ACCESS_KEY` of the service account of the S3 bucket. (Used together with secret-key().)
 
-### bucket()
+## bucket()
 
 |Type:|   string|
 |Default:|           N/A|
 
 *Description:* The name of the S3 bucket.
 For example: `my-bucket`.
 
-### canned-acl()
+## canned-acl()
 
 |Type:|   string|
 |Default:|           N/A|
@@ -34,46 +34,44 @@ For example: `my-bucket`.
 
 If an invalid value is configured, the default is used.
 
-### chunk-size()
+## chunk-size()
 
 |Type:|   string|
-|Default:|           N/A|
+|Default:|           5 MiB|
 
-*Description:* The size of log messages written by syslog-n OSE to the S3 object in a batch. If compression is enabled, the chunk-size() specifies the compressed size.
+*Description:* The size of log messages written by {{ site.product.short_name }} to the S3 object in a batch. If compression is enabled, the chunk-size() specifies the compressed size. Must be set to at least 5 MiB.
 
-### compression()
+## compression()
 
 |Type:|   boolean|
 |Default:|           no|
 
 *Description:* Setting compression to `yes` enables gzip compression, and implicitly adds a `.gz` suffix to the created object’s key. You can set the level of the compression using the compresslevel() option (`0-9`).
 
-### compresslevel() 
+## compresslevel()
 
 |Type:|   integer|
-|Default:|           0-9|
+|Default:|           9|
 
 Description: Only has effect if compression() is set to `yes`. The level of the compression can be set using the compresslevel() option (`0-9`).
 
+## flush-grace-period()
 
-### flush-grace-period()
-
-|Type:|   integer[minutes]|
+|Type:|   integer [minutes]|
 |Default:|           60|
 
 *Description:* After the grace period expires and no new messages are routed to the destination, {{ site.product.short_name }} flushes the contents of the buffer to the S3 object even if the volume of the messages in the buffer is lower than chunk-size().
 
-#{% include doc/admin-guide/options/log-fifo-size.md %}
+{% include doc/admin-guide/options/log-fifo-size.md %}
 
-### max-object-size()
+## max-object-size()
 
-|Type:|   number [GiB]|
+|Type:|   string|
 |Default:|           5120GiB|
 
 *Description:* The maximal size of the S3 object. If an object reaches this size, {{ site.product.short_name }} appends an index suffix ("-1", “-2”, …) to the object key and starts a new object after rotation.
 
-### max-pending-uploads()
-
+## max-pending-uploads()
 
 |Type:|   integer|
 |Default:|           32|
@@ -83,38 +81,37 @@ Description: The max-pending-uploads() and upload-threads() options configure th
 * upload-threads() limits the maximum number of parallel uploads.
 * max-pending-uploads() limits the number of chunks that are waiting in the work queue of the upload threads to get uploaded
 
-### object-key()
+## object-key()
 
 |Type:|   template|
 |Default:|           N/A|
 
 *Description:* The unique object key (or key name), which identifies the object in an Amazon S3 bucket.
 
-### object-key-timestamp()
+## object-key-timestamp()
 
 |Type:|   template|
 |Default:|           N/A|
 
 *Description:* The object-key-timestamp() option can be used to set a datetime-related template, which is appended to the end of the object, for example: "`${R_MONTH_ABBREV}${R_DAY}`". When a log message arrives with a newer timestamp template resolution, the previous timestamped object gets finished and a new one is started with the new timestamp. If an older message arrives, it does not reopen the old object, but starts a new object with the key having an index appended to the old object.
 
-#{% include doc/admin-guide/options/persist-name.md %}
+{% include doc/admin-guide/options/persist-name.md %}
 
-### region()
+## region()
 
 |Type:|   string|
 |Default:|           N/A|
 
-
 *Description:* The regional endpoint where the bucket is stored. For example, us-east-1.
 
-### secret-key()
+## secret-key()
 
 |Type:|   string|
 |Default:|           N/A|
 
 *Description:* The `SECRET_KEY` of the service account used to access the S3 bucket. (Together with access-key().)
 
-### storage-class()
+## storage-class()
 
 |Type:|   string|
 |Default:|           STANDARD|
@@ -125,20 +122,20 @@ Description: The max-pending-uploads() and upload-threads() options configure th
 
 If an invalid value is configured, the default is used.
 
-### upload-threads()
+## upload-threads()
 
 |Type:|   integer|
 |Default:|           8|
 
 *Description:* The number of {{ site.product.short_name }} worker threads that are used to upload data to S3 from this destination.
 
-### template()
+## template()
 
 *Description:* The message as written to the Amazon S3 object. You can use templates and template functions to format the message.
 
-### url()
+## url()
 
 |Type:|   string|
 |Default:|           N/A|
 
-*Description:* The URL of the S3 bucket, for example, `https://my-bucket.s3.us-west-2.amazonaws.com`.
+*Description:* The API endpoint URL of the S3 bucket. When used with Amazon AWS, the {{ site.product.short_name }} S3 destination automatically creates the service URL. It is recommended that you omit this option. This option is required only if the {{ site.product.short_name }} S3 driver is used in conjunction with third-party S3 service providers, such as MinIO or Google Cloud.

doc/_admin-guide/100_TLS-encrypted_message_transfer/002_Mutual_authentication_using_TLS/000_Configuring_mutual_TLS_client.md

@@ -17,28 +17,28 @@ syslog() driver):
 1. Create an X.509 certificate for the {{ site.product.short_name }} client.
 
 2. Copy the certificate (for example, client_cert.pem) and the
-    matching private key (for example, client.key) to the syslog-ng
+    matching private key (for example, client.key) to the {{ site.product.short_name }}
     client host, for example, into the
     /opt/syslog-ng/etc/syslog-ng/cert.d directory. The certificate must
-    be a valid X.509 certificate in PEM format. If you want to use a
-    password-protected key, see
-    Password-protected keys.
+    be a valid X.509 certificate in PEM format. The key must be in PEM format.
+    If you want to use a password-protected key, see Password-protected keys.
 
-3. Copy the CA certificate of the Certificate Authority (for example,
-    cacert.pem) that issued the certificate of the {{ site.product.short_name }} server (or
-    the self-signed certificate of the {{ site.product.short_name }} server) to the
-    {{ site.product.short_name }} client hosts, for example, into the
+3. Copy the CA certificate (for example, cacert.pem) of the Certificate
+    Authority that issued the certificate of the {{ site.product.short_name }} server
+    (or the self-signed certificate of the {{ site.product.short_name }} server) to the
+    {{ site.product.short_name }} client host, for example, into the
     /opt/syslog-ng/etc/syslog-ng/ca.d directory.
 
-    Issue the following command on the certificate: **openssl x509
-    -noout -hash -in cacert.pem** The result is a hash (for example,
+    If you wish to use the ca-dir() option, instead of the ca-file(), in the
+    {{ site.product.short_name }} configuration file (step 4.) then
+    - issue the following command on the certificate:\
+    `openssl x509 -noout -hash -in cacert.pem`\
+    The result is a hash (for example,
     6d2962a8), a series of alphanumeric characters based on the
     Distinguished Name of the certificate.
-
-    Issue the following command to create a symbolic link to the
+    - issue the following command to create a symbolic link to the
     certificate that uses the hash returned by the previous command and
-    the **.0** suffix.
-
+    the **.0** suffix:\
     `ln -s cacert.pem 6d2962a8.0`
 
 4. Add a destination statement to the {{ site.product.short_name }} configuration file that
@@ -49,6 +49,9 @@ syslog() driver):
     Include the client\'s certificate and private key in the tls()
     options.
 
+    For the details of the available tls() options, see
+    TLS options.
+
     Example: A destination statement using mutual authentication
 
     The following destination encrypts the log messages using TLS and

doc/_admin-guide/100_TLS-encrypted_message_transfer/002_Mutual_authentication_using_TLS/001_Configuring_mutual_TLS_server.md

@@ -11,38 +11,37 @@ Complete the following steps on the {{ site.product.short_name }} server:
 
 ## Steps
 
-1. Copy the certificate (for example, syslog-ng.cert) of the syslog-ng
-    server to the {{ site.product.short_name }} server host, for example, into the
+1. Create an X.509 certificate for the {{ site.product.short_name }} server.
+
+2. Copy the certificate (for example, syslog-ng.cert) and the
+    matching private key (for example, syslog-ng.key) to the {{ site.product.short_name }}
+    server host, for example, into the
     /opt/syslog-ng/etc/syslog-ng/cert.d directory. The certificate must
-    be a valid X.509 certificate in PEM format.
+    be a valid X.509 certificate in PEM format. The key must be in PEM format.
+    If you want to use a password-protected key, see Password-protected keys.
 
-2. Copy the CA certificate (for example, cacert.pem) of the Certificate
+3. Copy the CA certificate (for example, cacert.pem) of the Certificate
     Authority that issued the certificate of the {{ site.product.short_name }} clients to
-    the {{ site.product.short_name }} server, for example, into the
+    the {{ site.product.short_name }} server host, for example, into the
     /opt/syslog-ng/etc/syslog-ng/ca.d directory.
 
-    Issue the following command on the certificate: **openssl x509
-    -noout -hash -in cacert.pem** The result is a hash (for example,
+    If you wish to use the ca-dir() option, instead of the ca-file(), in the
+    {{ site.product.short_name }} configuration file (step 4.) then
+    - issue the following command on the certificate:\
+    `openssl x509 -noout -hash -in cacert.pem`\
+    The result is a hash (for example,
     6d2962a8), a series of alphanumeric characters based on the
     Distinguished Name of the certificate.
-
-    Issue the following command to create a symbolic link to the
+    - issue the following command to create a symbolic link to the
     certificate that uses the hash returned by the previous command and
-    the **.0** suffix.
-
+    the **.0** suffix:\
     `ln -s cacert.pem 6d2962a8.0`
 
-3. Copy the private key (for example, syslog-ng.key) matching the
-    certificate of the {{ site.product.short_name }} server to the {{ site.product.short_name }} server host,
-    for example, into the /opt/syslog-ng/etc/syslog-ng/key.d directory.
-    The key must be in PEM format. If you want to use a
-    password-protected key, see Password-protected keys.
-
 4. Add a source statement to the {{ site.product.short_name }} configuration file that uses
     the tls( key-file(key_file_fullpathname)
     cert-file(cert_file_fullpathname) ) option and specify the key and
     certificate files. The source must use the source driver (network()
-    or syslog()) matching the destination driver used by the syslog-ng
+    or syslog()) matching the destination driver used by the {{ site.product.short_name }}
     client. Also specify the directory storing the certificate of the CA
     that issued the client's certificate.
 
@@ -52,7 +51,7 @@ Complete the following steps on the {{ site.product.short_name }} server:
     Example: A source statement using TLS
 
     The following source receives log messages encrypted using TLS,
-    arriving to the 1999/TCP port of any interface of the syslog-ng
+    arriving to the 1999/TCP port of any interface of the {{ site.product.short_name }}
     server.
 
     ```config

doc/_admin-guide/120_Parser/002_Parsing_key-value_pairs/000_kv_parser_options.md

@@ -1,6 +1,6 @@
 ---
 title: Options of key=value parsers
-parser: kv-parser
+parser: kv
 prefix: kv
 id: adm-parser-kv-opt
 description: >-