Skip to content

Commit d3e2756

Browse files
authored
Merge branch 'master' into severity_fix
2 parents cff021c + 9df208f commit d3e2756

File tree

7 files changed

+105
-66
lines changed

7 files changed

+105
-66
lines changed

_data/external_links.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -608,6 +608,11 @@ gh-es:
608608
url: https://github.com/syslog-ng/syslog-ng/tree/master/scl/elasticsearch
609609
title: [ "Elasticsearch configuration snippet on GitHub" ]
610610

611+
gh-es-ds:
612+
id: gh-es-ds
613+
url: https://github.com/syslog-ng/syslog-ng/blob/master/scl/elasticsearch/elastic-datastream.conf
614+
title: [ "Elasticsearch datastream configuration snippet on GitHub" ]
615+
611616
gh-fortigate:
612617
id: gh-fortigate
613618
url: https://github.com/syslog-ng/syslog-ng/blob/master/scl/fortigate/fortigate.conf
@@ -796,6 +801,11 @@ ebpf:
796801
url: https://ebpf.io/
797802
title: [ "Extended Berkeley Packet Filter" ]
798803

804+
elastic-ds:
805+
id: elastic-ds
806+
url: https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html
807+
title: [ "Elasticsearch data streams" ]
808+
799809
gcp-ser:
800810
id: gcp-ser
801811
url: https://cloud.google.com/iam/docs/service-account-overview

doc/_admin-guide/060_Sources/220_unix-stream_unix-dgram/README.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,9 @@ short_title: unix-stream, unix-dgram
44
id: adm-src-unix
55
description: >-
66
The unix-stream() and unix-dgram() drivers open an AF_UNIX socket and
7-
start listening on it for messages. The unix-stream() driver is
8-
primarily used on Linux and uses SOCK_STREAM semantics (connection
9-
oriented, no messages are lost), while unix-dgram() is used on BSDs and
10-
uses SOCK_DGRAM semantics: this may result in lost local messages if
7+
start listening on it for messages. On Linux both the unix-stream() and unix-dgram() drivers are used and are always reliable. The unix-stream() driver uses SOCK_STREAM semantics (connection
8+
oriented, no messages are lost),
9+
while unix-dgram() is used on BSDs and uses SOCK_DGRAM semantics: this may result in lost local messages if
1110
the system is overloaded.
1211
---
1312

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
title: 'elasticsearch-datastream: Elasticsearch data streams'
3+
short_title: elasticsearch-datastream
4+
id: adm-dest-es-datastream
5+
description: >-
6+
From {{ site.product.short_name }} 4.8 and later versions, you can send messages and metrics to Elasticsearch data streams to store your log and metrics data as time series data.
7+
---
8+
9+
**Declaration**
10+
11+
```config
12+
destination d_elastic_data_stream {
13+
elasticsearch-datastream(
14+
url("https://elastic-endpoint:9200/my-data-stream/_bulk")
15+
user("elastic")
16+
password("ba253DOn434Tc0pY22OI")
17+
);
18+
};
19+
```
20+
21+
This driver is a reusable configuration snippet configured to send log messages using the http() driver using a template. You can find the Elasticsearch datastream configuration snippet on GitHub.
22+
23+
## Prerequisites
24+
25+
* An account for Elasticsearch datastreams with a username and a password.
26+
27+
## Options
28+
29+
Elasticsearch datastream is an HTTP based driver, hence it utilizes the HTTP destination options.
30+
31+
> *Copyright © 2024 Axoflow*

doc/_admin-guide/070_Destinations/225_Amazon-s3/000_amazon_s3_options.md

Lines changed: 25 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -7,22 +7,22 @@ description: >-
77

88
The following options are specific to the s3 destination.
99

10-
### access-key()
10+
## access-key()
1111

1212
|Type:| string|
1313
|Default:| N/A|
1414

1515
*Description:* The `ACCESS_KEY` of the service account of the S3 bucket. (Used together with secret-key().)
1616

17-
### bucket()
17+
## bucket()
1818

1919
|Type:| string|
2020
|Default:| N/A|
2121

2222
*Description:* The name of the S3 bucket.
2323
For example: `my-bucket`.
2424

25-
### canned-acl()
25+
## canned-acl()
2626

2727
|Type:| string|
2828
|Default:| N/A|
@@ -34,46 +34,44 @@ For example: `my-bucket`.
3434

3535
If an invalid value is configured, the default is used.
3636

37-
### chunk-size()
37+
## chunk-size()
3838

3939
|Type:| string|
40-
|Default:| N/A|
40+
|Default:| 5 MiB|
4141

42-
*Description:* The size of log messages written by syslog-n OSE to the S3 object in a batch. If compression is enabled, the chunk-size() specifies the compressed size.
42+
*Description:* The size of log messages written by {{ site.product.short_name }} to the S3 object in a batch. If compression is enabled, the chunk-size() specifies the compressed size. Must be set to at least 5 MiB.
4343

44-
### compression()
44+
## compression()
4545

4646
|Type:| boolean|
4747
|Default:| no|
4848

4949
*Description:* Setting compression to `yes` enables gzip compression, and implicitly adds a `.gz` suffix to the created object’s key. You can set the level of the compression using the compresslevel() option (`0-9`).
5050

51-
### compresslevel()
51+
## compresslevel()
5252

5353
|Type:| integer|
54-
|Default:| 0-9|
54+
|Default:| 9|
5555

5656
Description: Only has effect if compression() is set to `yes`. The level of the compression can be set using the compresslevel() option (`0-9`).
5757

58+
## flush-grace-period()
5859

59-
### flush-grace-period()
60-
61-
|Type:| integer[minutes]|
60+
|Type:| integer [minutes]|
6261
|Default:| 60|
6362

6463
*Description:* After the grace period expires and no new messages are routed to the destination, {{ site.product.short_name }} flushes the contents of the buffer to the S3 object even if the volume of the messages in the buffer is lower than chunk-size().
6564

66-
#{% include doc/admin-guide/options/log-fifo-size.md %}
65+
{% include doc/admin-guide/options/log-fifo-size.md %}
6766

68-
### max-object-size()
67+
## max-object-size()
6968

70-
|Type:| number [GiB]|
69+
|Type:| string|
7170
|Default:| 5120GiB|
7271

7372
*Description:* The maximal size of the S3 object. If an object reaches this size, {{ site.product.short_name }} appends an index suffix ("-1", “-2”, …) to the object key and starts a new object after rotation.
7473

75-
### max-pending-uploads()
76-
74+
## max-pending-uploads()
7775

7876
|Type:| integer|
7977
|Default:| 32|
@@ -83,38 +81,37 @@ Description: The max-pending-uploads() and upload-threads() options configure th
8381
* upload-threads() limits the maximum number of parallel uploads.
8482
* max-pending-uploads() limits the number of chunks that are waiting in the work queue of the upload threads to get uploaded
8583

86-
### object-key()
84+
## object-key()
8785

8886
|Type:| template|
8987
|Default:| N/A|
9088

9189
*Description:* The unique object key (or key name), which identifies the object in an Amazon S3 bucket.
9290

93-
### object-key-timestamp()
91+
## object-key-timestamp()
9492

9593
|Type:| template|
9694
|Default:| N/A|
9795

9896
*Description:* The object-key-timestamp() option can be used to set a datetime-related template, which is appended to the end of the object, for example: "`${R_MONTH_ABBREV}${R_DAY}`". When a log message arrives with a newer timestamp template resolution, the previous timestamped object gets finished and a new one is started with the new timestamp. If an older message arrives, it does not reopen the old object, but starts a new object with the key having an index appended to the old object.
9997

100-
#{% include doc/admin-guide/options/persist-name.md %}
98+
{% include doc/admin-guide/options/persist-name.md %}
10199

102-
### region()
100+
## region()
103101

104102
|Type:| string|
105103
|Default:| N/A|
106104

107-
108105
*Description:* The regional endpoint where the bucket is stored. For example, us-east-1.
109106

110-
### secret-key()
107+
## secret-key()
111108

112109
|Type:| string|
113110
|Default:| N/A|
114111

115112
*Description:* The `SECRET_KEY` of the service account used to access the S3 bucket. (Together with access-key().)
116113

117-
### storage-class()
114+
## storage-class()
118115

119116
|Type:| string|
120117
|Default:| STANDARD|
@@ -125,20 +122,20 @@ Description: The max-pending-uploads() and upload-threads() options configure th
125122

126123
If an invalid value is configured, the default is used.
127124

128-
### upload-threads()
125+
## upload-threads()
129126

130127
|Type:| integer|
131128
|Default:| 8|
132129

133130
*Description:* The number of {{ site.product.short_name }} worker threads that are used to upload data to S3 from this destination.
134131

135-
### template()
132+
## template()
136133

137134
*Description:* The message as written to the Amazon S3 object. You can use templates and template functions to format the message.
138135

139-
### url()
136+
## url()
140137

141138
|Type:| string|
142139
|Default:| N/A|
143140

144-
*Description:* The URL of the S3 bucket, for example, `https://my-bucket.s3.us-west-2.amazonaws.com`.
141+
*Description:* The API endpoint URL of the S3 bucket. When used with Amazon AWS, the {{ site.product.short_name }} S3 destination automatically creates the service URL. It is recommended that you omit this option. This option is required only if the {{ site.product.short_name }} S3 driver is used in conjunction with third-party S3 service providers, such as MinIO or Google Cloud.

doc/_admin-guide/100_TLS-encrypted_message_transfer/002_Mutual_authentication_using_TLS/000_Configuring_mutual_TLS_client.md

Lines changed: 17 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -17,28 +17,28 @@ syslog() driver):
1717
1. Create an X.509 certificate for the {{ site.product.short_name }} client.
1818

1919
2. Copy the certificate (for example, client_cert.pem) and the
20-
matching private key (for example, client.key) to the syslog-ng
20+
matching private key (for example, client.key) to the {{ site.product.short_name }}
2121
client host, for example, into the
2222
/opt/syslog-ng/etc/syslog-ng/cert.d directory. The certificate must
23-
be a valid X.509 certificate in PEM format. If you want to use a
24-
password-protected key, see
25-
Password-protected keys.
23+
be a valid X.509 certificate in PEM format. The key must be in PEM format.
24+
If you want to use a password-protected key, see Password-protected keys.
2625

27-
3. Copy the CA certificate of the Certificate Authority (for example,
28-
cacert.pem) that issued the certificate of the {{ site.product.short_name }} server (or
29-
the self-signed certificate of the {{ site.product.short_name }} server) to the
30-
{{ site.product.short_name }} client hosts, for example, into the
26+
3. Copy the CA certificate (for example, cacert.pem) of the Certificate
27+
Authority that issued the certificate of the {{ site.product.short_name }} server
28+
(or the self-signed certificate of the {{ site.product.short_name }} server) to the
29+
{{ site.product.short_name }} client host, for example, into the
3130
/opt/syslog-ng/etc/syslog-ng/ca.d directory.
3231

33-
Issue the following command on the certificate: **openssl x509
34-
-noout -hash -in cacert.pem** The result is a hash (for example,
32+
If you wish to use the ca-dir() option, instead of the ca-file(), in the
33+
{{ site.product.short_name }} configuration file (step 4.) then
34+
- issue the following command on the certificate:\
35+
`openssl x509 -noout -hash -in cacert.pem`\
36+
The result is a hash (for example,
3537
6d2962a8), a series of alphanumeric characters based on the
3638
Distinguished Name of the certificate.
37-
38-
Issue the following command to create a symbolic link to the
39+
- issue the following command to create a symbolic link to the
3940
certificate that uses the hash returned by the previous command and
40-
the **.0** suffix.
41-
41+
the **.0** suffix:\
4242
`ln -s cacert.pem 6d2962a8.0`
4343

4444
4. Add a destination statement to the {{ site.product.short_name }} configuration file that
@@ -49,6 +49,9 @@ syslog() driver):
4949
Include the client\'s certificate and private key in the tls()
5050
options.
5151

52+
For the details of the available tls() options, see
53+
TLS options.
54+
5255
Example: A destination statement using mutual authentication
5356

5457
The following destination encrypts the log messages using TLS and

doc/_admin-guide/100_TLS-encrypted_message_transfer/002_Mutual_authentication_using_TLS/001_Configuring_mutual_TLS_server.md

Lines changed: 18 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -11,38 +11,37 @@ Complete the following steps on the {{ site.product.short_name }} server:
1111

1212
## Steps
1313

14-
1. Copy the certificate (for example, syslog-ng.cert) of the syslog-ng
15-
server to the {{ site.product.short_name }} server host, for example, into the
14+
1. Create an X.509 certificate for the {{ site.product.short_name }} server.
15+
16+
2. Copy the certificate (for example, syslog-ng.cert) and the
17+
matching private key (for example, syslog-ng.key) to the {{ site.product.short_name }}
18+
server host, for example, into the
1619
/opt/syslog-ng/etc/syslog-ng/cert.d directory. The certificate must
17-
be a valid X.509 certificate in PEM format.
20+
be a valid X.509 certificate in PEM format. The key must be in PEM format.
21+
If you want to use a password-protected key, see Password-protected keys.
1822

19-
2. Copy the CA certificate (for example, cacert.pem) of the Certificate
23+
3. Copy the CA certificate (for example, cacert.pem) of the Certificate
2024
Authority that issued the certificate of the {{ site.product.short_name }} clients to
21-
the {{ site.product.short_name }} server, for example, into the
25+
the {{ site.product.short_name }} server host, for example, into the
2226
/opt/syslog-ng/etc/syslog-ng/ca.d directory.
2327

24-
Issue the following command on the certificate: **openssl x509
25-
-noout -hash -in cacert.pem** The result is a hash (for example,
28+
If you wish to use the ca-dir() option, instead of the ca-file(), in the
29+
{{ site.product.short_name }} configuration file (step 4.) then
30+
- issue the following command on the certificate:\
31+
`openssl x509 -noout -hash -in cacert.pem`\
32+
The result is a hash (for example,
2633
6d2962a8), a series of alphanumeric characters based on the
2734
Distinguished Name of the certificate.
28-
29-
Issue the following command to create a symbolic link to the
35+
- issue the following command to create a symbolic link to the
3036
certificate that uses the hash returned by the previous command and
31-
the **.0** suffix.
32-
37+
the **.0** suffix:\
3338
`ln -s cacert.pem 6d2962a8.0`
3439

35-
3. Copy the private key (for example, syslog-ng.key) matching the
36-
certificate of the {{ site.product.short_name }} server to the {{ site.product.short_name }} server host,
37-
for example, into the /opt/syslog-ng/etc/syslog-ng/key.d directory.
38-
The key must be in PEM format. If you want to use a
39-
password-protected key, see Password-protected keys.
40-
4140
4. Add a source statement to the {{ site.product.short_name }} configuration file that uses
4241
the tls( key-file(key_file_fullpathname)
4342
cert-file(cert_file_fullpathname) ) option and specify the key and
4443
certificate files. The source must use the source driver (network()
45-
or syslog()) matching the destination driver used by the syslog-ng
44+
or syslog()) matching the destination driver used by the {{ site.product.short_name }}
4645
client. Also specify the directory storing the certificate of the CA
4746
that issued the client's certificate.
4847

@@ -52,7 +51,7 @@ Complete the following steps on the {{ site.product.short_name }} server:
5251
Example: A source statement using TLS
5352

5453
The following source receives log messages encrypted using TLS,
55-
arriving to the 1999/TCP port of any interface of the syslog-ng
54+
arriving to the 1999/TCP port of any interface of the {{ site.product.short_name }}
5655
server.
5756

5857
```config

doc/_admin-guide/120_Parser/002_Parsing_key-value_pairs/000_kv_parser_options.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Options of key=value parsers
3-
parser: kv-parser
3+
parser: kv
44
prefix: kv
55
id: adm-parser-kv-opt
66
description: >-

0 commit comments

Comments
 (0)