Skip to content

Commit 2fc7093

Browse files
committed
docs: outputs: file: add log rotation support and configuration examples
Signed-off-by: SagiROosto <sagi.rosenthal@oosto.com>
1 parent 44f66da commit 2fc7093

1 file changed

Lines changed: 62 additions & 0 deletions

File tree

pipeline/outputs/file.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,11 @@ The _File_ output plugin lets you write the data received through the input plug
1111
| Key | Description | Default |
1212
| :--- | :--- | :--- |
1313
| `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. | _none_ |
14+
| `files_rotation` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `max_size` are rotated and optionally compressed. | `false` |
1415
| `format` | The [format](#format) of the file content. | _none_ |
16+
| `gzip` | Compress rotated files using gzip. Only applies when `files_rotation` is enabled. | `true` |
17+
| `max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `files_rotation` is enabled. | `7` |
18+
| `max_size` | Maximum size of the active output file before rotation is triggered. Supports size suffixes: `k` (kilobytes), `m` (megabytes), `g` (gigabytes). Only applies when `files_rotation` is enabled. | `100m` |
1519
| `mkdir` | Recursively create output directory if it doesn't exist. Permissions set to `0755`. | `false` |
1620
| `path` | Directory path to store files. If not set, Fluent Bit will write the files in its own working directory. | _none_ |
1721
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `1` |
@@ -112,6 +116,64 @@ You will get the following output:
112116
1564462620.000254 used=1045448 free=31760160 total=32805608
113117
```
114118

119+
## Log rotation
120+
121+
The File output plugin supports size-based log rotation.
122+
123+
When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the plugin rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The YYYMMDD_HHMMSS is timestamp of the time when the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second.
124+
125+
If `gzip` is enabled (the default), rotated files are compressed with gzip and stored with an additional `.gz` extension (for example, `cpu.log.20260512_134500_a1b2c3d4.gz`).
126+
127+
The plugin retains up to `max_files` rotated files per output file. When the limit is reached, the oldest rotated files are deleted automatically.
128+
129+
Log rotation works with all supported output [formats](#format), including `out_file`, plain, CSV, LTSV, template, and msgpack. File operations are thread-safe, so rotation can be used alongside multiple [workers](../../administration/multithreading.md#outputs).
130+
131+
### Log rotation example
132+
133+
The following configuration writes CPU metrics to file with rotation enabled. Files are rotated at 50 MB and the five most recent rotated files are retained with gzip compression:
134+
135+
{% tabs %}
136+
{% tab title="fluent-bit.yaml" %}
137+
138+
```yaml
139+
pipeline:
140+
inputs:
141+
- name: cpu
142+
tag: cpu
143+
144+
outputs:
145+
- name: file
146+
match: '*'
147+
path: /var/log/fluent-bit
148+
file: cpu.log
149+
files_rotation: true
150+
max_size: 50m
151+
max_files: 5
152+
gzip: true
153+
```
154+
155+
{% endtab %}
156+
{% tab title="fluent-bit.conf" %}
157+
158+
```text
159+
[INPUT]
160+
Name cpu
161+
Tag cpu
162+
163+
[OUTPUT]
164+
Name file
165+
Match *
166+
Path /var/log/fluent-bit
167+
File cpu.log
168+
Files_Rotation true
169+
Max_Size 50m
170+
Max_Files 5
171+
Gzip true
172+
```
173+
174+
{% endtab %}
175+
{% endtabs %}
176+
115177
## Get started
116178

117179
You can run the plugin from the command line or through the configuration file.

0 commit comments

Comments
 (0)