You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/filebeat/faq-deleted-files-are-not-freed.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,5 +10,5 @@ applies_to:
10
10
11
11
In the default behaviour, Filebeat opens the files and keeps them open until it reaches the end of them. In situations when the configured output is blocked (e.g. {{es}} or {{ls}} is unavailable) for a long time, this can cause Filebeat to keep file handlers to files that were deleted from the file system in the mean time. As long as Filebeat keeps the deleted files open, the operating system doesn’t free up the space on disk, which can lead to increase disk utilisation or even out of disk situations.
12
12
13
-
To mitigate this issue, you can set the [`close_timeout`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-close-timeout) setting to `5m`. This will ensure every file handler is closed once every 5 minutes, regardless of whether it reached EOF or not. Note that this option can lead to data loss if the file is deleted before Filebeat reaches the end of the file.
13
+
To mitigate this issue, you can set the [`close.reader.after_interval`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-close-timeout) setting to `5m`. This will ensure every file handler is closed once every 5 minutes, regardless of whether it reached EOF or not. Note that this option can lead to data loss if the file is deleted before Filebeat reaches the end of the file.
Copy file name to clipboardExpand all lines: docs/reference/filebeat/filebeat-cpu.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,5 +8,5 @@ applies_to:
8
8
9
9
# Filebeat is using too much CPU [filebeat-cpu]
10
10
11
-
Filebeat might be configured to scan for files too frequently. Check the setting for `scan_frequency` in the `filebeat.yml` config file. Setting `scan_frequency` to less than 1s may cause Filebeat to scan the disk in a tight loop.
11
+
Filebeat might be configured to scan for files too frequently. Check the setting for [`prospector.scanner.check_interval`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-scan-frequency)) in the `filebeat.yml` config file. Setting `prospector.scanner.check_interval` to less than 1s may cause Filebeat to scan the disk in a tight loop.
Copy file name to clipboardExpand all lines: docs/reference/filebeat/filebeat-not-collecting-lines.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ applies_to:
11
11
Filebeat might be incorrectly configured or unable to send events to the output. To resolve the issue:
12
12
13
13
* If using modules, make sure the `var.paths` setting points to the file. If configuring an input manually, make sure the `paths` setting is correct.
14
-
* Verify that the file is not older than the value specified by [`ignore_older`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-ignore-older). `ignore_older` is disable by default so this depends on the value you have set. You can change this behavior by specifying a different value for [`ignore_older`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-ignore-older).
14
+
* Verify that the file is not older than the value specified by [`ignore_older`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-ignore-older). `ignore_older` is disabled by default so this depends on the value you have set. You can change this behavior by specifying a different value for [`ignore_older`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-ignore-older).
15
15
* Make sure that Filebeat is able to send events to the configured output. Run Filebeat in debug mode to determine whether it’s publishing events successfully:
Copy file name to clipboardExpand all lines: docs/reference/filebeat/open-file-handlers.md
+7-8Lines changed: 7 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,17 +8,16 @@ applies_to:
8
8
9
9
# Too many open file handlers [open-file-handlers]
10
10
11
-
Filebeat keeps the file handler open in case it reaches the end of a file so that it can read new log lines in near real time. If Filebeat is harvesting a large number of files, the number of open files can become an issue. In most environments, the number of files that are actively updated is low. The `close_inactive` configuration option should be set accordingly to close files that are no longer active.
11
+
Filebeat keeps the file handler open in case it reaches the end of a file so that it can read new log lines in near real time. If Filebeat is harvesting a large number of files, the number of open files can become an issue. In most environments, the number of files that are actively updated is low. The [`close.on_state_change.inactive`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-close-inactive) configuration option should be set accordingly to close files that are no longer active.
12
12
13
13
There are additional configuration options that you can use to close file handlers, but all of them should be used carefully because they can have side effects. The options are:
The `close_renamed` and `close_removed` options can be useful on Windows to resolve issues related to file rotation. See [Open file handlers cause issues with Windows file rotation](/reference/filebeat/windows-file-rotation.md). The `close_eof` option can be useful in environments with a large number of files that have only very few entries. The `close_timeout` option is useful in environments where closing file handlers is more important than sending all log lines. For more details, see [Inputs](/reference/filebeat/configuration-filebeat-options.md).
21
+
The `close.on_state_change.renamed` and `close.on_state_change.removed` options can be useful on Windows to resolve issues related to file rotation. See [Open file handlers cause issues with Windows file rotation](/reference/filebeat/windows-file-rotation.md). The `close.reader.on_eof` option can be useful in environments with a large number of files that have only very few entries. The `close.reader.after_interval` option is useful in environments where closing file handlers is more important than sending all log lines. For more details, see [Inputs](/reference/filebeat/configuration-filebeat-options.md).
22
22
23
23
Make sure that you read the documentation for these configuration options before using any of them.
Copy file name to clipboardExpand all lines: docs/reference/filebeat/reduce-registry-size.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ applies_to:
8
8
9
9
# Registry file is too large [reduce-registry-size]
10
10
11
-
Filebeat keeps the state of each file and persists the state to disk in the registry file. The file state is used to continue file reading at a previous position when Filebeat is restarted. If a large number of new files are produced every day, the registry file might grow to be too large. To reduce the size of the registry file, there are two configuration options available: [`clean_removed`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-clean-removed) and [`clean_inactive`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-clean-inactive).
11
+
Filebeat keeps the state of each file and persists the state to disk in the registry file. The file state is used to continue file reading at a previous position when Filebeat is restarted. If a large number of new files are produced every day, the registry file might grow to be too large. To reduce the size of the registry file, there are two configuration options available: [`clean_removed`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-clean-removed) and [`clean_inactive`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-clean-inactive).
12
12
13
-
For old files that you no longer touch and are ignored (see [`ignore_older`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-ignore-older)), we recommended that you use `clean_inactive`. If old files get removed from disk, then use the `clean_removed` option.
13
+
For old files that you no longer touch and are ignored (see [`ignore_older`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-ignore-older)), we recommended that you use `clean_inactive`. If old files get removed from disk, then use the `clean_removed` option.
Copy file name to clipboardExpand all lines: docs/reference/filebeat/windows-file-rotation.md
+1-3Lines changed: 1 addition & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,10 +8,8 @@ applies_to:
8
8
9
9
# Open file handlers cause issues with Windows file rotation [windows-file-rotation]
10
10
11
-
On Windows, you might have problems renaming or removing files because Filebeat keeps the file handlers open. This can lead to issues with the file rotating system. To avoid this issue, you can use the [`close_removed`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-close-removed) and [`close_renamed`](/reference/filebeat/filebeat-input-log.md#filebeat-input-log-close-renamed) options together.
11
+
On Windows, you might have problems renaming or removing files because Filebeat keeps the file handlers open. This can lead to issues with the file rotating system. To avoid this issue, you can use the [`close.on_state_change.removed`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-close-removed) and [`close.on_state_change.renamed`](/reference/filebeat/filebeat-input-filestream.md#filebeat-input-filestream-close-renamed) options together.
12
12
13
13
::::{important}
14
14
When you configure these options, files may be closed before the harvester has finished reading the files. If the file cannot be picked up again by the input and the harvester hasn’t finish reading the file, the missing lines will never be sent to the output.
0 commit comments