Skip to content

Commit 45a76fb

Browse files
committed
docs: add "got incomplete line before first line" warning in in_tail
Add an FAQ entry explaining that in_tail emits this warning only in multiline mode (format_firstline), when it reads a continuation line before matching a first line. List the common causes: reading that starts in the middle of a record (e.g. after rotation or with read_from_head false), a format_firstline that does not match the real first line, and a single <parse> shared by files of different formats through a path glob. Note that these lines are dropped unless emit_unmatched_lines is set to true. Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
1 parent f10996c commit 45a76fb

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

input/tail.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -522,6 +522,24 @@ Maximum number of lines allowed from a group in `rate_period` time interval. The
522522

523523
See also `emit_unmatched_lines` parameter.
524524

525+
### What does the `got incomplete line before first line` warning mean?
526+
527+
This warning appears only when you use a multiline parser (a `<parse>` section with `@type multiline` and `format_firstline`). In multiline mode, `in_tail` starts buffering a record from the line that matches `format_firstline` and appends the following lines to it. If `in_tail` reads a continuation line before it has ever matched a first line, the line cannot be assembled into a complete event, so `in_tail` discards it and prints a warning like this for each such line:
528+
529+
```text
530+
2023-01-01 00:00:00 +0900 [warn]: #0 got incomplete line before first line from /path/to/file: " at Foo.bar(Foo.java:42)\n"
531+
```
532+
533+
The common causes are:
534+
535+
* Reading starts in the middle of a record, for example right after a log rotation, or on startup when `read_from_head` is `false`. In this case the warning is temporary and harmless: only the incomplete head of the record at the starting position is skipped, and the subsequent complete records are processed normally.
536+
* `format_firstline` does not match the actual first line of your records. Then no line ever becomes the start of a record, so the warning keeps appearing. Review the pattern.
537+
* A single `<parse>` section is shared, through a `path` glob, by files whose formats differ. Files that never match `format_firstline` keep producing this warning. Use a separate `in_tail` source for each format instead.
538+
539+
**Note:** these lines are dropped. Because `emit_unmatched_lines` is `false` by default, the warning is the only sign that the data is being lost. Set `emit_unmatched_lines true` to emit them instead; each line is then emitted as `{"unmatched_line" : incoming line}`.
540+
541+
See the [multiline parser](../parser/multiline.md) for the parser configuration.
542+
525543
### `in_tail` doesn't start to read the log file, why?
526544

527545
`in_tail` follows `tail -F` command's behavior by default, so `in_tail` reads only the new logs. If you want to read the existing lines for the batch use case, set `read_from_head true`.

0 commit comments

Comments
 (0)