Skip to content

Backport(v1.19): test_fluentd: fix flaky zero_downtime_restart from per-chunk log parsing (#5450) - #5452

Merged
Watson1978 merged 1 commit into
v1.19from
backport-to-v1.19/pr5450
Jul 27, 2026
Merged

Backport(v1.19): test_fluentd: fix flaky zero_downtime_restart from per-chunk log parsing (#5450)#5452
Watson1978 merged 1 commit into
v1.19from
backport-to-v1.19/pr5450

Conversation

@github-actions

Copy link
Copy Markdown

Which issue(s) this PR fixes:
Backport #5450
Fixes #

What this PR does / why we need it:
The read loop in run_fluentd split each eager_read chunk on its own (logs = buf.split("\n")) and yielded that to the caller for counting.
A single read can end in the middle of a line -- this happens when the source-only buffer is replayed and a whole chunk of records is flushed to stdout in one burst while the reader catches the pipe mid-write, which is much more likely on a loaded CI runner.
When a record's line straddles two reads, neither half matches the record regexp, so the caller drops it from its count while the full line is still present in the accumulated output.

Carry the incomplete trailing line across reads and yield only complete lines, the same way assert_log_matches already reads against the accumulated buffer.

This PR will fix following error:

4) Failure: test: should restart with zero downtime (no data loss)(TestFluentdCommand::zero_downtime_restart)
/Users/runner/work/fluentd/fluentd/test/command/test_fluentd.rb:1574:in `block (2 levels) in <class:TestFluentdCommand>'
     1571:         end
     1572:       end
     1573: 
  => 1574:       assert_equal(
     1575:         [(0..499).to_a, (0..499).to_a, (0..499).to_a],
     1576:         [
     1577:           records_by_type["udp"].sort,
<[[0,
  1,
  2,
  3,
  4,
  5,
...

Ref. https://github.com/fluent/fluentd/actions/runs/29796336422/job/88528337017#step:6:5375

Docs Changes:
N/A

Release Note:
N/A

…ing (#5450)

**Which issue(s) this PR fixes**:
Fixes #

**What this PR does / why we need it**:
The read loop in `run_fluentd` split each `eager_read` chunk on its own
(`logs = buf.split("\n")`) and yielded that to the caller for counting.
A single read can end in the middle of a line -- this happens when the
source-only buffer is replayed and a whole chunk of records is flushed
to `stdout` in one burst while the reader catches the pipe mid-write,
which is much more likely on a loaded CI runner.
When a record's line straddles two reads, neither half matches the
record regexp, so the caller drops it from its count while the full line
is still present in the accumulated output.

Carry the incomplete trailing line across reads and yield only complete
lines, the same way `assert_log_matches` already reads against the
accumulated buffer.

This PR will fix following error:
```
4) Failure: test: should restart with zero downtime (no data loss)(TestFluentdCommand::zero_downtime_restart)
/Users/runner/work/fluentd/fluentd/test/command/test_fluentd.rb:1574:in `block (2 levels) in <class:TestFluentdCommand>'
     1571:         end
     1572:       end
     1573:
  => 1574:       assert_equal(
     1575:         [(0..499).to_a, (0..499).to_a, (0..499).to_a],
     1576:         [
     1577:           records_by_type["udp"].sort,
<[[0,
  1,
  2,
  3,
  4,
  5,
...
```
Ref.
https://github.com/fluent/fluentd/actions/runs/29796336422/job/88528337017#step:6:5375

**Docs Changes**:
N/A

**Release Note**:
N/A

Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@Watson1978
Watson1978 merged commit 74734fb into v1.19 Jul 27, 2026
18 checks passed
@Watson1978
Watson1978 deleted the backport-to-v1.19/pr5450 branch July 27, 2026 01:52
@Watson1978 Watson1978 added this to the v1.19.4 milestone Jul 27, 2026
@Watson1978 Watson1978 added the CI Test/CI issues label Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Test/CI issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant