Backport(v1.19): test_fluentd: fix flaky zero_downtime_restart from per-chunk log parsing (#5450) - #5452
Merged
Merged
Conversation
…ing (#5450) **Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: The read loop in `run_fluentd` split each `eager_read` chunk on its own (`logs = buf.split("\n")`) and yielded that to the caller for counting. A single read can end in the middle of a line -- this happens when the source-only buffer is replayed and a whole chunk of records is flushed to `stdout` in one burst while the reader catches the pipe mid-write, which is much more likely on a loaded CI runner. When a record's line straddles two reads, neither half matches the record regexp, so the caller drops it from its count while the full line is still present in the accumulated output. Carry the incomplete trailing line across reads and yield only complete lines, the same way `assert_log_matches` already reads against the accumulated buffer. This PR will fix following error: ``` 4) Failure: test: should restart with zero downtime (no data loss)(TestFluentdCommand::zero_downtime_restart) /Users/runner/work/fluentd/fluentd/test/command/test_fluentd.rb:1574:in `block (2 levels) in <class:TestFluentdCommand>' 1571: end 1572: end 1573: => 1574: assert_equal( 1575: [(0..499).to_a, (0..499).to_a, (0..499).to_a], 1576: [ 1577: records_by_type["udp"].sort, <[[0, 1, 2, 3, 4, 5, ... ``` Ref. https://github.com/fluent/fluentd/actions/runs/29796336422/job/88528337017#step:6:5375 **Docs Changes**: N/A **Release Note**: N/A Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Watson1978
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Backport #5450
Fixes #
What this PR does / why we need it:
The read loop in
run_fluentdsplit eacheager_readchunk on its own (logs = buf.split("\n")) and yielded that to the caller for counting.A single read can end in the middle of a line -- this happens when the source-only buffer is replayed and a whole chunk of records is flushed to
stdoutin one burst while the reader catches the pipe mid-write, which is much more likely on a loaded CI runner.When a record's line straddles two reads, neither half matches the record regexp, so the caller drops it from its count while the full line is still present in the accumulated output.
Carry the incomplete trailing line across reads and yield only complete lines, the same way
assert_log_matchesalready reads against the accumulated buffer.This PR will fix following error:
Ref. https://github.com/fluent/fluentd/actions/runs/29796336422/job/88528337017#step:6:5375
Docs Changes:
N/A
Release Note:
N/A