Skip to content

Commit 2bc338f

Browse files
Watson1978github-actions[bot]
authored andcommitted
test_fluentd: fix flaky zero_downtime_restart from per-chunk log parsing (#5450)
**Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: The read loop in `run_fluentd` split each `eager_read` chunk on its own (`logs = buf.split("\n")`) and yielded that to the caller for counting. A single read can end in the middle of a line -- this happens when the source-only buffer is replayed and a whole chunk of records is flushed to `stdout` in one burst while the reader catches the pipe mid-write, which is much more likely on a loaded CI runner. When a record's line straddles two reads, neither half matches the record regexp, so the caller drops it from its count while the full line is still present in the accumulated output. Carry the incomplete trailing line across reads and yield only complete lines, the same way `assert_log_matches` already reads against the accumulated buffer. This PR will fix following error: ``` 4) Failure: test: should restart with zero downtime (no data loss)(TestFluentdCommand::zero_downtime_restart) /Users/runner/work/fluentd/fluentd/test/command/test_fluentd.rb:1574:in `block (2 levels) in <class:TestFluentdCommand>' 1571: end 1572: end 1573: => 1574: assert_equal( 1575: [(0..499).to_a, (0..499).to_a, (0..499).to_a], 1576: [ 1577: records_by_type["udp"].sort, <[[0, 1, 2, 3, 4, 5, ... ``` Ref. https://github.com/fluent/fluentd/actions/runs/29796336422/job/88528337017#step:6:5375 **Docs Changes**: N/A **Release Note**: N/A Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent 0579b12 commit 2bc338f

1 file changed

Lines changed: 14 additions & 2 deletions

File tree

test/command/test_fluentd.rb

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1413,6 +1413,13 @@ def run_fluentd(config)
14131413
stdio_buf = ""
14141414
execute_command(cmdline) do |pid, stdout|
14151415
begin
1416+
# Keep an incomplete trailing line across reads. A single read can end
1417+
# in the middle of a line (e.g. when the buffered records are flushed
1418+
# in a burst and the reader catches the pipe mid-write), so splitting
1419+
# each read chunk independently would drop the straddling record from
1420+
# the caller's counting. Carry the partial line over and only yield
1421+
# complete lines, exactly once each.
1422+
line_buf = +""
14161423
waiting(60) do
14171424
while true
14181425
readables, _, _ = IO.select([stdout], nil, nil, 1)
@@ -1421,11 +1428,16 @@ def run_fluentd(config)
14211428

14221429
buf = eager_read(readables.first)
14231430
stdio_buf << buf
1424-
logs = buf.split("\n")
1431+
line_buf << buf
1432+
# Complete lines to `logs`, trailing remainder to `line_buf`.
1433+
# The `-1` is required: without it split drops the trailing empty
1434+
# field, so a buffer ending in "\n" (the usual case) would carry
1435+
# over the last complete line and merge it with the next read.
1436+
*logs, line_buf = line_buf.split("\n", -1)
14251437

14261438
yield logs
14271439

1428-
break if buf.include? "finish test"
1440+
break if logs.any? { |log| log.include?("finish test") }
14291441
end
14301442
end
14311443
ensure

0 commit comments

Comments
 (0)