fix(core): retain buffered data in WriteGenerator on write failure#7400
Open
TennyZhuang wants to merge 2 commits intomainfrom
Open
fix(core): retain buffered data in WriteGenerator on write failure#7400TennyZhuang wants to merge 2 commits intomainfrom
TennyZhuang wants to merge 2 commits intomainfrom
Conversation
WriteGenerator's write() and close() methods called buffer.take() before attempting to write to the underlying writer. If the write failed, the buffered data was lost, making retry impossible without data loss. Fix by cloning the taken buffer before writing, and restoring it on failure. This matches the invariant maintained by other writer implementations (AppendWriter, BlockWriter, MultipartWriter, PositionWriter). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dosubot
Bot
added
size:L
… duplication On write failure in inexact mode, only restore the previously buffered data (not the current bs). The caller will retry with the same bs, so including it in the restored buffer would cause data duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dentiny
reviewed
Apr 20, 2026
Contributor
dentiny
left a comment
dentiny
left a comment
There was a problem hiding this comment.
May I ask some dumb questions?
- It seems to me that the "issue" reproduces if users call
writeagain after a failedwrite. In production, why would users do that? - In terms of retry layer (which calls write repeatedly after failure), it seems to sit below the write generator. I'm curious if we could provide a reproducible example to show data loss with retry layer used together?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A - discovered via deterministic testing of error-handling paths.
Rationale
WriteGenerator'swrite()andclose()methods calledbuffer.take()before attempting to write to the underlying writer. If the write failed, the buffered data was permanently lost, making retry impossible without data loss.This violated the invariant maintained by other writer implementations (
AppendWriter,BlockWriter,MultipartWriter,PositionWriter), which only clear state after a successful write.Changes
close()method: Same clone-and-restore pattern in the flush loop.FailOnceMockWriterthat fails the first write call, then succeeds — verifying no data loss in inexact mode, exact mode, and close() paths.Are there any user-facing changes?
Users who rely on retry (e.g., via
RetryLayer) will no longer silently lose data when the underlying storage temporarily fails during a write flush.This PR was generated with the assistance of AI (Claude).