Summary
After a mid-segment connection failure during a structured message (XSM/1.0) download, RetryReader resumes from an HTTP range offset past bytes that were already emitted to the caller but whose segment CRC footer has not yet been validated. The retried response validates only the suffix of that segment, so corrupted bytes in the already-emitted prefix can survive a successful retry.
Current state
PR #27348 fixed several related bugs:
- Exact-buffer-fill validation skipping
- Error wrapping (
%w) so RetryReader recognizes transient failures
- Missing CRC64 flag rejection
- Truncated framing retryability (
fillFrame now wraps io.ErrUnexpectedEOF)
- Errors at exact segment completion boundaries
These fixes close the more common failure modes. The mid-segment retry gap remains as a known limitation.
Proposed fix
Either:
- Buffer full segments in
SMDecoder before emitting data to the caller, so RetryReader never advances past unvalidated bytes, or
- Make
RetryReader checkpoint at validated segment boundaries rather than on every Read call
Both approaches change the reader contract and need careful design.
References
Summary
After a mid-segment connection failure during a structured message (XSM/1.0) download,
RetryReaderresumes from an HTTP range offset past bytes that were already emitted to the caller but whose segment CRC footer has not yet been validated. The retried response validates only the suffix of that segment, so corrupted bytes in the already-emitted prefix can survive a successful retry.Current state
PR #27348 fixed several related bugs:
%w) soRetryReaderrecognizes transient failuresfillFramenow wrapsio.ErrUnexpectedEOF)These fixes close the more common failure modes. The mid-segment retry gap remains as a known limitation.
Proposed fix
Either:
SMDecoderbefore emitting data to the caller, soRetryReadernever advances past unvalidated bytes, orRetryReadercheckpoint at validated segment boundaries rather than on everyReadcallBoth approaches change the reader contract and need careful design.
References