fix: clear the target chunk when the copy goes through the hypertable - #207
Open
alejandrodnm wants to merge 1 commit into
Open
Conversation
Point the delete and the cagg invalidation trigger at the chunk that ends up holding the rows, rather than at wherever the copy happens to be written. Add a regression test covering a partial source chunk copied twice with the target recompressed in between. Claude-Session: https://claude.ai/code/session_01DMDRhYXfUK4WpAD6zUfeaW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the source chunk is partial and the target chunk isn't,
copy_chunkwrites the uncompressed rows through the hypertable instead of into the chunk, because setting a chunk's status to partial needs a permissiontsdbadmindoesn't have. The delete meant to clear the range followed the copy to the hypertable, andDELETE FROM ONLY <hypertable>clears nothing: a hypertable keeps every row in its chunks, so its own heap is always empty. On that path the copy adds to whatever the chunk already held.It stays invisible while the branch is only reached on a first copy, where the chunk is empty anyway. Writing the rows through the hypertable makes the chunk partial, so a second copy takes the other branch and deletes normally. But anything that folds those rows back into compressed form, which is what a compression policy does, clears the partial status and puts a second copy back on this branch with data already in place, and the target ends up with twice the rows.
The delete and the cagg invalidation trigger now act on the chunk, the relation that ends up holding the rows, while the copy still goes wherever the caller needs it. That also restores the assumption #206 relies on: from TimescaleDB 2.23 the delete that clears a chunk's uncompressed rows removes its compressed batches too, which only holds if that delete reaches the chunk in the first place.
Stacked on #206, review that one first.
https://claude.ai/code/session_01DMDRhYXfUK4WpAD6zUfeaW