Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

return block data when deleting it #175

Open
wants to merge 1 commit into
base: 03-13-do_not_pass_around_pointers_to_slices_and_maps
Choose a base branch
from

Conversation

iuwqyir
Copy link
Collaborator

@iuwqyir iuwqyir commented Mar 13, 2025

TL;DR

Enhanced DeleteBlockData to return deleted block data, enabling better handling of reorgs.

What changed?

Modified the DeleteBlockData method in the storage interface to return the deleted block data:

  • Updated the method signature in IMainStorage interface to return ([]common.BlockData, error)
  • Implemented the new functionality in ClickHouseConnector to collect and return deleted data
  • Modified the individual delete methods (deleteBlocks, deleteLogs, etc.) to return the deleted entities
  • Updated the MemoryConnector implementation to match the new interface
  • Fixed the mock implementation to support the new signature
  • Updated the caller in ReorgHandler to handle the new return value

How to test?

  1. Run unit tests to verify the updated interface implementations
  2. Test a reorg scenario to ensure deleted block data is properly returned
  3. Verify that the reorg handler correctly processes the returned data

Why make this change?

This change improves the reorg handling process by providing access to the deleted block data, which can be useful for:

  • Auditing what was removed during a reorg
  • Potentially using the deleted data for other purposes
  • Making the reorg process more transparent and traceable
  • Setting the foundation for making delete and insert operations atomic

Copy link
Collaborator Author

iuwqyir commented Mar 13, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@iuwqyir iuwqyir marked this pull request as ready for review March 13, 2025 19:55
@iuwqyir iuwqyir force-pushed the 03-13-do_not_pass_around_pointers_to_slices_and_maps branch from c577d09 to 89189a0 Compare March 13, 2025 19:57
@iuwqyir iuwqyir force-pushed the 03-13-return_block_data_when_deleting_it branch 2 times, most recently from e3d5d28 to 1f71d1d Compare March 13, 2025 19:59
@iuwqyir iuwqyir force-pushed the 03-13-do_not_pass_around_pointers_to_slices_and_maps branch from 89189a0 to 1e5ad3e Compare March 17, 2025 12:16
@iuwqyir iuwqyir force-pushed the 03-13-return_block_data_when_deleting_it branch 2 times, most recently from 0072cf9 to edaf37c Compare March 18, 2025 10:15
@iuwqyir iuwqyir force-pushed the 03-13-do_not_pass_around_pointers_to_slices_and_maps branch from 1e5ad3e to 2d907a6 Compare March 18, 2025 10:15
@iuwqyir iuwqyir force-pushed the 03-13-return_block_data_when_deleting_it branch from edaf37c to 5504a56 Compare March 18, 2025 22:49
@iuwqyir iuwqyir force-pushed the 03-13-do_not_pass_around_pointers_to_slices_and_maps branch from 2d907a6 to 3f090a9 Compare March 18, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants