Skip to content

Conversation

@DrakeLin
Copy link
Collaborator

@DrakeLin DrakeLin commented Sep 19, 2025

What changes are proposed in this pull request?

Moved partition value parsing, transform expression generation, and transform spec computation from log_replay.rs into a new dedicated transform_processing.rs module. This improves code organization and reusability.

This is a prefactor for CDF TransformSpec support

How was this change tested?

Existing tests

@github-actions github-actions bot added the breaking-change Change that require a major version bump label Sep 19, 2025
@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch 3 times, most recently from cae3732 to 4594949 Compare September 19, 2025 18:17
@DrakeLin DrakeLin changed the title ref refactor: pull out transform spec utils Sep 19, 2025
@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch from 4594949 to 630663b Compare September 19, 2025 18:54
@DrakeLin DrakeLin changed the title refactor: pull out transform spec utils refactor: pull out transform spec utils and definitions Sep 19, 2025
@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch 3 times, most recently from 2c91a87 to 42b7500 Compare September 19, 2025 19:03
@codecov
Copy link

codecov bot commented Sep 19, 2025

Codecov Report

❌ Patch coverage is 96.78899% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.39%. Comparing base (b9977cc) to head (3713575).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
kernel/src/transforms.rs 97.61% 2 Missing and 3 partials ⚠️
kernel/src/scan/log_replay.rs 75.00% 0 Missing and 1 partial ⚠️
kernel/src/table_changes/physical_to_logical.rs 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1326      +/-   ##
==========================================
+ Coverage   84.28%   84.39%   +0.11%     
==========================================
  Files         111      112       +1     
  Lines       27641    27773     +132     
  Branches    27641    27773     +132     
==========================================
+ Hits        23296    23440     +144     
+ Misses       3215     3201      -14     
- Partials     1130     1132       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch 2 times, most recently from e533c1e to f5e84e0 Compare September 19, 2025 19:20
Copy link
Collaborator

@OussamaSaoudi OussamaSaoudi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just one comment. I love that we're consolidating all this.

pub mod table_features;
pub mod table_properties;
pub mod transaction;
pub mod transforms;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there's nothing public, I think we can just make this pub(crate). If things break, just keep it pub mod

Suggested change
pub mod transforms;
pub(crate) mod transforms;

@OussamaSaoudi
Copy link
Collaborator

This isn't a breaking change, so pls remove the breaking change label.

Copy link
Collaborator

@nicklan nicklan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, l lgtm! thanks, let's just make this pub(crate) as Oussama suggested

@DrakeLin
Copy link
Collaborator Author

This isn't a breaking change, so pls remove the breaking change label.

booo github actions booooo

@DrakeLin DrakeLin removed the breaking-change Change that require a major version bump label Sep 19, 2025
@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch from f5e84e0 to 281b620 Compare September 19, 2025 22:15
@DrakeLin DrakeLin force-pushed the drake-lin_data/stack/exp-refactor branch from 281b620 to e15f8ea Compare September 19, 2025 22:18
@DrakeLin DrakeLin merged commit c1c301b into delta-io:main Sep 19, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants