Skip to content

feat: support streaming iceberg source #20568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 36 commits into from
Apr 1, 2025
Merged

feat: support streaming iceberg source #20568

merged 36 commits into from
Apr 1, 2025

Conversation

xxchan
Copy link
Member

@xxchan xxchan commented Feb 21, 2025 •
edited
Loading

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

image

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

This was referenced Feb 21, 2025
Merged
Merged
@xxchan Graphite App
Copy link
Member Author

xxchan commented Feb 21, 2025 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@xxchan xxchan marked this pull request as ready for review February 21, 2025 10:19
@xxchan xxchan requested a review from a team as a code owner February 21, 2025 10:19
@xxchan xxchan requested review from cyliu0 and removed request for a team February 21, 2025 10:19
@xxchan xxchan changed the title . feat: support streaming iceberg source Feb 21, 2025
@xxchan xxchan removed the request for review from cyliu0 February 21, 2025 10:19
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch from 88a5368 to 743f7de Compare February 21, 2025 15:22
@xxchan xxchan mentioned this pull request Feb 21, 2025
Merged
8 tasks
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch from 743f7de to 13105d9 Compare February 24, 2025 05:47
@xxchan xxchan force-pushed the xxchan/ice-2 branch 2 times, most recently from bfc984a to ffcd470 Compare February 24, 2025 05:51
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch 2 times, most recently from ad67478 to 0fd9ca2 Compare February 24, 2025 05:54
@xxchan xxchan force-pushed the xxchan/ice-2 branch 2 times, most recently from 2e87928 to 48fe8b5 Compare February 24, 2025 06:56
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch from 0fd9ca2 to b2d6e95 Compare February 24, 2025 06:56
Base automatically changed from xxchan/ice-2 to main February 24, 2025 14:04
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch from b2d6e95 to 262e3cc Compare February 26, 2025 02:40
@xxchan xxchan mentioned this pull request Feb 26, 2025
Closed
8 tasks
@xxchan xxchan force-pushed the xxchan/iceberg-stream-source branch from 262e3cc to f4b30c2 Compare March 13, 2025 06:43
@xxchan xxchan requested a review from chenzl25 March 24, 2025 08:29
xxchan added 12 commits March 25, 2025 14:55
@xxchan
Merge remote-tracking branch 'origin/main' into xxchan/iceberg-stream…
b32b02e 
…-source
@xxchan
Merge remote-tracking branch 'origin/main' into xxchan/iceberg-stream…
465cb08 
…-source
@xxchan
- dead code
7dae0aa 
- config chunk size

Signed-off-by: xxchan <[email protected]>
@xxchan
Merge remote-tracking branch 'origin/main' into xxchan/iceberg-stream…
4c84b48 
…-source

Signed-off-by: xxchan <[email protected]>
@xxchan
fix
e405eba 
Signed-off-by: xxchan <[email protected]>
@xxchan
fix example.toml
e630f30 
Signed-off-by: xxchan <[email protected]>
@xxchan
compatibility layer
8fcaf2e 
Signed-off-by: xxchan <[email protected]>
@xxchan
fix
ddb98d5 
Signed-off-by: xxchan <[email protected]>
@xxchan
fix
f89aa1a 
Signed-off-by: xxchan <[email protected]>
@xxchan
fix
98805d5 
Signed-off-by: xxchan <[email protected]>
@xxchan
mod
ee2e049 
Signed-off-by: xxchan <[email protected]>
@xxchan
fix
79a16f9 
Signed-off-by: xxchan <[email protected]>
@xxchan xxchan requested a review from chenzl25 March 26, 2025 15:55
chenzl25
chenzl25 approved these changes Mar 27, 2025
View reviewed changes
Copy link
Contributor

@chenzl25 chenzl25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@xxchan xxchan enabled auto-merge March 27, 2025 05:26
@xxchan
Copy link
Member Author

xxchan commented Mar 27, 2025

image 🤔

@xxchan xxchan disabled auto-merge March 27, 2025 12:33
@xxchan xxchan enabled auto-merge March 31, 2025 14:13
stdrc
stdrc approved these changes Apr 1, 2025
View reviewed changes
Copy link
Member

@stdrc stdrc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cargo.lock

@xxchan xxchan added this pull request to the merge queue Apr 1, 2025
Merged via the queue into main with commit b936b99 Apr 1, 2025
30 of 31 checks passed
@xxchan xxchan deleted the xxchan/iceberg-stream-source branch April 1, 2025 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

Copilot code review Copilot Copilot left review comments

@stdrc stdrc stdrc approved these changes

@chenzl25 chenzl25 chenzl25 approved these changes

Assignees
No one assigned
Labels
ci/main-cron/run-selected ci/run-e2e-iceberg-tests ci/run-s3-source-tests type/feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@xxchan @chenzl25 @stdrc @wcy-fdu