Skip to content

Conversation

@fvaleye
Copy link
Collaborator

@fvaleye fvaleye commented Sep 16, 2025

Description

Implement TableProvider::insert_into to support programmatic API calls, and extend DataSink to enable SQL insert operations in DataFusion.

Related Issue(s)

Documentation

…se it via SQL through DataSink

Signed-off-by: Florian Valeye <[email protected]>
@fvaleye fvaleye added enhancement New feature or request binding/rust Issues for the Rust crate labels Sep 16, 2025
@codecov
Copy link

codecov bot commented Sep 16, 2025

Codecov Report

❌ Patch coverage is 87.39837% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.71%. Comparing base (19e1eb2) to head (54e45f4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
crates/core/src/delta_datafusion/table_provider.rs 87.39% 7 Missing and 24 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3762      +/-   ##
==========================================
+ Coverage   75.63%   75.71%   +0.08%     
==========================================
  Files         145      145              
  Lines       44465    44709     +244     
  Branches    44465    44709     +244     
==========================================
+ Hits        33629    33852     +223     
+ Misses       9190     9185       -5     
- Partials     1646     1672      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@rtyler rtyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to refactor this code out of the monster table_provider.rs file, but that is an exercise which can be done after merge

@rtyler rtyler merged commit 1d6ba3d into delta-io:main Sep 16, 2025
26 checks passed

/// Write the data to the delta table
/// This is used for insert into operation
async fn write_all(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rtyler @fvaleye I believe this should have been piped through the WriteBuilder because now we have another surface layer where we write differently

@fvaleye
Copy link
Collaborator Author

fvaleye commented Sep 16, 2025

@rtyler @ion-elgreco here to help with refactoring 👋 .
I have added this feature to understand the integration better, but the DataFusion integration requires further refactoring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binding/rust Issues for the Rust crate enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support DataFusion Insert Into

3 participants