Skip to content

feat: new cookbook for data movement#15

Merged
esadek merged 4 commits into
columnar-tech:mainfrom
amoeba:feat/movement
May 18, 2026
Merged

feat: new cookbook for data movement#15
esadek merged 4 commits into
columnar-tech:mainfrom
amoeba:feat/movement

Conversation

@amoeba
Copy link
Copy Markdown
Member

@amoeba amoeba commented Apr 29, 2026

I keep seeing people not getting my favorite point about ADBC: The RecordBatchReader-as-a-service. Here's an attempt.

@amoeba amoeba requested review from esadek and ianmcook April 29, 2026 19:03
Comment thread notebooks/moving-data-between-databases-with-adbc.ipynb Outdated
Comment thread notebooks/moving-data-between-databases-with-adbc.ipynb Outdated
"cell_type": "markdown",
"id": "d9e9e7af",
"metadata": {},
"source": "## Takeaways\n\nWhile the above example moved only 344 rows of data between our two databases, it illustrates an important point that can be extrapolated to data of any scale: ADBC is a fast but also uniform API for not only connecting to databases but also connecting databases together. Without a standard interface like ADBC, every pair of databases you wanted to move data between would need a separate solution. In a world with ADBC, you only need two ADBC drivers and a little bit of Python (or your language of choice) code."
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be cool to actually use a dataset that doesn't fit in memory.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure.

@amoeba amoeba requested a review from esadek April 29, 2026 21:32
@amoeba
Copy link
Copy Markdown
Member Author

amoeba commented Apr 29, 2026

I removed the outputs. GizmoSQL doesn't start up fast so to make the notebook runnable with nbconvert I'd have to add a code cell I'd rather not. It looks like the other examples don't have any similar logic so maybe this is fine?

@amoeba
Copy link
Copy Markdown
Member Author

amoeba commented May 6, 2026

Hi @esadek do you want to take a look again? I fixed the issues and I re-ran all the cells and selectively kept just the output I think we want in the final output.

@esadek esadek merged commit 43b405f into columnar-tech:main May 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants