storage/sources/postgres: support choosing a subset of columns #29689
benesch
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Overview
When creating PostgreSQL sources, allow users to replicate only a subset of columns for each table.
Details
Since PostgreSQL v15, PostgreSQL has supported restricting a publication to just a subset of columns on each table.
This might just work out of the box with our existing PostgreSQL source ... but I think it's more likely that this situation causes an immediate replication failure because we'll include all columns from the base table and do a snapshot including all columns, and then complain when those columns aren't present in the replication stream.
Supporting this probably isn't too much work. When someone creates a PostgreSQL source or adds a new table to an existing PostgreSQL source, we just need to teach Materialize to check whether the PostgreSQL server is v15+ and if so, exclude any columns from the table that are not present in the publication.
Beta Was this translation helpful? Give feedback.
All reactions