-
Notifications
You must be signed in to change notification settings - Fork 95
✨ Postgres sink #2029
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
✨ Postgres sink #2029
Conversation
3f61302
to
fd00020
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome! Looking forward to try out the PG sink soon! I am aware this PR is on the works but I'm leaving a couple of notes in case they are useful
end | ||
|
||
@impl SinkPipeline | ||
def handle_batch(:default, messages, _batch_info, context) do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a reminder to call the routing logic here in the pipeline
{:ok, _} <- | ||
Postgrex.query( | ||
conn, | ||
"INSERT INTO #{sink.table_name} (action, record, changes, metadata) VALUES ($1, $2, $3, $4)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we should figure out a way to escape the table_name to avoid sql injections?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh I noticed you added a validation to not include whitespace, good idea
In case a more restrictive validation were useful, here is the official definition of an identifier from the PG docs:
SQL identifiers and key words must begin with a letter (a-z, but also letters with diacritical marks and non-Latin letters) or an underscore (_). Subsequent characters in an identifier or key word can be letters, underscores, digits (0-9), or dollar signs ($). Note that dollar signs are not allowed in identifiers according to the letter of the SQL standard, so their use might render applications less portable. The SQL standard will not define a key word that contains digits or starts or ends with an underscore, so identifiers of this form are safe against possible conflict with future extensions of the standard.
https://www.postgresql.org/docs/17/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS
Yes this PR is very early still! Should have been draft. But I appreciate the early feedback 🙏 |
fd00020
to
1a0aada
Compare
1a0aada
to
9f31e33
Compare
c9b03c3
to
343c3c6
Compare
No description provided.