Extend query config with query.masked config with Type.PASSWORD for secure query masking#1593
Conversation
…ecure query masking
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
src/main/java/io/confluent/connect/jdbc/source/JdbcSourceConnectorConfig.java
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Few things KGaneshDatta
- Since query was never exposed on cloud before this, we should perform some manual testing to understand/validate the behaviour. (How frequently will the query be executed, What if the query is taking too long to exceute ? do we have any observability into this ? Can multiple queries be provided (
select * from table1;select* from table2;)) - If user has specified both table include/exclude list and query, we don't know which way they wanna go. The validation exception message currently instructs them to remove table include/exclude configs. Which might be other way round also.
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Outdated
Show resolved
Hide resolved
…on to allow only select queries in query configs
…masked has been set
src/main/java/io/confluent/connect/jdbc/source/JdbcSourceConnectorConfig.java
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/JdbcSourceConnectorConfig.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/JdbcSourceTask.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerierProcessor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerierProcessor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerierProcessor.java
Show resolved
Hide resolved
Animesh Kumar (akanimesh7)
left a comment
There was a problem hiding this comment.
Missing validation -- If mode is query and timestamp.initial is provided validation passes.
src/main/java/io/confluent/connect/jdbc/source/TableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Show resolved
Hide resolved
The query mode would be working in timestamp criteria also. More details here - https://docs.confluent.io/kafka-connectors/jdbc/current/source-connector/overview.html#specifying-a-where-clause-with-query-modes |
src/main/java/io/confluent/connect/jdbc/source/BulkTableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TimestampIncrementingTableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TimestampTableQuerier.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Show resolved
Hide resolved
src/test/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidationTest.java
Outdated
Show resolved
Hide resolved
Animesh Kumar (akanimesh7)
left a comment
There was a problem hiding this comment.
trim.sensitive.log is a config which was already existing in the sink connectors.
Now we are using the same config in source connectors. Although we have defined it in the ConfigDef, but there is no way to set a value for this config. Please check this.
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/validation/JdbcSourceConnectorValidation.java
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerierProcessor.java
Outdated
Show resolved
Hide resolved
src/main/java/io/confluent/connect/jdbc/source/TableQuerierProcessor.java
Outdated
Show resolved
Hide resolved
|
Animesh Kumar (akanimesh7)
left a comment
There was a problem hiding this comment.
Thanks KGaneshDatta



Problem
Type.Passwordto ensure secure logging by masking sensitive values and preventing accidental exposure.table.modeorquery.mode.Solution
Does this solution apply anywhere else?
If yes, where?
Test Strategy
Testing done:
Release Plan