-
Notifications
You must be signed in to change notification settings - Fork 11
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Feature request
As per the discussion here: MaterializeInc/materialize#27907 and based on the design doc here: #645 we need to introduce the support for tables from sources on the Terraform provider side as well.
Extra details to be discussed in the design doc pull request, but the main things to keep in mind are:
Initial tasks
- Deprecated the related table attributes on the source resources, eg. MySQL, Postgres, Loadgens
- Introduce new
resource_source_table_{mysql|postgres|load_generator}resources - Add tests including unit tests and acceptance tests
- Handle source-specific attributes like the MySQL
ignore_columnsin theresource_table_from_sourceresource - Update all of the related documentation
- Write a migrations guide for users using the old implementation
- Add data sources
resource_source_tableresources - Add import and documentation examples
- Once ready add upstream table name to read query and read funcs + acceptance tests
- Add integration tests
Kafka Tasks
- Deprecated the related table attributes on the source resources for the Kafka source resource
- Introduce a new
resource_source_table_kafkaresource - Add tests including unit tests
- Update all of the related Kafka source documentation
- Add import and documentation examples
- Update the migrations guide for users using the old implementation to include Kafka sources
- Add acceptance tests
- Add integration tests
- Add the envelope, key_format and value_format columns to the read func
SQL server Tasks
- Deprecated the related table attributes on the source resources for the SQL Server source resource
- Introduce a new
resource_source_table_sqlserverresource - Add tests including unit tests
- Update all of the related SQL Server source documentation
- Add import and documentation examples
- Update the migrations guide for users using the old implementation to include SQL server sources
- Add acceptance tests
- Add integration tests
Webhook Tasks
- Deprecated the related table attributes on the source resources for the Webhook source resource
- Introduce a new
resource_source_table_webhookresource - Add tests including unit tests, integration tests and acceptance tests
- Update all of the related Webhook source documentation
- Add import and documentation examples
- Update the migrations guide for users using the old implementation to include webhooks sources
Additional Tasks
- Data source for
mz_internal.available_source_references - Add unit tests for the source reference data source
- Add acceptance tests for the source reference data source
- Add
ALTER SOURCE .. REFRESH REFERENCES
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request