Open
Description
Feature request
@rjobanp has added envelope
, key_format
, and value_format
columns to the mz_internal.mz_kafka_source_tables
table in this PR.
We need to extend the logic in the read
function within resource_source_table_kafka.go
to introduce parsing and mapping logic for the complex envelope, key, and value Terraform schema structures.
The new table values look like:
materialize=> select * from mz_internal.mz_kafka_source_tables;
id | topic | envelope_type | key_format | value_format
------+--------+-------------------------+------------+--------------
u333 | topic1 | none | |
u529 | topic1 | upsert-value-err-inline | text | text
We need to:
- Parse the
envelope
,key_format
, andvalue_format
columns from themz_internal.mz_kafka_source_tables
. - Map these parsed values to the appropriate Terraform schema.
- Ensure the complex structures are accurately reflected in the resource state, especially for
envelope_type
, which may impact the resource behavior in Terraform.