Skip to content

Commit c327d6d

Browse files
authored
docs: add in-progress note w.r.t. supporting upstream table schema ch… (#32436)
…anges
1 parent 0cd0fd1 commit c327d6d

File tree

4 files changed

+36
-17
lines changed

4 files changed

+36
-17
lines changed

doc/user/content/sql/create-source/mysql.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -206,11 +206,13 @@ debugging related issues, see [Troubleshooting](/ops/troubleshooting/).
206206

207207
## Known limitations
208208

209-
##### Schema changes
209+
### Schema changes
210+
211+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
210212

211213
{{% schema-changes %}}
212214

213-
##### Supported types
215+
### Supported types
214216

215217
Materialize natively supports the following MySQL types:
216218

@@ -259,7 +261,7 @@ The specified columns will be treated as `text`, and will thus not offer the
259261
expected MySQL type features. For any unsupported data types not listed above,
260262
use the [`EXCLUDE COLUMNS`](#excluding-columns) option.
261263

262-
##### Truncation
264+
### Truncation
263265

264266
Tables replicated into Materialize should not be truncated. If a table is
265267
truncated while replicated, the whole source becomes inaccessible and will not
@@ -272,15 +274,15 @@ DELETE FROM t;
272274

273275
## Examples
274276

275-
{{< warning >}}
277+
{{< important >}}
276278
Before creating a MySQL source, you must enable GTID-based binlog replication in the
277279
upstream database. For step-by-step instructions, see the integration guide for
278280
your MySQL service: [Amazon RDS](/ingest-data/mysql/amazon-rds/),
279281
[Amazon Aurora](/ingest-data/mysql/amazon-aurora/),
280282
[Azure DB](/ingest-data/mysql/azure-db/),
281283
[Google Cloud SQL](/ingest-data/mysql/google-cloud-sql/),
282284
[Self-hosted](/ingest-data/mysql/self-hosted/).
283-
{{< /warning >}}
285+
{{< /important >}}
284286

285287
### Creating a connection
286288

@@ -409,6 +411,8 @@ CREATE SOURCE mz_source
409411

410412
### Handling errors and schema changes
411413

414+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
415+
412416
To handle upstream [schema changes](#schema-changes) or errored subsources, use
413417
the [`DROP SOURCE`](/sql/alter-source/#context) syntax to drop the affected
414418
subsource, and then [`ALTER SOURCE...ADD SUBSOURCE`](/sql/alter-source/) to add

doc/user/content/sql/create-source/postgres.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -196,11 +196,13 @@ ingestion progress and debugging related issues, see [Troubleshooting](/ops/trou
196196

197197
## Known limitations
198198

199-
##### Schema changes
199+
### Schema changes
200+
201+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
200202

201203
{{% schema-changes %}}
202204

203-
#### Publication membership
205+
### Publication membership
204206

205207
PostgreSQL's logical replication API does not provide a signal when users remove
206208
tables from publications. Because of this, Materialize relies on periodic checks
@@ -217,7 +219,7 @@ To mitigate this issue, if you need to drop and re-add a table to a publication,
217219
ensure that you remove the table/subsource from the source _before_ re-adding it
218220
using the [`DROP SOURCE`](/sql/drop-source/) command.
219221

220-
##### Supported types
222+
### Supported types
221223

222224
Materialize natively supports the following PostgreSQL types (including the
223225
array type for each of the types):
@@ -264,7 +266,7 @@ example:
264266
* [`money`]: the resulting `text` value cannot be cast back to e.g. `numeric`,
265267
since PostgreSQL adds typical currency formatting to the output.
266268

267-
##### Truncation
269+
### Truncation
268270

269271
Tables replicated into Materialize should not be truncated. If a table is
270272
truncated while replicated, the whole source becomes inaccessible and will not
@@ -275,7 +277,7 @@ using an unqualified `DELETE`.
275277
DELETE FROM t;
276278
```
277279

278-
##### Inherited tables
280+
### Inherited tables
279281

280282
When using [PostgreSQL table inheritance](https://www.postgresql.org/docs/current/tutorial-inheritance.html),
281283
PostgreSQL serves data from `SELECT`s as if the inheriting tables' data is also
@@ -297,7 +299,7 @@ non-) that unions the new table.
297299

298300
## Examples
299301

300-
{{< warning >}}
302+
{{< important >}}
301303
Before creating a PostgreSQL source, you must set up logical replication in the
302304
upstream database. For step-by-step instructions, see the integration guide for
303305
your PostgreSQL service: [AlloyDB](/ingest-data/postgres-alloydb/),
@@ -306,7 +308,7 @@ your PostgreSQL service: [AlloyDB](/ingest-data/postgres-alloydb/),
306308
[Azure DB](/ingest-data/postgres-azure-db/),
307309
[Google Cloud SQL](/ingest-data/postgres-google-cloud-sql/),
308310
[Self-hosted](/ingest-data/postgres-self-hosted/).
309-
{{< /warning >}}
311+
{{< /important >}}
310312

311313
### Creating a connection
312314

@@ -431,6 +433,8 @@ CREATE SOURCE mz_source
431433

432434
### Handling errors and schema changes
433435

436+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
437+
434438
To handle upstream [schema changes](#schema-changes) or errored subsources, use
435439
the [`DROP SOURCE`](/sql/alter-source/#context) syntax to drop the affected
436440
subsource, and then [`ALTER SOURCE...ADD SUBSOURCE`](/sql/alter-source/) to add

doc/user/content/sql/create-source/sql-server.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,11 +123,13 @@ ingestion progress and debugging related issues, see [Troubleshooting](/ops/trou
123123

124124
## Known limitations
125125

126-
##### Schema changes
126+
### Schema changes
127+
128+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
127129

128130
{{% schema-changes %}}
129131

130-
##### Supported types
132+
### Supported types
131133

132134
Materialize natively supports the following SQL Server types:
133135

@@ -176,7 +178,7 @@ Columns with the specified types need to be excluded because [SQL Server does no
176178
the "before"](https://learn.microsoft.com/en-us/sql/relational-databases/system-tables/cdc-capture-instance-ct-transact-sql?view=sql-server-2017#large-object-data-types)
177179
value when said column is updated.
178180

179-
#### Timestamp Rounding
181+
### Timestamp Rounding
180182

181183
The `time`, `datetime2`, and `datetimeoffset` types in SQL Server have a default
182184
scale of 7 decimal places, or in other words a accuracy of 100 nanoseconds. But
@@ -201,10 +203,10 @@ SELECT * FROM my_timestamps;
201203

202204
## Examples
203205

204-
{{< warning >}}
206+
{{< important >}}
205207
Before creating a SQL Server source, you must enable Change Data Capture and
206208
`SNAPSHOT` transaction isolation in the upstream database.
207-
{{< /warning >}}
209+
{{</ important >}}
208210

209211
### Creating a connection
210212

@@ -305,6 +307,8 @@ CREATE SOURCE mz_source
305307

306308
### Handling errors and schema changes
307309

310+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
311+
308312
To handle upstream [schema changes](#schema-changes) or errored subsources, use
309313
the [`DROP SOURCE`](/sql/alter-source/#context) syntax to drop the affected
310314
subsource, and then [`ALTER SOURCE...ADD SUBSOURCE`](/sql/alter-source/) to add
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{{< note >}}
2+
3+
Work to more smoothly support ddl changes to upstream tables is currently in
4+
progress. The work introduces the ability to re-ingest the same upstream table
5+
under a new schema and switch over without downtime.
6+
7+
{{</ note >}}

0 commit comments

Comments
 (0)