Skip to content

Commit 6aa350e

Browse files
committed
docs: add in-progress note w.r.t. supporting upstream table schema ch… (MaterializeInc#32436)
…anges
1 parent 27dbb54 commit 6aa350e

File tree

3 files changed

+27
-12
lines changed

3 files changed

+27
-12
lines changed

doc/user/content/sql/create-source/mysql.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -206,11 +206,13 @@ debugging related issues, see [Troubleshooting](/ops/troubleshooting/).
206206

207207
## Known limitations
208208

209-
##### Schema changes
209+
### Schema changes
210+
211+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
210212

211213
{{% schema-changes %}}
212214

213-
##### Supported types
215+
### Supported types
214216

215217
Materialize natively supports the following MySQL types:
216218

@@ -258,7 +260,7 @@ The specified columns will be treated as `text`, and will thus not offer the
258260
expected MySQL type features. For any unsupported data types not listed above,
259261
use the [`EXCLUDE COLUMNS`](#excluding-columns) option.
260262

261-
##### Truncation
263+
### Truncation
262264

263265
Tables replicated into Materialize should not be truncated. If a table is
264266
truncated while replicated, the whole source becomes inaccessible and will not
@@ -271,15 +273,15 @@ DELETE FROM t;
271273

272274
## Examples
273275

274-
{{< warning >}}
276+
{{< important >}}
275277
Before creating a MySQL source, you must enable GTID-based binlog replication in the
276278
upstream database. For step-by-step instructions, see the integration guide for
277279
your MySQL service: [Amazon RDS](/ingest-data/mysql/amazon-rds/),
278280
[Amazon Aurora](/ingest-data/mysql/amazon-aurora/),
279281
[Azure DB](/ingest-data/mysql/azure-db/),
280282
[Google Cloud SQL](/ingest-data/mysql/google-cloud-sql/),
281283
[Self-hosted](/ingest-data/mysql/self-hosted/).
282-
{{< /warning >}}
284+
{{< /important >}}
283285

284286
### Creating a connection
285287

@@ -381,6 +383,8 @@ CREATE SOURCE mz_source
381383

382384
### Handling errors and schema changes
383385

386+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
387+
384388
To handle upstream [schema changes](#schema-changes) or errored subsources, use
385389
the [`DROP SOURCE`](/sql/alter-source/#context) syntax to drop the affected
386390
subsource, and then [`ALTER SOURCE...ADD SUBSOURCE`](/sql/alter-source/) to add

doc/user/content/sql/create-source/postgres.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -196,11 +196,13 @@ ingestion progress and debugging related issues, see [Troubleshooting](/ops/trou
196196

197197
## Known limitations
198198

199-
##### Schema changes
199+
### Schema changes
200+
201+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
200202

201203
{{% schema-changes %}}
202204

203-
#### Publication membership
205+
### Publication membership
204206

205207
PostgreSQL's logical replication API does not provide a signal when users remove
206208
tables from publications. Because of this, Materialize relies on periodic checks
@@ -217,7 +219,7 @@ To mitigate this issue, if you need to drop and re-add a table to a publication,
217219
ensure that you remove the table/subsource from the source _before_ re-adding it
218220
using the [`DROP SOURCE`](/sql/drop-source/) command.
219221

220-
##### Supported types
222+
### Supported types
221223

222224
Materialize natively supports the following PostgreSQL types (including the
223225
array type for each of the types):
@@ -264,7 +266,7 @@ example:
264266
* [`money`]: the resulting `text` value cannot be cast back to e.g. `numeric`,
265267
since PostgreSQL adds typical currency formatting to the output.
266268

267-
##### Truncation
269+
### Truncation
268270

269271
Tables replicated into Materialize should not be truncated. If a table is
270272
truncated while replicated, the whole source becomes inaccessible and will not
@@ -275,7 +277,7 @@ using an unqualified `DELETE`.
275277
DELETE FROM t;
276278
```
277279

278-
##### Inherited tables
280+
### Inherited tables
279281

280282
When using [PostgreSQL table inheritance](https://www.postgresql.org/docs/current/tutorial-inheritance.html),
281283
PostgreSQL serves data from `SELECT`s as if the inheriting tables' data is also
@@ -297,7 +299,7 @@ non-) that unions the new table.
297299

298300
## Examples
299301

300-
{{< warning >}}
302+
{{< important >}}
301303
Before creating a PostgreSQL source, you must set up logical replication in the
302304
upstream database. For step-by-step instructions, see the integration guide for
303305
your PostgreSQL service: [AlloyDB](/ingest-data/postgres-alloydb/),
@@ -306,7 +308,7 @@ your PostgreSQL service: [AlloyDB](/ingest-data/postgres-alloydb/),
306308
[Azure DB](/ingest-data/postgres-azure-db/),
307309
[Google Cloud SQL](/ingest-data/postgres-google-cloud-sql/),
308310
[Self-hosted](/ingest-data/postgres-self-hosted/).
309-
{{< /warning >}}
311+
{{< /important >}}
310312

311313
### Creating a connection
312314

@@ -400,6 +402,8 @@ CREATE SOURCE mz_source
400402

401403
### Handling errors and schema changes
402404

405+
{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
406+
403407
To handle upstream [schema changes](#schema-changes) or errored subsources, use
404408
the [`DROP SOURCE`](/sql/alter-source/#context) syntax to drop the affected
405409
subsource, and then [`ALTER SOURCE...ADD SUBSOURCE`](/sql/alter-source/) to add
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{{< note >}}
2+
3+
Work to more smoothly support ddl changes to upstream tables is currently in
4+
progress. The work introduces the ability to re-ingest the same upstream table
5+
under a new schema and switch over without downtime.
6+
7+
{{</ note >}}

0 commit comments

Comments
 (0)