Docs ingest data considerations (#32493)

kay-kim · web-flow · commit b7fe616aae9d · 2025-05-14T19:06:28.000-04:00
diff --git a/doc/user/content/ingest-data/mysql/_index.md b/doc/user/content/ingest-data/mysql/_index.md
@@ -49,3 +49,7 @@ MySQL hosted services.
 If there is a hosted service or MySQL distribution that is not listed above but
 you would like to use with Materialize, please submit a [feature request](https://github.com/MaterializeInc/materialize/discussions/new?category=feature-requests&labels=A-integration)
 or reach out in the Materialize [Community Slack](https://materialize.com/s/chat).
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/mysql/amazon-aurora.md b/doc/user/content/ingest-data/mysql/amazon-aurora.md
@@ -295,6 +295,10 @@ available (also for PostgreSQL)."
 
 {{% mysql-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% mysql-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/mysql/amazon-rds.md b/doc/user/content/ingest-data/mysql/amazon-rds.md
@@ -348,6 +348,10 @@ available (also for PostgreSQL)."
 
 {{% mysql-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% mysql-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/mysql/azure-db.md b/doc/user/content/ingest-data/mysql/azure-db.md
@@ -168,6 +168,10 @@ available(also for PostgreSQL)."
 
 {{% mysql-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% mysql-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/mysql/google-cloud-sql.md b/doc/user/content/ingest-data/mysql/google-cloud-sql.md
@@ -158,6 +158,10 @@ networking configuration, so start by selecting the relevant option.
 
 {{% mysql-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% mysql-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/mysql/self-hosted.md b/doc/user/content/ingest-data/mysql/self-hosted.md
@@ -160,6 +160,10 @@ available(also for PostgreSQL)."
 
 {{% mysql-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% mysql-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/mysql-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/_index.md b/doc/user/content/ingest-data/postgres/_index.md
@@ -53,3 +53,7 @@ but you would like to use with Materialize, please submit a [feature
 request](https://github.com/MaterializeInc/materialize/discussions/new?category=feature-requests&labels=A-integration)
 or reach out in the Materialize [Community
 Slack](https://materialize.com/s/chat).
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/alloydb.md b/doc/user/content/ingest-data/postgres/alloydb.md
@@ -156,6 +156,10 @@ Materialize to your AlloyDB instance and begin the data ingestion process.
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{< include-md file="shared-content/postgres-considerations.md" >}}
diff --git a/doc/user/content/ingest-data/postgres/amazon-aurora.md b/doc/user/content/ingest-data/postgres/amazon-aurora.md
@@ -548,6 +548,10 @@ password for the `materialize` PostgreSQL user you created [earlier](#2-create-a
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{< include-md file="shared-content/postgres-considerations.md" >}}
diff --git a/doc/user/content/ingest-data/postgres/amazon-rds.md b/doc/user/content/ingest-data/postgres/amazon-rds.md
@@ -617,6 +617,10 @@ start by selecting the relevant option.
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/azure-db.md b/doc/user/content/ingest-data/postgres/azure-db.md
@@ -297,6 +297,10 @@ created [earlier](#2-create-a-publication-and-a-replication-user):
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/cloud-sql.md b/doc/user/content/ingest-data/postgres/cloud-sql.md
@@ -151,6 +151,10 @@ start by selecting the relevant option.
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/neon.md b/doc/user/content/ingest-data/postgres/neon.md
@@ -306,6 +306,10 @@ ingesting data.
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/postgres/self-hosted.md b/doc/user/content/ingest-data/postgres/self-hosted.md
@@ -485,6 +485,10 @@ start by selecting the relevant option.
 
 {{% postgres-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% postgres-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/postgres-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/sql-server/_index.md b/doc/user/content/ingest-data/sql-server/_index.md
@@ -30,8 +30,14 @@ SQL Server Change Data Capture (CDC) in Materialize gives you the following bene
     a read-replica to build views on top of your SQL Server data that are
     efficiently maintained and always up-to-date.
 
-* **Supported versions:** Materialize supports replicating data from SQL Server 2016
-    or higher.
+## Supported versions
 
-* **Integration Guides**
-    * [Self-hosted SQL Server](/ingest-data/sql-server/self-hosted/)
+Materialize supports replicating data from SQL Server 2016 or higher.
+
+## Integration Guides
+
+- [Self-hosted SQL Server](/ingest-data/sql-server/self-hosted/)
+
+## Considerations
+
+{{% include-md file="shared-content/sql-server-considerations.md" %}}
diff --git a/doc/user/content/ingest-data/sql-server/self-hosted.md b/doc/user/content/ingest-data/sql-server/self-hosted.md
@@ -155,6 +155,10 @@ available(also for PostgreSQL)."
 
 {{% sql-server-direct/right-size-the-cluster %}}
 
-## Next steps
+## D. Explore your data
 
 {{% sql-server-direct/next-steps %}}
+
+## Considerations
+
+{{% include-md file="shared-content/sql-server-considerations.md" %}}
diff --git a/doc/user/content/sql/create-source/mysql.md b/doc/user/content/sql/create-source/mysql.md
@@ -206,71 +206,7 @@ debugging related issues, see [Troubleshooting](/ops/troubleshooting/).
 
 ## Known limitations
 
-### Schema changes
-
-{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
-
-{{% schema-changes %}}
-
-### Supported types
-
-Materialize natively supports the following MySQL types:
-
-<ul style="column-count: 3">
-<li><code>bigint</code></li>
-<li><code>binary</code></li>
-<li><code>bit</code></li>
-<li><code>blob</code></li>
-<li><code>boolean</code></li>
-<li><code>char</code></li>
-<li><code>date</code></li>
-<li><code>datetime</code></li>
-<li><code>decimal</code></li>
-<li><code>double</code></li>
-<li><code>float</code></li>
-<li><code>int</code></li>
-<li><code>json</code></li>
-<li><code>longblob</code></li>
-<li><code>longtext</code></li>
-<li><code>mediumblob</code></li>
-<li><code>mediumint</code></li>
-<li><code>mediumtext</code></li>
-<li><code>numeric</code></li>
-<li><code>real</code></li>
-<li><code>smallint</code></li>
-<li><code>text</code></li>
-<li><code>time</code></li>
-<li><code>timestamp</code></li>
-<li><code>tinyblob</code></li>
-<li><code>tinyint</code></li>
-<li><code>tinytext</code></li>
-<li><code>varbinary</code></li>
-<li><code>varchar</code></li>
-</ul>
-
-Replicating tables that contain **unsupported [data types](/sql/types/)** is
-possible via the [`TEXT COLUMNS` option](#handling-unsupported-types) for the
-following types:
-
-<ul style="column-count: 1">
-<li><code>enum</code></li>
-<li><code>year</code></li>
-</ul>
-
-The specified columns will be treated as `text`, and will thus not offer the
-expected MySQL type features. For any unsupported data types not listed above,
-use the [`EXCLUDE COLUMNS`](#excluding-columns) option.
-
-### Truncation
-
-Tables replicated into Materialize should not be truncated. If a table is
-truncated while replicated, the whole source becomes inaccessible and will not
-produce any data until it is recreated. Instead, remove all rows from a table
-using an unqualified `DELETE`.
-
-```mzsql
-DELETE FROM t;
-```
+{{< include-md file="shared-content/mysql-considerations.md" >}}
 
 ## Examples
 
diff --git a/doc/user/content/sql/create-source/postgres.md b/doc/user/content/sql/create-source/postgres.md
@@ -196,106 +196,7 @@ ingestion progress and debugging related issues, see [Troubleshooting](/ops/trou
 
 ## Known limitations
 
-### Schema changes
-
-{{< include-md file="shared-content/schema-changes-in-progress.md" >}}
-
-{{% schema-changes %}}
-
-### Publication membership
-
-PostgreSQL's logical replication API does not provide a signal when users remove
-tables from publications. Because of this, Materialize relies on periodic checks
-to determine if a table has been removed from a publication, at which time it
-generates an irrevocable error, preventing any values from being read from the
-table.
-
-However, it is possible to remove a table from a publication and then re-add it
-before Materialize notices that the table was removed. In this case, Materialize
-can no longer provide any consistency guarantees about the data we present from
-the table and, unfortunately, is wholly unaware that this occurred.
-
-To mitigate this issue, if you need to drop and re-add a table to a publication,
-ensure that you remove the table/subsource from the source _before_ re-adding it
-using the [`DROP SOURCE`](/sql/drop-source/) command.
-
-### Supported types
-
-Materialize natively supports the following PostgreSQL types (including the
-array type for each of the types):
-
-<ul style="column-count: 3">
-<li><code>bool</code></li>
-<li><code>bpchar</code></li>
-<li><code>bytea</code></li>
-<li><code>char</code></li>
-<li><code>date</code></li>
-<li><code>daterange</code></li>
-<li><code>float4</code></li>
-<li><code>float8</code></li>
-<li><code>int2</code></li>
-<li><code>int2vector</code></li>
-<li><code>int4</code></li>
-<li><code>int4range</code></li>
-<li><code>int8</code></li>
-<li><code>int8range</code></li>
-<li><code>interval</code></li>
-<li><code>json</code></li>
-<li><code>jsonb</code></li>
-<li><code>numeric</code></li>
-<li><code>numrange</code></li>
-<li><code>oid</code></li>
-<li><code>text</code></li>
-<li><code>time</code></li>
-<li><code>timestamp</code></li>
-<li><code>timestamptz</code></li>
-<li><code>tsrange</code></li>
-<li><code>tstzrange</code></li>
-<li><code>uuid</code></li>
-<li><code>varchar</code></li>
-</ul>
-
-Replicating tables that contain **unsupported [data types](/sql/types/)** is
-possible via the `TEXT COLUMNS` option. The specified columns will be treated
-as `text`, and will thus not offer the expected PostgreSQL type features. For
-example:
-
-* [`enum`]: the implicit ordering of the original PostgreSQL `enum` type is not
-  preserved, as Materialize will sort values as `text`.
-
-* [`money`]: the resulting `text` value cannot be cast back to e.g. `numeric`,
-  since PostgreSQL adds typical currency formatting to the output.
-
-### Truncation
-
-Tables replicated into Materialize should not be truncated. If a table is
-truncated while replicated, the whole source becomes inaccessible and will not
-produce any data until it is recreated. Instead, remove all rows from a table
-using an unqualified `DELETE`.
-
-```mzsql
-DELETE FROM t;
-```
-
-### Inherited tables
-
-When using [PostgreSQL table inheritance](https://www.postgresql.org/docs/current/tutorial-inheritance.html),
-PostgreSQL serves data from `SELECT`s as if the inheriting tables' data is also
-present in the inherited table. However, both PostgreSQL's logical replication
-and `COPY` only present data written to the tables themselves, i.e. the
-inheriting data is _not_ treated as part of the inherited table.
-
-PostgreSQL sources use logical replication and `COPY` to ingest table data, so
-inheriting tables' data will only be ingested as part of the inheriting table,
-i.e. in Materialize, the data will not be returned when serving `SELECT`s from
-the inherited table.
-
-You can mimic PostgreSQL's `SELECT` behavior with inherited tables by creating a
-materialized view that unions data from the inherited and inheriting tables
-(using `UNION ALL`). However, if new tables inherit from the table, data from
-the inheriting tables will not be available in the view. You will need to add
-the inheriting tables via `ADD SUBSOURCE` and create a new view (materialized or
-non-) that unions the new table.
+{{% include-md file="shared-content/postgres-considerations.md" %}}
 
 ## Examples
 
diff --git a/doc/user/content/sql/create-source/sql-server.md b/doc/user/content/sql/create-source/sql-server.md
diff --git a/doc/user/shared-content/mysql-considerations.md b/doc/user/shared-content/mysql-considerations.md
diff --git a/doc/user/shared-content/postgres-considerations.md b/doc/user/shared-content/postgres-considerations.md
diff --git a/doc/user/shared-content/sql-server-considerations.md b/doc/user/shared-content/sql-server-considerations.md
