Skip to content

Commit 54ed898

Browse files
kay-kimkay-kim
committed
Updates + start create table examples
1 parent e9ea25a commit 54ed898

14 files changed

+405
-253
lines changed

‎doc/user/content/sql/create-table.md‎

Lines changed: 155 additions & 151 deletions
Large diffs are not rendered by default.

‎doc/user/content/sql/identifiers.md‎

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -8,28 +8,33 @@ menu:
88
weight: 125
99
---
1010

11-
In Materialize, identifiers are used to refer to columns and database objects
12-
like sources, views, and indexes.
11+
In Materialize, identifiers are names used to refer to columns and database
12+
objects like sources, views, and indexes.
1313

1414
## Naming restrictions
1515

16-
- The first character of an identifier must be an ASCII letter
17-
(`a`-`z` and `A`-`Z`), an underscore (`_`), or any non-ASCII character.
16+
Materialize has the following naming restrictions for identifiers:
1817

19-
- The remaining characters of an identifier must be ASCII letters
20-
(`a`-`z` and `A`-`Z`), ASCII digits (`0`-`9`), underscores (`_`),
21-
dollar signs (`$`), or any non-ASCII characters.
18+
- The first character must be either an ASCII letter (`a`-`z` and `A`-`Z`), an
19+
underscore (`_`), or any non-ASCII character.
2220

23-
You can circumvent any of the above rules by double-quoting the identifier,
24-
e.g. `"123_source"` or `"fun_source_@"`. All characters inside a quoted
25-
identifier are taken literally, except that double-quotes must be escaped by
26-
writing two adjacent double-quotes, as in `"includes""quote"`.
21+
- The remaining characters can be ASCII letters (`a`-`z` and `A`-`Z`), ASCII
22+
digits (`0`-`9`), underscores (`_`), dollar signs (`$`), or any non-ASCII
23+
characters.
2724

28-
Additionally, the identifiers `"."` and `".."` are not permitted.
25+
To override these restrictions, you can enclose the identifier in double quotes;
26+
e.g., `"123_source"` or `"fun_source_@"`. When enclosed in double-quotes, all
27+
characters in the identifier are interpreted literally with the exception double
28+
quotes. To include double quotes within a double-quoted identifier, you can
29+
escape them by writing two adjacent double-quotes, as in `"includes""quote"`.
30+
31+
{{< note >}}
32+
The identifiers `"."` and `".."` are not allowed.
33+
{{</ note >}}
2934

3035
## Case sensitivity
3136

32-
Materialize performs case folding (the caseless comparison of text) for identifiers, which means that identifiers are effectively case-insensitive (`foo` is the same as `FOO` is the same as `fOo`). This can cause issues when column names come from data sources which do support case-sensitive names, such as Avro-formatted sources or CSV headers.
37+
Materialize performs case folding (the caseless comparison of text) for identifiers, which means that identifiers are effectively case-insensitive (`foo` is the same as `FOO` is the same as `fOo`). This can cause issues when column names come from data sources which do support case-sensitive names, such as Avro-formatted sources.
3338

3439
To avoid conflicts, double-quote all field names (`"field_name"`) when working with case-sensitive sources.
3540

‎doc/user/data/examples/create-table/example_postgres_table.yml‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
- name: "create-table"
2+
description: |
3+
To create new **read-only** tables from a source, in the `CREATE TABLE FROM
4+
SOURCE` statement, specify the table in the publication using the
5+
`REFERENCE` field.
6+
7+
The following example creates **read-only** tables `items` and `orders` that
8+
are populated from the corresponding `items` and `orders` tables from the
9+
PostgreSQL source `mz_source`.
10+
11+
{{< note >}}
12+
13+
- Although the example creates the tables with the same names as the
14+
upstream tables, the tables in Materialize can have names that differ from
15+
the referenced table names.
16+
17+
- For supported PostgreSQL data types, refer to [supported
18+
types](#upstream-sources-and-supported-data-types).
19+
20+
- You can create multiple tables that reference the same upstream table.
21+
22+
{{< /note >}}
23+
24+
code: |
25+
/* This example assumes:
26+
- In the upstream PostgreSQL, you have defined:
27+
- replication user and password with the appropriate access.
28+
- a publication named `mz_source` for the `items` and `orders` tables.
29+
- In Materialize,
30+
- You have defined the connection to the upstream PostgreSQL.
31+
- You have used the connection to create a source; e.g.,
32+
CREATE SECRET pgpass AS '<replication user password>'; -- substitute
33+
CREATE CONNECTION pg TO POSTGRES (
34+
HOST '<hostname>', -- substitute
35+
DATABASE <db>, -- substitute
36+
USER <replication user>, -- substitute
37+
PASSWORD SECRET pgpass
38+
-- [, AWS Privatelink <svc>| SSH TUNNEL <ssh_connection> ]
39+
);
40+
41+
CREATE SOURCE mz_source
42+
FROM POSTGRES CONNECTION pg (
43+
PUBLICATION 'mz_source' -- substitute
44+
);
45+
*/
46+
47+
CREATE TABLE items
48+
FROM SOURCE mz_source(REFERENCE items)
49+
;
50+
CREATE TABLE orders
51+
FROM SOURCE mz_source(REFERENCE orders)
52+
;
53+
54+
- name: "show-tables"
55+
description: |
56+
To verify that the table has been created, you can run [`SHOW
57+
TABLES`](/sql/show-tables/) to list all tables in the current [schema](/sql/namespaces/#namespace-hierarchy):
58+
code: |
59+
SHOW TABLES;
60+
results: |
61+
The results should include the tables `items` and `orders`:
62+
63+
```hc {hl_lines="3-4"}
64+
| name | comment |
65+
| ----------- | ------- |
66+
| items | |
67+
| orders | |
68+
```
69+
70+
- name: "show-columns"
71+
description: |
72+
Inspect the table columns using the [`SHOW COLUMNS`](/sql/show-columns/) command:
73+
code: |
74+
SHOW COLUMNS FROM items;
75+
SHOW COLUMNS FROM orders;
76+
results: |
77+
The results should display information on the table columns. The types
78+
should match those from the upstream table with the exception of those that
79+
were explicitly converted to `text` in the `WITH` clause.
80+
81+
For the list of supported PostgreSQL data types, refer to [supported
82+
types](#upstream-sources-and-supported-data-types).
83+
84+
- name: "read-from-table"
85+
code: |
86+
SELECT * FROM items;
87+
88+
- name: "create-view-from-tables"
89+
code: |
90+
CREATE VIEW orders_view AS
91+
SELECT o.*,i.price,o.quantity * i.price as subtotal
92+
FROM orders as o
93+
JOIN items as i
94+
ON o.item = i.item;

‎doc/user/data/examples/create-table/example_user_defined_table.yml‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
- name: "create-table"
2+
description: |
3+
The following example uses `CREATE TABLE` to create a new table `mytable`
4+
with two columns `a` (of type `int`) and `b` (of type `text` and not
5+
nullable):
6+
code: |
7+
CREATE TABLE mytable (a int, b text NOT NULL);
8+
9+
- name: "show-tables"
10+
description: |
11+
To verify that the table has been created, you can run [`SHOW
12+
TABLES`](/sql/show-tables/) to list all tables in the current [schema](/sql/namespaces/#namespace-hierarchy):
13+
code: |
14+
SHOW TABLES;
15+
results: |
16+
The results should include the table `mytable`:
17+
18+
```hc {hl_lines="3"}
19+
| name | comment |
20+
| ---------- | ------- |
21+
| mytable | |
22+
```
23+
24+
- name: "show-columns"
25+
description: |
26+
Inspect the table columns using the [`SHOW COLUMNS`](/sql/show-columns/) command:
27+
code: |
28+
SHOW COLUMNS FROM mytable;
29+
results: |
30+
The results should display information on columns `a` and `b`; these should
31+
match the specification in your `CREATE TABLE` statement:
32+
33+
```none
34+
| name | nullable | type | comment |
35+
| ---- | -------- | ------- | ------- |
36+
| a | true | integer | |
37+
| b | false | text | |
38+
```
39+
40+
- name: "write-to-table"
41+
description: |
42+
The following example uses [`INSERT`](/sql/insert/) to write two rows to the table:
43+
code: |
44+
INSERT INTO mytable VALUES
45+
(1, 'hello'),
46+
(2, 'goodbye')
47+
;
48+
49+
- name: "read-from-table"
50+
description: |
51+
The following example uses [`SELECT`](/sql/select/) to read all rows from the table:
52+
code: |
53+
SELECT * FROM mytable;
54+
results: |
55+
The results should display the two rows inserted:
56+
57+
```none
58+
| a | b |
59+
| - | ------- |
60+
| 1 | hello |
61+
| 2 | goodbye |
62+
```

‎doc/user/data/syntax_options/create_table/create_table_options_source_populated_db.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,6 @@ rows:
4040
4141
| Option | Description |
4242
|--------|-------------|
43-
| <a name="text-columns"></a>`TEXT COLUMNS (<fq_column_name> [, ...])` | *Optional.* If specified, decode data as `text` for the listed column(s), such as for unsupported data types. Use fully qualified column names. See also [supported types](#supported-db-source-types). |
44-
| <a name="exclude-columns"></a>`EXCLUDE COLUMNS (<fq_column_name> [, ...])` | *Optional.* If specified, exclude the listed column(s) from the table. Use fully qualified column names. |
45-
| <a name="partition-by"></a>`PARTITION BY (<column> [, ...])` | {{< include-md file="shared-content/partition-by-option-description.md" >}} |
43+
| <a name="text-columns"></a>`TEXT COLUMNS (<column_name> [, ...])` | *Optional.* If specified, decode data as `text` for the listed column(s), such as for **certain** unsupported data types. See also [supported types](#upstream-sources-and-supported-data-types). **Available for PostgreSQL and MySQL source only**. |
44+
| <a name="exclude-columns"></a>`EXCLUDE COLUMNS (<column_name> [, ...])` | *Optional.* If specified, exclude the listed column(s) from the table. **Available for MySQL and SQL Server source only**. |
45+
| <a name="partition-by"></a>`PARTITION BY (<column_name> [, ...])` | {{< include-md file="shared-content/partition-by-option-description.md" >}} |

‎doc/user/layouts/shortcodes/include-example.html‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{{- $pathArray := split (lower (.Get "file")) "/" -}}
2+
{{- $data := .Site.Data -}}
3+
{{- range $pathArray }}
4+
{{- $data = index $data . -}}
5+
{{- end }}
6+
{{- $example := .Get "example" -}}
7+
{{- range $data }}
8+
{{- if eq .name $example }}
9+
{{ .description | markdownify }}
10+
11+
```mzsql
12+
{{ .code | safeHTML }}
13+
```
14+
15+
{{- if .results }}
16+
{{ .results | markdownify }}
17+
{{- end }}
18+
{{- end }}
19+
{{- end }}

‎doc/user/layouts/shortcodes/json-parser.html‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,3 @@
1-
Manually parsing JSON-formatted data in SQL can be tedious. 🫠 You can use the
2-
widget below to <b>automatically</b> turn a sample JSON payload into a parsing
3-
view with the individual fields mapped to columns.
4-
51
<div class="json_widget">
62
<div class="json">
73
<textarea title="JSON sample" id="json_sample" placeholder="JSON sample">
@@ -13,9 +9,12 @@
139
</div>
1410
<span class="input_container">
1511
<span class="input_container-text">
16-
<input title="View Name" id="view_name" placeholder="View Name" value="my_view">
12+
<label for="relation">Enter your relation name</label>
1713
<input title="Relation Name" id="source_name" placeholder="Relation Name" value="my_source">
14+
<label for="column">Enter your JSON column name</label>
1815
<input title="JSON Column Name" id="column_name" placeholder="JSON Column Name" value="json_column">
16+
<label for="view">Enter your view/materialized view name</label>
17+
<input title="View Name" id="view_name" placeholder="View Name" value="my_view">
1918
</span>
2019
<fieldset title="Target object type" class="input_container-radio">
2120
<legend>Target object type</legend>

‎doc/user/shared-content/create-table-supported-types.md‎

Lines changed: 3 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -3,59 +3,23 @@
33

44
{{< include-md file="shared-content/mysql-supported-types.md" >}}
55

6-
Replicating tables that contain **unsupported data types** is
7-
possible via the [`TEXT COLUMNS` option](#text-columns) for the
8-
following types:
9-
10-
<ul style="column-count: 1">
11-
<li><code>enum</code></li>
12-
<li><code>year</code></li>
13-
</ul>
14-
15-
The specified columns will be treated as `text`, and will thus not offer the
16-
expected MySQL type features. For any unsupported data types not listed above,
17-
use the [`EXCLUDE COLUMNS`](#exclude-columns) option.
6+
{{< include-md file="shared-content/mysql-unsupported-type-handling.md" >}}
187

198
{{</ tab >}}
209

2110
{{< tab "Supported PostgreSQL types">}}
2211

2312
{{< include-md file="shared-content/postgres-supported-types.md" >}}
2413

25-
Replicating tables that contain **unsupported data types** is possible via the
26-
[`TEXT COLUMNS` option](#text-columns). When decoded as `text`, the specified
27-
columns will not have the expected PostgreSQL type features. For example:
28-
29-
* [`enum`]: When decoded as `text`, the resulting `text` values will
30-
not observe the implicit ordering of the original PostgreSQL `enum`; instead,
31-
Materialize will sort the values as `text`.
32-
33-
* [`money`]: When decoded as `text`, the resulting `text` value
34-
cannot be cast back to `numeric` since PostgreSQL adds typical currency
35-
formatting to the output.
36-
37-
[`enum`]: https://www.postgresql.org/docs/current/datatype-enum.html
38-
[`money`]: https://www.postgresql.org/docs/current/datatype-money.html
14+
{{< include-md file="shared-content/postgres-unsupported-type-handling.md" >}}
3915

4016
{{</ tab >}}
4117

4218
{{< tab "Supported SQL Server types">}}
4319

4420
{{< include-md file="shared-content/sql-server-supported-types.md" >}}
4521

46-
Replicating tables that contain **unsupported data types** is possible via the
47-
[`EXCLUDE COLUMNS`
48-
option](#exclude-columns) for the
49-
following types:
50-
51-
<ul style="column-count: 3">
52-
<li><code>text</code></li>
53-
<li><code>ntext</code></li>
54-
<li><code>image</code></li>
55-
<li><code>varchar(max)</code></li>
56-
<li><code>nvarchar(max)</code></li>
57-
<li><code>varbinary(max)</code></li>
58-
</ul>
22+
{{< include-md file="shared-content/sql-server-unsupported-type-handling.md" >}}
5923

6024
**Timestamp rounding**
6125

‎doc/user/shared-content/mysql-considerations.md‎

Lines changed: 1 addition & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,20 +8,7 @@
88

99
{{< include-md file="shared-content/mysql-supported-types.md" >}}
1010

11-
Replicating tables that contain **unsupported data types** is possible via the
12-
[`TEXT COLUMNS`
13-
option](/sql/create-table/#syntax) for the
14-
following types:
15-
16-
<ul style="column-count: 1">
17-
<li><code>enum</code></li>
18-
<li><code>year</code></li>
19-
</ul>
20-
21-
The specified columns will be treated as `text`, and will thus not offer the
22-
expected MySQL type features. For any unsupported data types not listed above,
23-
use the [`EXCLUDE
24-
COLUMNS`](/sql/create-table/#syntax) option.
11+
{{< include-md file="shared-content/mysql-unsupported-type-handling.md" >}}
2512

2613
### Truncation
2714

‎doc/user/shared-content/mysql-unsupported-type-handling.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Replicating tables that contain unsupported data types is possible via:
2+
3+
- the [`TEXT
4+
COLUMNS`](/sql/create-table/#tab-source-populated-tables-via-db-connector) for
5+
the following unsupported types:
6+
7+
- `enum`
8+
- `year`
9+
10+
The specified columns will be treated as `text` and will not offer the
11+
original MySQL type features.
12+
13+
- the [`EXCLUDE
14+
COLUMNS`](/sql/create-table/#tab-source-populated-tables-via-db-connector)
15+
option for any other unsupported data types.

0 commit comments

Comments
 (0)