Skip to content

Commit 582d56b

Browse files
author
风离
committed
[docs] Add docs for flink materialized table
1 parent 8c4004c commit 582d56b

File tree

2 files changed

+106
-0
lines changed

2 files changed

+106
-0
lines changed

website/docs/engine-flink/ddl.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,3 +273,106 @@ ALTER TABLE my_multi_fields_part_log_table DROP PARTITION (dt = '2025-03-05', na
273273
```
274274

275275
For more details, refer to the [Flink ALTER TABLE(DROP)](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/alter/#drop) documentation.
276+
277+
## Materialized Table
278+
### Overview
279+
Flink Materialized Table is a new table type introduced in Flink SQL, aimed at simplifying both batch and stream data pipelines, providing a consistent development experience. By specifying data freshness and query when creating Materialized Table, the engine automatically derives the schema for the materialized table and creates corresponding data refresh pipeline to achieve the specified freshness.
280+
For more details, refer to the [Flink Materialized Table](https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/dev/table/materialized-table/overview/).
281+
282+
Starting from Fluss version 0.8, Flink Materialized Table is now supported, which can significantly reduce the cost of building real-time data pipelines with Flink + Fluss. Materialized tables in Fluss are implemented as regular Fluss tables with special metadata to identify them as materialized tables.
283+
284+
### Create Materialized Table
285+
286+
Materialized tables are created using the `CREATE MATERIALIZED TABLE` statement with a freshness interval and a query definition:
287+
288+
```sql title="Flink SQL"
289+
CREATE MATERIALIZED TABLE shop_summary
290+
FRESHNESS = INTERVAL '5' SECOND
291+
AS SELECT
292+
DATE_FORMAT(order_time, 'yyyy-MM-dd') AS order_date,
293+
shop_id,
294+
COUNT(*) AS order_count,
295+
SUM(amount) AS total_amount
296+
FROM orders
297+
GROUP BY DATE_FORMAT(order_time, 'yyyy-MM-dd'), shop_id;
298+
```
299+
300+
#### Supported Refresh Mode
301+
302+
Fluss currently supports **CONTINUOUS** refresh mode for materialized table, which means the materialized table is continuously refreshed. The **FULL** refresh mode will be supported in future releases.
303+
304+
#### Schema Definition
305+
306+
The schema of a materialized table is automatically inferred from the query definition. You cannot manually specify column names and types - they are derived from the SELECT statement.
307+
308+
```sql title="Flink SQL"
309+
-- The schema will be automatically inferred as:
310+
-- order_date: STRING
311+
-- shop_id: BIGINT
312+
-- order_count: BIGINT
313+
-- total_amount: BIGINT
314+
CREATE MATERIALIZED TABLE daily_sales
315+
FRESHNESS = INTERVAL '1' MINUTE
316+
AS SELECT
317+
DATE_FORMAT(created_at, 'yyyy-MM-dd') AS order_date,
318+
shop_id,
319+
COUNT(*) AS order_count,
320+
SUM(amount) AS total_amount
321+
FROM sales_events
322+
GROUP BY DATE_FORMAT(created_at, 'yyyy-MM-dd'), shop_id;
323+
```
324+
325+
### Alter Materialized Table
326+
327+
You can suspend and resume materialized tables to control their refresh behavior:
328+
329+
#### Suspend Materialized Table
330+
331+
```sql title="Flink SQL"
332+
ALTER MATERIALIZED TABLE shop_summary SUSPEND;
333+
```
334+
335+
This stops the automatic refresh of the materialized table and saves the current state.
336+
337+
#### Resume Materialized Table
338+
339+
```sql title="Flink SQL"
340+
ALTER MATERIALIZED TABLE shop_summary RESUME;
341+
```
342+
343+
This resumes the automatic refresh of the materialized table from the last saved state.
344+
345+
### Drop Materialized Table
346+
347+
To delete a materialized table:
348+
349+
```sql title="Flink SQL"
350+
DROP MATERIALIZED TABLE shop_summary;
351+
```
352+
353+
This will drop the materialized table and stop the background refresh job.
354+
355+
### Materialized Table Options
356+
357+
Materialized tables support the same table options as regular Fluss tables, including partitioning and bucketing:
358+
359+
```sql title="Flink SQL"
360+
CREATE MATERIALIZED TABLE partitioned_summary
361+
FRESHNESS = INTERVAL '10' SECOND
362+
AS SELECT
363+
dt,
364+
shop_id,
365+
COUNT(*) AS order_count
366+
FROM orders
367+
GROUP BY dt, shop_id
368+
PARTITIONED BY (dt)
369+
WITH (
370+
'bucket.num' = '4'
371+
);
372+
```
373+
374+
### Limitations
375+
376+
- Only continuous refresh mode is supported
377+
- Schema is automatically derived from the query
378+
- Materialized tables are stored as regular Fluss tables with special metadata

website/docs/engine-flink/getting-started.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ For Flink's Table API, Fluss supports the following features:
2828
| [SQL create table](ddl.md#create-table) | ✔️ | |
2929
| [SQL create table like](ddl.md#create-table-like) | ✔️ | |
3030
| [SQL drop table](ddl.md#drop-table) | ✔️ | |
31+
| [SQL create materialized table](ddl.md#materialized-table) | ✔️ | Continuous refresh mode only |
32+
| [SQL alter materialized table](ddl.md#alter-materialized-table) | ✔️ | Suspend/Resume support |
33+
| [SQL drop materialized table](ddl.md#drop-materialized-table) | ✔️ | |
3134
| [SQL show partitions](ddl.md#show-partitions) | ✔️ | |
3235
| [SQL add partition](ddl.md#add-partition) | ✔️ | |
3336
| [SQL drop partition](ddl.md#drop-partition) | ✔️ | |

0 commit comments

Comments
 (0)