You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/table-design/table-types/pk-table/merge-engines/versioned.md
+22-18Lines changed: 22 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,13 +5,15 @@ sidebar_position: 3
5
5
6
6
# Versioned Merge Engine
7
7
8
-
By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.
8
+
By setting `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than or equal to the stored value. If it is less than or null, no update will be made.
9
+
This feature is particularly valuable for replacing [Deduplication](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/queries/deduplication/) transformations in streaming computations, reducing complexity and improving overall efficiency.
9
10
10
11
:::note
11
12
When using `versioned` merge engine, there are the following limits:
12
-
-`UPDATE` and `DELETE` statements are not supported
13
-
- Partial update is not supported
14
-
:::
13
+
-`UPDATE` and `DELETE` statements are not supported.
14
+
- Partial update is not supported.
15
+
-`UPDATE_BEFORE` and `DELETE` changelog events are ignored automatically.
16
+
:::
15
17
16
18
## Versioned Merge Column
17
19
@@ -23,24 +25,24 @@ The versioned merge column supports the following data types.
23
25
- TIMESTAMP(p)
24
26
- TIMESTAMP_LTZ
25
27
- TIMESTAMP_LTZ(p)
26
-
:::
28
+
:::
27
29
28
30
example:
29
-
```sql title="versioned"
31
+
```sql title="Flink SQL"
30
32
31
-
createtablemerge_engine_with_version (
32
-
a intnot nullprimary key not enforced,
33
-
b string,
34
-
ts bigint
35
-
) with(
33
+
CREATETABLEVERSIONED (
34
+
a INTNOT NULLPRIMARY KEY NOT ENFORCED,
35
+
b STRING,
36
+
ts BIGINT
37
+
) WITH (
36
38
'table.merge-engine'='versioned',
37
39
'table.merge-engine.versioned.ver-column'='ts'
38
40
);
39
-
insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v1', 1000);
41
+
INSERT INTO VERSIONED (a, b, ts) VALUES (1, 'v1', 1000);
40
42
41
43
-- insert data with ts < 1000, no update will be made
42
-
insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 999);
43
-
select*from merge_engine_with_version;
44
+
INSERT INTO VERSIONED (a, b, ts) VALUES (1, 'v2', 999);
45
+
SELECT*FROM VERSIONED;
44
46
-- Output
45
47
-- +---+-----+------+
46
48
-- | a | b | ts |
@@ -50,21 +52,23 @@ select * from merge_engine_with_version;
50
52
51
53
52
54
-- insert data with ts > 1000, update will be made
53
-
insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 2000);
54
-
select*from merge_engine_with_version;
55
+
INSERT INTO VERSIONED (a, b, ts) VALUES (1, 'v2', 2000);
56
+
SELECT*FROM VERSIONED;
55
57
-- Output
56
58
-- +---+-----+------+
57
59
-- | a | b | ts |
58
60
-- +---+-----+------+
59
61
-- | 1 | v2 | 2000 |
62
+
-- +---+-----+------+
60
63
61
64
-- insert data with ts = null, no update will be made
62
-
nsert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', null);
63
-
select*from merge_engine_with_version;
65
+
INSERT INTO VERSIONED (a, b, ts) VALUES (1, 'v2', null);
0 commit comments