Add document about versioned merge engine #523

sunxiaojian · 2025-03-04T02:54:23Z

Purpose

Linked issue: close #459

Tests

N/A

API and Format

N/A

Documentation

N/A

sunxiaojian · 2025-03-04T02:56:20Z

@wuchong @luoyuxia PTAL， thanks

caozhen1937 · 2025-03-04T06:43:58Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+-- | 1 | v2  | 2000 |
+
+-- insert data with ts = null, no update will be made
+nsert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', null);


typo: nsert -> insert.

caozhen1937 · 2025-03-04T06:46:33Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+-- | a | b   | ts   |
+-- +---+-----+------+
+-- | 1 | v2  | 2000 |
+


Maybe add -- +---+-----+------+ in table end.

caozhen1937 · 2025-03-04T06:47:22Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+-- | a | b   | ts   |
+-- +---+-----+------+
+-- | 1 | v2  | 2000 |
+


same as above.

luoyuxia

@sunxiaojian Thanks for the pr. PTAL

luoyuxia · 2025-03-04T06:55:36Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+  :::
+
+example:
+```sql title="versioned"


Suggested change

```sql title="versioned"

```sql title="Flink SQL"

luoyuxia · 2025-03-04T06:56:34Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+    'table.merge-engine' = 'versioned',
+    'table.merge-engine.versioned.ver-column' = 'ts'
+);
+insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v1', 1000);


nit:

Suggested change

insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v1', 1000);

insert into merge_engine_with_version (a, b, ts ) VALUES (1, 'v1', 1000);

@luoyuxia Whether to remove the space after ts?
(a, b, ts ) -> (a, b, ts)

Haven't noticed that

luoyuxia · 2025-03-04T06:56:56Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v1', 1000);
+
+-- insert data with ts < 1000, no update will be made
+insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 999);


nit:

Suggested change

insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 999);

insert into merge_engine_with_version (a, b, ts ) VALUES (1, 'v2', 999);

luoyuxia · 2025-03-04T06:57:11Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+
+
+-- insert data with ts > 1000, update will be made
+insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 2000);


nit

Suggested change

insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 2000);

insert into merge_engine_with_version (a, b, ts ) VALUES (1, 'v2', 2000);

luoyuxia · 2025-03-04T06:57:52Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+-- | a | b   | ts   |
+-- +---+-----+------+
+-- | 1 | v2  | 2000 |
+


nit: add the bottom line to align with the style of above content

-- +---+-----+------+ -- | a | b | ts | -- +---+-----+------+ -- | 1 | v2 | 2000 | -- +---+-----+------+

luoyuxia · 2025-03-04T07:01:14Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+When using `versioned` merge engine, there are the following limits:
+- `UPDATE` and `DELETE` statements are not supported
+- Partial update is not supported
+  :::


Suggested change

:::

:::

luoyuxia · 2025-03-04T07:02:39Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+:::note
+When using `versioned` merge engine, there are the following limits:
+- `UPDATE` and `DELETE` statements are not supported
+- Partial update is not supported


Add one more note just like first-row merge engine:

`UPDATE_BEFORE` and `DELETE` changelog events are ignored automatically

luoyuxia · 2025-03-04T07:09:22Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

 # Versioned Merge Engine

-TODO: Fill me #459
+By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.


nit:

Suggested change

By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.

By setting `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than or equal to the stored value. If it is less than or null, no update will be made.

luoyuxia · 2025-03-04T07:15:08Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md


-TODO: Fill me #459
+By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.
+


I'd like to suggest to add a statement to explain how the feature is useful just like first-row merge page does, maybe something like:

This feature is particularly valuable for replacing [Deduplication](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/queries/deduplication/) transformations in streaming computations, reducing complexity and improving overall efficiency.

luoyuxia · 2025-03-04T07:15:58Z

website/docs/table-design/table-types/pk-table/merge-engines/versioned.md

+example:
+```sql title="versioned"
+
+create table merge_engine_with_version (


Could you please change SQL keyword to uppercase to align the style of first-row merge engine page?

sunxiaojian · 2025-03-04T10:33:32Z

@luoyuxia fixed

luoyuxia

Thanks @sunxiaojian LGTM. @wuchong Could you please help review agian?

Add document about versioned merge engine apache#459

a4408d2

sunxiaojian changed the title ~~Add document about versioned merge engine #459~~ Add document about versioned merge engine Mar 4, 2025

caozhen1937 reviewed Mar 4, 2025

View reviewed changes

luoyuxia reviewed Mar 4, 2025

View reviewed changes

sunxiaojian force-pushed the version-merge-doc branch from 7b8ef36 to 2296af8 Compare March 4, 2025 10:36

fixed doc

f26743c

sunxiaojian force-pushed the version-merge-doc branch from 2296af8 to f26743c Compare March 4, 2025 10:40

luoyuxia approved these changes Mar 4, 2025

View reviewed changes

updates

bb6fcbd

wuchong approved these changes Mar 5, 2025

View reviewed changes

wuchong merged commit cc6b9af into apache:main Mar 5, 2025
2 checks passed

ZmmBigdata pushed a commit to ZmmBigdata/fluss that referenced this pull request Jun 20, 2025

[docs] Add document for versioned merge engine (apache#523)

1faedcf

	insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v1', 1000);
	insert into merge_engine_with_version (a, b, ts ) VALUES (1, 'v1', 1000);



		-- insert data with ts > 1000, update will be made
		insert into merge_engine_with_version ( a,b,ts ) VALUES (1, 'v2', 2000);

	By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.
	By setting `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than or equal to the stored value. If it is less than or null, no update will be made.


		TODO: Fill me #459
		By specifying `'table.merge-engine' = 'versioned'`, users can update data based on the configured version column. Updates will be carried out when the latest value of the specified field is greater than the stored value. If it is less than or empty, no update will be made.

Add document about versioned merge engine #523

Add document about versioned merge engine #523

Uh oh!

Conversation

sunxiaojian commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Tests

API and Format

Documentation

Uh oh!

sunxiaojian commented Mar 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luoyuxia left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sunxiaojian commented Mar 4, 2025

Uh oh!

luoyuxia left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sunxiaojian commented Mar 4, 2025 •

edited

Loading