You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/table-design/table-types/log-table.md
+43Lines changed: 43 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -42,3 +42,46 @@ Log Tables in Fluss allow real-time data consumption, preserving the order of da
42
42
## Log Tiering
43
43
Log Table supports tiering data to different storage tiers. See more details about [Remote Log](/docs/maintenance/tiered-storage/remote-storage/).
44
44
45
+
## Log Compression
46
+
Log Table supports the end-to-end block compression feature for arrow log format. If enabled, data will be compressed
47
+
by the writer in client, written in compressed format and decompressed by the log scanner in client. Log compression
48
+
can significantly reduce storage costs on the server side.
49
+
50
+
Currently, Log Table is defaulted to using `ZSTD` compression codec, with the compression level set to `3`. If you want to
51
+
modify the compression codec or compression level of `ZSTD` compression codec, you can create table as:
52
+
53
+
```sql title="Flink SQL"
54
+
-- Set the compression codec to LZ4_FRAME
55
+
CREATETABLElog_table (
56
+
order_id BIGINT,
57
+
item_id BIGINT,
58
+
amount INT,
59
+
address STRING
60
+
)
61
+
WITH (
62
+
'table.log.arrow.compression.type'='LZ4_FRAME'
63
+
);
64
+
65
+
-- Set the 'ZSTD' compression level to 2
66
+
CREATETABLElog_table (
67
+
order_id BIGINT,
68
+
item_id BIGINT,
69
+
amount INT,
70
+
address STRING
71
+
)
72
+
WITH (
73
+
'table.log.arrow.compression.zstd.level'='2'
74
+
);
75
+
```
76
+
In the above example, we set the compression codec to `LZ4_FRAME` and the compression level to `2`.
77
+
78
+
:::note
79
+
1. Currently, the compression codec and compression level are only supported for arrow format. If you set `'table.log.format'='indexed'`, the compression codec and compression level will be ignored.
80
+
2. The valid range of `table.log.arrow.compression.zstd.level` is 1 to 22.
81
+
:::
82
+
83
+
### Supported Compression Codecs
84
+
Currently, Fluss supports the following compression codecs for arrow format:
0 commit comments