You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/maintenance/operations/upgrade-notes-0.8.md
+48-1Lines changed: 48 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,4 +50,51 @@ The following metrics are changed:
50
50
-`fluss_tabletserver_table_bucket_kv_preWriteBufferTruncateAsErrorPerSecond` to `fluss_tabletserver_preWriteBufferTruncateAsErrorPerSecond`
51
51
- Correction addresses reporting errors in metric names by changing the `table` level metric prefix from `fluss_tabletserver_table__` (used a double underscore (__)) to `fluss_tabletserver_table_`.
52
52
- The affected metrics are all metrics with [Scope: tableserver, infix: table](docs/maintenance/observability/monitor-metrics.md#tablebucket)
53
-
- For example, change `fluss_tabletserver_table__messagesInPerSecond` to `fluss_tabletserver_table_messagesInPerSecond`.
53
+
- For example, change `fluss_tabletserver_table__messagesInPerSecond` to `fluss_tabletserver_table_messagesInPerSecond`.
54
+
55
+
## Fluss Datalake Tiering Service
56
+
57
+
### Disables Auto-Compaction By Default
58
+
59
+
⚠️ **Breaking Change**: Beginning with Fluss v0.8, auto-compaction during datalake tiering is **disabled by default**. This is a significant behavioral change that may affect your existing workflows.
|**v0.7 and earlier**| ✅ **Enabled by default** - Compaction runs automatically during tiering |
66
+
|**v0.8**| ❌ **Disabled by default** - Only data movement occurs; compaction must be explicitly enabled |
67
+
68
+
#### Impact Assessment
69
+
70
+
**If you rely on automatic compaction for storage optimization**, you will need to take action to maintain the same behavior.
71
+
72
+
#### How to Enable Auto-Compaction
73
+
74
+
To restore the previous behavior, configure auto-compaction on a per-table basis via setting the table option `'table.datalake.auto-compaction' = 'true'`:
75
+
76
+
```sql title="Flink SQL"
77
+
-- Enable auto-compaction for a specific table
78
+
CREATETABLEyour_table_name (
79
+
col1 INT,
80
+
col2 INT
81
+
) WITH (
82
+
'table.datalake.auto-compaction'='true'
83
+
);
84
+
```
85
+
86
+
#### Why This Change Was Made
87
+
88
+
This change prioritizes **tiering service stability and performance**:
89
+
90
+
-**🚀 Better Performance**: Compaction is CPU/IO intensive and can slow down the core tiering process
91
+
-**🎯 Focused Functionality**: The tiering service now focuses solely on reliable data movement
92
+
-**⚙️ Granular Control**: You can now optimize compaction strategy per table based on your specific needs
93
+
-**🔧 Resource Management**: Better control over when and where resource-intensive operations occur
94
+
95
+
#### Best Practices
96
+
97
+
-**Enable auto-compaction** for tables with high write frequency and storage cost concerns
98
+
-**Disable auto-compaction** for tables where tiering speed is more important than storage optimization
99
+
-**Monitor resource usage** when enabling auto-compaction to ensure it doesn't impact tiering performance
100
+
-**Consider manual compaction** for large tables during maintenance windows
0 commit comments