Skip to content

Commit f0a13fe

Browse files
authored
[docs] Add lake tiering behavior changes to upgrade notes (apache#1761)
1 parent 3ec81a8 commit f0a13fe

File tree

1 file changed

+48
-1
lines changed

1 file changed

+48
-1
lines changed

website/docs/maintenance/operations/upgrade-notes-0.8.md

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,4 +50,51 @@ The following metrics are changed:
5050
- `fluss_tabletserver_table_bucket_kv_preWriteBufferTruncateAsErrorPerSecond` to `fluss_tabletserver_preWriteBufferTruncateAsErrorPerSecond`
5151
- Correction addresses reporting errors in metric names by changing the `table` level metric prefix from `fluss_tabletserver_table__` (used a double underscore (__)) to `fluss_tabletserver_table_`.
5252
- The affected metrics are all metrics with [Scope: tableserver, infix: table](docs/maintenance/observability/monitor-metrics.md#tablebucket)
53-
- For example, change `fluss_tabletserver_table__messagesInPerSecond` to `fluss_tabletserver_table_messagesInPerSecond`.
53+
- For example, change `fluss_tabletserver_table__messagesInPerSecond` to `fluss_tabletserver_table_messagesInPerSecond`.
54+
55+
## Fluss Datalake Tiering Service
56+
57+
### Disables Auto-Compaction By Default
58+
59+
⚠️ **Breaking Change**: Beginning with Fluss v0.8, auto-compaction during datalake tiering is **disabled by default**. This is a significant behavioral change that may affect your existing workflows.
60+
61+
#### What Changed
62+
63+
| Version | Auto-Compaction Behavior |
64+
|-------------------------|---------------------------------------------------------------------------------------------------|
65+
| **v0.7 and earlier** |**Enabled by default** - Compaction runs automatically during tiering |
66+
| **v0.8** |**Disabled by default** - Only data movement occurs; compaction must be explicitly enabled |
67+
68+
#### Impact Assessment
69+
70+
**If you rely on automatic compaction for storage optimization**, you will need to take action to maintain the same behavior.
71+
72+
#### How to Enable Auto-Compaction
73+
74+
To restore the previous behavior, configure auto-compaction on a per-table basis via setting the table option `'table.datalake.auto-compaction' = 'true'`:
75+
76+
```sql title="Flink SQL"
77+
-- Enable auto-compaction for a specific table
78+
CREATE TABLE your_table_name (
79+
col1 INT,
80+
col2 INT
81+
) WITH (
82+
'table.datalake.auto-compaction' = 'true'
83+
);
84+
```
85+
86+
#### Why This Change Was Made
87+
88+
This change prioritizes **tiering service stability and performance**:
89+
90+
- **🚀 Better Performance**: Compaction is CPU/IO intensive and can slow down the core tiering process
91+
- **🎯 Focused Functionality**: The tiering service now focuses solely on reliable data movement
92+
- **⚙️ Granular Control**: You can now optimize compaction strategy per table based on your specific needs
93+
- **🔧 Resource Management**: Better control over when and where resource-intensive operations occur
94+
95+
#### Best Practices
96+
97+
- **Enable auto-compaction** for tables with high write frequency and storage cost concerns
98+
- **Disable auto-compaction** for tables where tiering speed is more important than storage optimization
99+
- **Monitor resource usage** when enabling auto-compaction to ensure it doesn't impact tiering performance
100+
- **Consider manual compaction** for large tables during maintenance windows

0 commit comments

Comments
 (0)