You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Expand the **Advanced Config** panel in the modal.
30
30
- Specify the **Target File Size** for the table (default **512 MB** if you leave it unchanged).
31
31
32
-
For how **target file size** affects **Lite**, **Medium**, and **Full** compaction, see [Types of Compaction Supported in OLake](/docs/iceberg-maintenance/compaction/overview).
32
+
For how **target file size** affects **Lite**, **Medium**, and **Full** compaction, see [Types of Compaction Supported in OLake Fusion](/docs/fusion/compaction/types-of-compaction/#types-of-compaction).
33
33
34
34
> **Tip:** Choose a target size based on your query patterns and table size. Larger files can improve scan efficiency based on the query but may increase the cost of rewriting files.
- Once saved, the **Status** toggle for that table turns on automatically and compaction runs on the schedule you configured. Confirm this in the **Status** column as the toggle should appear active.
- After saving, you will be redirected to the **Tables** page.
52
-
- Locate the table and **toggle the Status switch** to activate scheduled compaction for that specific table.
52
+
## 2. Bulk Configuration
53
53
54
-
> **Important:** The **Status toggle must be switched on**. Even if a cron schedule is configured, compaction will not execute unless the table is enabled.
54
+
Bulk configuration lets you apply the same compaction schedule and settings to multiple tables in one go, instead of configuring each table individually.
To get started, select the tables you want to configure from the **Tables** page using their checkboxes, then click **Bulk Configure**. The same configuration modal opens — set the schedule, target file size, and save, exactly as described in Steps 2–4 of [Single Table Configuration](#1-single-table-configuration).
If any configuration is left unset, the following defaults apply: **24 hours** as the frequency for each compaction type and **512 MB** as the target file size.
Copy file name to clipboardExpand all lines: docs/fusion/compaction/types-of-compaction.mdx
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,19 +17,21 @@ OLake supports three types of compaction:
17
17
18
18
### 1. Lite Compaction
19
19
20
-
Lite compaction is the lightest and most frequently run type. It focuses on two things, **merging fragment into larger ones** and **converting Equality Delete Files into Position Delete Files**. Position Delete Files are cheaper for query engines to process, so this conversion alone improves read performance without doing a heavy rewrite. Since streaming writes and high-frequency CDC constantly produce small fragment files, Minor compaction is typically scheduled to run frequentlly to keep the table tidy before the clutter builds up.
20
+
Lite compaction is the lightest and most frequently run type. It focuses on two things, **merging fragment into larger ones** and **converting Equality Delete Files into Position Delete Files**. Position Delete Files are cheaper for query engines to process, so this conversion alone improves read performance without doing a heavy rewrite. Since streaming writes and high-frequency CDC constantly produce small fragment files, Lite compaction is typically scheduled to run frequentlly to keep the table tidy before the clutter builds up.
21
21
22
22
### 2. Medium Compaction
23
23
24
-
Medium compaction goes a step further. It **merges segment files up to the target file size**, and when too many Position Delete Files have accumulated, it merges them directly into the corresponding Data Files that is physically removing deleted rows from the table. This is more thorough than Minor compaction but still does not rewrite the entire table. The Medium Compaction is typically scheduled less frequently than the Lite Compaction to keep the table efficient and not spend too much compute.
24
+
Medium compaction goes a step further. It **merges segment files up to the target file size**, and when too many Position Delete Files have accumulated, it merges them directly into the corresponding Data Files that is physically removing deleted rows from the table. This is more thorough than Lite compaction but still does not rewrite the entire table. The Medium Compaction is typically scheduled less frequently than the Lite Compaction to keep the table efficient and not spend too much compute.
25
25
26
26
### 3. Full Compaction
27
27
28
28
Full compaction is the deepest and most comprehensive type. It rewrites all data files fragments, segments, and delete files into optimally sized files that exactly match the configured target file size. Because it rewrites the entire table, it is the most compute-intensive option and is typically run less frequently. Use it when tables have accumulated heavy fragmentation over time or when you need the best possible query performance.
29
29
30
30
:::info Compaction precedence
31
31
32
-
When more than one type of compaction is scheduled for a table to run at the same time, **only the highest runs**: **Full** overrides **Medium** and **Lite**; **Medium** overrides **Lite**. For example, if **Full**, **Medium**, and **Lite** are all due together, **Full** runs alone; if **Medium** and **Lite** are due together, **Medium** runs alone.
32
+
When more than one type is scheduled at the same time, only the highest-priority type runs:
Saving the configuration does not start compaction automatically. You must enable it:
139
-
140
-
1. On the **Tables** page, locate the **Status** column next to the **Configure** button for your table.
141
-
2. Use the **toggle** in the **Status** column to enable the compaction configuration.
135
+
Once saved, the **Status** toggle for that table turns on automatically and compaction runs on the schedule you configured. Confirm this in the **Status** column as the toggle should appear active.
Once enabled, OLake Fusion will start running compaction for that table according to the schedule you configured.
146
-
147
139
## Health Score and Last Run Status
148
140
149
141
With a catalog and database selected, the **Tables** page shows one row per table. The sections below explain **Health Score** (overall table health) and **Last Run status** (per-type status for Lite, Medium, and Full compaction).
Copy file name to clipboardExpand all lines: docs/fusion/getting-started/overview.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,7 +34,7 @@ You should consider running maintenance in the following scenarios:
34
34
Iceberg Maintenance is available starting from v0.4.0. Upgrade OLake UI to access the **Maintenance** module.
35
35
36
36
-**Existing users (Docker):** If you are already using OLake Go for Ingestion, follow the [upgrade guide](/docs/fusion/install/olake-ui/?setup-mode=configuration#updating-olake-ui-version) to access the Maintenance module.
37
-
-**Existing users (Helm / Kubernetes):** If you are running OLake Go on Kubernetes, follow the [chart upgrade guide](http://localhost:3000/docs/fusion/install/kubernetes-compaction/?setup-mode=installation#upgrading-chart-version) to access the Maintenance module.
37
+
-**Existing users (Helm / Kubernetes):** If you are running OLake Go on Kubernetes, follow the [chart upgrade guide](/docs/fusion/install/kubernetes-compaction/?setup-mode=installation#upgrading-chart-version) to access the Maintenance module.
38
38
-**New Users (Docker):** Follow the [quickstart guide](/docs/fusion/install/olake-ui/?setup-mode=installation&quick-start=Ingestion+%2B+Maintenance#one-command-setup) to get started.
39
39
-**New Users (Helm / Kubernetes):** Follow the [quickstart guide](/docs/fusion/install/kubernetes-compaction/?setup-mode=installation#quick-start) to get started.
Copy file name to clipboardExpand all lines: docs/intro.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -99,7 +99,7 @@ import Head from '@docusaurus/Head'
99
99
</a>
100
100
</h1>
101
101
102
-
<palign="center">Fastest open-source tool for replicating Databases to Apache Iceberg or Data Lakehouse. ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Visit <ahref="https://olake.io"target="_blank">olake.io</a> for the full documentation, and benchmarks</p>
102
+
<palign="center">Fastest open-source tool for replicating Databases to Apache Iceberg or Data Lakehouse. ⚡ Efficient, quick and scalable data ingestion for real-time analytics.</p>
0 commit comments