DeltaLake support `optimize` output metrics in result by Max-Cheng · Pull Request #29040 · trinodb/trino

Max-Cheng · 2026-04-08T07:20:39Z

Description

Allow distributed table procedures to output metrics in result.
DeltaLake OPTIMIZE procedure returns the output below:

trino> ALTER TABLE deltalake.tpch.region EXECUTE optimize;
ALTER TABLE EXECUTE
        metric_name         | metric_value
----------------------------+--------------
 rewritten_data_files_count |            2
 removed_delete_files_count |            0
 added_data_files_count     |            1
(3 rows)

Fixes Return execution metrics while running the OPTIMIZE command in Delta Lake #28999

Release notes

## Delta Lake
* Add support for execution metrics while running the `optimize` procedure. ({issue}`28992`)

cla-bot · 2026-04-08T07:20:42Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Ray Cheng.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email email@example.com
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

Max-Cheng · 2026-04-08T07:45:55Z

@ebyhr OMG your response speed is so fast!

docs/src/main/sphinx/connector/delta-lake.md

findinpath · 2026-04-08T10:57:02Z

plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java

    }

-    private void finishOptimize(ConnectorSession session, DeltaLakeTableExecuteHandle executeHandle, Collection<Slice> fragments, List<Object> splitSourceInfo)
+    private Map<String, Long> finishOptimize(ConnectorSession session, DeltaLakeTableExecuteHandle executeHandle, Collection<Slice> fragments, List<Object> splitSourceInfo)


nit: I'd recommend rather returning OptimizeResult and doing the toMap in the calling context instead.
If you take the suggestion, pls do a preparatory commit as well for IcebergMetadata

Let's avoid touching unrelated modules in this PR. There is no need to modify IcebergMetadata in my opinion.

@ebyhr Maybe it's another task?

findinpath · 2026-04-08T11:00:42Z

...rino-delta-lake/src/test/java/io/trino/plugin/deltalake/BaseDeltaLakeConnectorSmokeTest.java

            for (int i = 0; i < 3; i++) {
                Set<String> initialFiles = getActiveFiles(tableName);
-                computeActual("ALTER TABLE " + tableName + " EXECUTE OPTIMIZE");
+                computeActual(withSingleWriterPerTask(getSession()), "ALTER TABLE " + tableName + " EXECUTE OPTIMIZE");


Let's rather do a preparatory commit that makes use of withSingleWriterPerTask to explain that we're aiming for stable results.

but, it seems an un-related to the change, is it required by the current(pr's) change?

@chenjian2664 @findinpath yep, this is an un-related change: to avoid flaky test results, I’ll split these commits into separate ones.

findinpath

LGTM % nits

chenjian2664 · 2026-04-08T11:04:00Z

docs/src/main/sphinx/connector/delta-lake.md

+        metric_name         | metric_value
+----------------------------+--------------
+ rewritten_data_files_count |            2
+ removed_delete_files_count |            0


We can remove it if it is guaranteed to always be 0

plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java

chenjian2664 · 2026-04-08T11:10:50Z

...rino-delta-lake/src/test/java/io/trino/plugin/deltalake/BaseDeltaLakeConnectorSmokeTest.java

            for (int i = 0; i < 3; i++) {
                Set<String> initialFiles = getActiveFiles(tableName);
-                computeActual("ALTER TABLE " + tableName + " EXECUTE OPTIMIZE");
+                computeActual(withSingleWriterPerTask(getSession()), "ALTER TABLE " + tableName + " EXECUTE OPTIMIZE");


but, it seems an un-related to the change, is it required by the current(pr's) change?

chenjian2664 · 2026-04-08T11:11:09Z

...rino-delta-lake/src/test/java/io/trino/plugin/deltalake/BaseDeltaLakeConnectorSmokeTest.java

+            MaterializedResult optimizeResult = computeActual(withSingleWriterPerTask(getSession()), "ALTER TABLE " + tableName + " EXECUTE OPTIMIZE");
+
+            Map<String, Long> metrics = optimizeResult.getMaterializedRows().stream()
+                    .collect(ImmutableMap.toImmutableMap(


static import

plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java

ebyhr · 2026-04-09T19:05:13Z

plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java

+        return new OptimizeResult(scannedDataFiles.size(), filesToDelete.size(), dataFileInfos.size());
+    }
+
+    private record OptimizeResult(long rewrittenDataFiles, long removedDeleteFiles, long addedDataFiles)


We don't need this record in this connector. Please remove.

Iceberg connector uses a record class because there are 2 return in finishOptimize method.

ebyhr · 2026-04-09T19:08:45Z

...rino-delta-lake/src/test/java/io/trino/plugin/deltalake/BaseDeltaLakeConnectorSmokeTest.java

    }

+    @Test
+    public void testOptimizeWithDeletionVectors()


BaseDeltaLakeConnectorSmokeTest is used for running tests on several storages. Please consider reverting changes in this class, and updating TestDeltaLakeConnectorTest instead.

github-actions bot added docs delta-lake Delta Lake connector labels Apr 8, 2026

Max-Cheng changed the title ~~DeltaLake support optimzie output metrics in result~~ DeltaLake support optimize output metrics in result Apr 8, 2026

Max-Cheng force-pushed the deltalake_optimize_metrics branch from ae8ceb4 to 1bbdd9b Compare April 8, 2026 07:25

cla-bot bot added the cla-signed label Apr 8, 2026

ebyhr requested review from chenjian2664, ebyhr and findinpath April 8, 2026 07:28

Max-Cheng marked this pull request as ready for review April 8, 2026 07:29

Max-Cheng force-pushed the deltalake_optimize_metrics branch from 1bbdd9b to c6e9dfd Compare April 8, 2026 07:29

Max-Cheng force-pushed the deltalake_optimize_metrics branch from c6e9dfd to c544c15 Compare April 8, 2026 08:26

findinpath reviewed Apr 8, 2026

View reviewed changes

docs/src/main/sphinx/connector/delta-lake.md Show resolved Hide resolved

Max-Cheng force-pushed the deltalake_optimize_metrics branch from c544c15 to 1eebb67 Compare April 8, 2026 10:56

findinpath reviewed Apr 8, 2026

View reviewed changes

chenjian2664 reviewed Apr 8, 2026

View reviewed changes

ebyhr reviewed Apr 8, 2026

View reviewed changes

plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java Outdated Show resolved Hide resolved

Max-Cheng force-pushed the deltalake_optimize_metrics branch from 1eebb67 to 3abfa15 Compare April 9, 2026 10:57

Max-Cheng added 2 commits April 9, 2026 18:59

Use single writer per task for DeltaLake stable optimize test results

2761b84

Add execution metrics output for Delta Lake optimize procedure

5665182

Max-Cheng force-pushed the deltalake_optimize_metrics branch from 3abfa15 to 5665182 Compare April 9, 2026 10:59

ebyhr reviewed Apr 9, 2026

View reviewed changes

Conversation

Max-Cheng commented Apr 8, 2026 • edited by ebyhr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Release notes

Uh oh!

cla-bot bot commented Apr 8, 2026

Uh oh!

Max-Cheng commented Apr 8, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

findinpath left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ebyhr Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Max-Cheng commented Apr 8, 2026 •

edited by ebyhr

Loading

ebyhr Apr 9, 2026 •

edited

Loading