Skip to content

Conversation

@raunaqmorarka
Copy link
Member

Description

Adds useful information to TableFinish operator summary of CTAS, INSERT and REFRESH MV queries
e.g.

"connectorOutputMetadata" : {
  "trino_query_id" : "20260120_051928_00000_uh66u",
  "trino_user" : "raunaq.morarka",
  "added-data-files" : "5",
  "added-records" : "25",
  "added-files-size" : "5577",
  "changed-partition-count" : "5",
  "total-records" : "25",
  "total-files-size" : "5577",
  "total-data-files" : "5",
  "total-delete-files" : "0",
  "total-position-deletes" : "0",
  "total-equality-deletes" : "0",
  "engine-version" : "dev",
  "engine-name" : "trino",
  "iceberg-version" : "Apache Iceberg 1.10.1 (commit ccb8bc435062171e64bc8b7e5f56e6aed9c5b934)"
}

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances Iceberg table operations by populating snapshot summary metadata in ConnectorOutputMetadata for CTAS, INSERT, and REFRESH MATERIALIZED VIEW queries. The metadata includes useful metrics like added records, data files, file sizes, and partition counts that are now visible in the TableFinish operator summary.

Changes:

  • Created new IcebergCommitMetadata class to wrap Iceberg snapshot summary information
  • Modified finishInsert to return snapshot summary instead of file paths
  • Modified finishRefreshMaterializedView to return snapshot summary instead of file paths
  • Added test coverage for INSERT, CTAS, and REFRESH MV operations

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
IcebergCommitMetadata.java New class implementing ConnectorOutputMetadata to hold snapshot summary metrics
IcebergMetadata.java Modified finishInsert and finishRefreshMaterializedView to capture and return snapshot summaries; removed HiveWrittenPartitions dependency
BaseIcebergConnectorTest.java Added testCommitMetrics to verify snapshot metadata is properly populated for INSERT, CTAS, and REFRESH MV operations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@raunaqmorarka raunaqmorarka merged commit 07413b8 into trinodb:master Jan 20, 2026
45 checks passed
@raunaqmorarka raunaqmorarka deleted the raunaq/commit-metrics branch January 20, 2026 07:32
@github-actions github-actions bot added this to the 480 milestone Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed iceberg Iceberg connector

Development

Successfully merging this pull request may close these issues.

3 participants