Skip to content

Remove legacy Iceberg MV separate storage table support #21370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

findepi
Copy link
Member

@findepi findepi commented Apr 3, 2024

No description provided.

@findepi findepi requested review from electrum, ebyhr and alexjo2144 April 3, 2024 08:43
@cla-bot cla-bot bot added the cla-signed label Apr 3, 2024
@findepi

This comment was marked as outdated.

@github-actions github-actions bot added docs iceberg Iceberg connector labels Apr 3, 2024
@findepi findepi force-pushed the findepi/iceberg-mv-hideon branch 2 times, most recently from 8896096 to 59d5700 Compare April 3, 2024 09:02
@findepi findepi force-pushed the findepi/iceberg-mv-hideon branch from 59d5700 to 3e0090a Compare April 3, 2024 10:29
@findepi findepi force-pushed the findepi/iceberg-mv-hideon branch from 3e0090a to 8197f0f Compare April 3, 2024 13:29

private final List<PropertyMetadata<?>> materializedViewProperties;

@Inject
public IcebergMaterializedViewProperties(IcebergConfig icebergConfig, IcebergTableProperties tableProperties)
public IcebergMaterializedViewProperties(
@EnableMaterializedViewSeparateStorageTable boolean enableMaterializedViewSeparateStorageTable, // for tests of legacy materialized views
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don’t add this. We don’t want special code just for testing. We’re removing this pattern from the code base.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed this shouldn't be here, and should eventually move to test code. I didn't see the urgency to do this within one PR. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think we should add more test-only annotations to production code. This is an anti-pattern and we don’t want people to see this and think it is ok.

We’re keeping legacy code to create these views, so it’s not clear how much this change is helping at this stage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I could save you some time on your next review round, if you gave me a hint how you prefer want to see this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create the legacy view manually in the test code? Or maybe we start an old version of Trino in Testcontainers to write the legacy view?

@electrum
Copy link
Member

electrum commented Apr 8, 2024

What is the deprecation plan here? I see that we're just removing these configs, but they aren't deprecated today. What makes them legacy?

How long do we plan to support reading these old tables? How do users convert from the legacy to the new way? Should we have a callable procedure to migrate?

If this is deprecated and we plan to remove read support, in say six months, then we could simply deprecate this for now and remove the code entirely. Otherwise, we need a way to test this that doesn't require leaving the writing code in the production code path.

Copy link

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Apr 30, 2024
@findepi findepi added the stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed. label May 6, 2024
@ebyhr ebyhr removed their request for review September 4, 2024 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed docs iceberg Iceberg connector stale stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed.
Development

Successfully merging this pull request may close these issues.

2 participants