You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
Filtering for Hudi incremental queries on tables with version >= 6 depends on commit completion time which is not present in the output for the users. Filtering with requested times (currently outputted) gives inconsistent and wrong results. It is difficult for the users to give proper times as part of the filter if it is not outputted.
What you expected:
Implement a completion time virtual column (_hoodie_commit_completion_time) for Hudi incremental queries on tables with version >= 6. This virtual column is dynamically added during query execution and provides completion time semantics for filtering and display.
Steps to reproduce:
Create a Hudi Table with incremental query support
Performing Incremental Query with requested times (which is outputted currently) gives inconsistent and wrong results.