[CALCITE-6846] Support basic dphyp join reorder algorithm

[silundong]

The main enumeration process of dphyp has been implemented. There are two new rules, one for converting Joins into a HyperGraph and the other for triggering the dphyp algorithm. More details could see:https://issues.apache.org/jira/browse/CALCITE-6846

[mihaibudiu]

Is this ready for review?
If not, please mark as draft.

[silundong]

Is this ready for review? If not, please mark as draft.

Ready, please review.
There are still some checks that are not passed, but I think I have taken care of them in the code. Can you give me some advice on how to pass these checks?

[NobiGo]

Some parameters might need to be checked whether they are NULL or undergo active type conversion.

[silundong]

Some parameters might need to be checked whether they are NULL or undergo active type conversion.

Thank you, I will update it.

[rubenada]

This is a great contribution @silundong.
Just out of curiosity, have you tested locally what happens if you add the new HyperGraph rules into the "default rules" (e.g. RelOptRules#BASE_RULES, or Programs#RULE_SET) and execute Calcite tests? Do you get any any suspicious error (except for the expected plan changes regarding Join-ordering that the new rules may generate)?

[silundong]

This is a great contribution @silundong. Just out of curiosity, have you tested locally what happens if you add the new HyperGraph rules into the "default rules" (e.g. RelOptRules#BASE_RULES, or Programs#RULE_SET) and execute Calcite tests? Do you get any any suspicious error (except for the expected plan changes regarding Join-ordering that the new rules may generate)?

Thank you very much for your recognition. I did not add it to the "default rules" (e.g. RelOptRules#BASE_RULES, or Programs#RULE_SET). However, in other projects, I have integrated the Calcite parsing and optimization module and added HyperGraph rules. It has worked without exceptions under TPC-DS.

[rubenada]

Thanks for addressing my initial comments @silundong , I'll take another look in the coming days.

[silundong]

@rubenada Thank you very much for your review!

[rubenada]

@silundong I have left some remarks , rather generic or Calcite-related, not really dphyp-algo related, which I find I bit hard to follow, but it's good that you provided tests showing the results.
IMO we could consider adding this feature, maybe initially flagging the new classes and rule definitions as @Experimental (as it happened in the past with other features). Let's see what other contributors think about it...

[silundong]

@silundong I have left some remarks , rather generic or Calcite-related, not really dphyp-algo related, which I find I bit hard to follow, but it's good that you provided tests showing the results. IMO we could consider adding this feature, maybe initially flagging the new classes and rule definitions as @Experimental (as it happened in the past with other features). Let's see what other contributors think about it...

Thank you very much for your suggestion, I will update them. I fully support adding the @Experimental annotation. It is a new feature that needs to be validated through more extensive use. In the early stages, users can use it selectively.

[sonarqubecloud]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rubenada]

@silundong could you please take a look at Alessandro's minor comments in the Jira ticket?