Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataflow: Minor join-order fix #18507

Merged
merged 2 commits into from
Jan 16, 2025
Merged

Conversation

aschackmull
Copy link
Contributor

This was identified by the join-order metric. It's not recursive, so it didn't hurt us too much, but it's an easy fix.

Before:

[2025-01-16 10:32:29] Evaluated non-recursive predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@3f9c0ft4 in 13ms (size: 38324).
Evaluated relational algebra for predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@3f9c0ft4 with tuple counts:
          15972   ~0%    {7} r1 = JOIN `num#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxNone#7d1df047` WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInNoThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#ed2cbed3` CARTESIAN PRODUCT OUTPUT Rhs.0, Rhs.1, Rhs.4, Rhs.5, Lhs.0, Rhs.2, Rhs.3
          26378   ~0%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.0, Lhs.5, Rhs.4, Lhs.1, Lhs.6, Rhs.5, Lhs.4, Rhs.6, Rhs.7, Rhs.8
                     
           4063   ~0%    {6} r2 = SCAN `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxSome#abb16013` OUTPUT In.1, In.2, In.3, In.4, In.0, In.5
        3282287   ~0%    {11}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7_467801523#join_rhs` ON FIRST 4 OUTPUT Rhs.4, Rhs.5, Lhs.4, Rhs.7, Rhs.8, Lhs.0, Lhs.1, Lhs.2, Lhs.3, Lhs.5, Rhs.6
          11946   ~1%    {10}    | JOIN WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#5803fd18_012453#join_rhs` ON FIRST 5 OUTPUT Lhs.0, Lhs.2, Lhs.5, Lhs.1, Rhs.5, Lhs.10, Lhs.9, Lhs.6, Lhs.7, Lhs.8
                     
          38324   ~0%    {10} r3 = r1 UNION r2
                         return r3

After:

[2025-01-16 10:31:51] Evaluated non-recursive predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@d5c761bm in 5ms (size: 38324).
Evaluated relational algebra for predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@d5c761bm with tuple counts:
        15972   ~0%    {7} r1 = JOIN `num#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxNone#7d1df047` WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInNoThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#ed2cbed3` CARTESIAN PRODUCT OUTPUT Rhs.0, Rhs.1, Rhs.4, Rhs.5, Lhs.0, Rhs.2, Rhs.3
        26378   ~0%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.0, Lhs.5, Rhs.4, Lhs.1, Lhs.6, Rhs.5, Lhs.4, Rhs.6, Rhs.7, Rhs.8
                   
         6214   ~5%    {6} r2 = SCAN `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#5803fd18` OUTPUT In.0, In.1, In.4, In.5, In.2, In.3
        11946   ~0%    {9}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.4, Rhs.4, Rhs.6, Rhs.7, Rhs.8, Lhs.0, Lhs.1, Lhs.5, Rhs.5
        11946   ~1%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxSome#abb16013` ON FIRST 5 OUTPUT Lhs.5, Lhs.0, Lhs.1, Lhs.6, Lhs.7, Lhs.8, Rhs.5, Lhs.2, Lhs.3, Lhs.4
                   
        38324   ~0%    {10} r3 = r1 UNION r2
                       return r3

(I've also piggy-backed a commit that deletes an outdated comment)

@aschackmull aschackmull added the no-change-note-required This PR does not need a change note label Jan 16, 2025
@Copilot Copilot bot review requested due to automatic review settings January 16, 2025 09:36

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)
  • shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qll: Language not supported

Tip: Copilot only keeps its highest confidence comments to reduce noise and keep you focused. Learn more

@aschackmull aschackmull merged commit fbf25e4 into github:main Jan 16, 2025
33 checks passed
@aschackmull aschackmull deleted the dataflow/join-fix branch January 16, 2025 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataFlow Library no-change-note-required This PR does not need a change note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants