Skip to content

Conversation

@aschackmull
Copy link
Contributor

This was identified by the join-order metric. It's not recursive, so it didn't hurt us too much, but it's an easy fix.

Before:

[2025-01-16 10:32:29] Evaluated non-recursive predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@3f9c0ft4 in 13ms (size: 38324).
Evaluated relational algebra for predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@3f9c0ft4 with tuple counts:
          15972   ~0%    {7} r1 = JOIN `num#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxNone#7d1df047` WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInNoThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#ed2cbed3` CARTESIAN PRODUCT OUTPUT Rhs.0, Rhs.1, Rhs.4, Rhs.5, Lhs.0, Rhs.2, Rhs.3
          26378   ~0%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.0, Lhs.5, Rhs.4, Lhs.1, Lhs.6, Rhs.5, Lhs.4, Rhs.6, Rhs.7, Rhs.8
                     
           4063   ~0%    {6} r2 = SCAN `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxSome#abb16013` OUTPUT In.1, In.2, In.3, In.4, In.0, In.5
        3282287   ~0%    {11}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7_467801523#join_rhs` ON FIRST 4 OUTPUT Rhs.4, Rhs.5, Lhs.4, Rhs.7, Rhs.8, Lhs.0, Lhs.1, Lhs.2, Lhs.3, Lhs.5, Rhs.6
          11946   ~1%    {10}    | JOIN WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#5803fd18_012453#join_rhs` ON FIRST 5 OUTPUT Lhs.0, Lhs.2, Lhs.5, Lhs.1, Rhs.5, Lhs.10, Lhs.9, Lhs.6, Lhs.7, Lhs.8
                     
          38324   ~0%    {10} r3 = r1 UNION r2
                         return r3

After:

[2025-01-16 10:31:51] Evaluated non-recursive predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@d5c761bm in 5ms (size: 38324).
Evaluated relational algebra for predicate DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::S6Graph::fwdFlowInStep/10#316df909@d5c761bm with tuple counts:
        15972   ~0%    {7} r1 = JOIN `num#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxNone#7d1df047` WITH `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInNoThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#ed2cbed3` CARTESIAN PRODUCT OUTPUT Rhs.0, Rhs.1, Rhs.4, Rhs.5, Lhs.0, Rhs.2, Rhs.3
        26378   ~0%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.0, Lhs.5, Rhs.4, Lhs.1, Lhs.6, Rhs.5, Lhs.4, Rhs.6, Rhs.7, Rhs.8
                   
         6214   ~5%    {6} r2 = SCAN `project#DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::FwdFlowInThrough::fwdFlowInValidEdgeTypeFlowEnabled/8#5803fd18` OUTPUT In.0, In.1, In.4, In.5, In.2, In.3
        11946   ~0%    {9}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::fwdFlowIntoArg/9#973fa733#reorder_0_2_6_8_1_3_4_5_7` ON FIRST 4 OUTPUT Lhs.4, Rhs.4, Rhs.6, Rhs.7, Rhs.8, Lhs.0, Lhs.1, Lhs.5, Rhs.5
        11946   ~1%    {10}    | JOIN WITH `DataFlowImpl::MakeImpl<Location::Location,DataFlowImplSpecific::JavaDataFlow>::Impl<SensitiveLoggingQuery::SensitiveLoggerFlow::C>::Stage6::TSummaryCtxSome#abb16013` ON FIRST 5 OUTPUT Lhs.5, Lhs.0, Lhs.1, Lhs.6, Lhs.7, Lhs.8, Rhs.5, Lhs.2, Lhs.3, Lhs.4
                   
        38324   ~0%    {10} r3 = r1 UNION r2
                       return r3

(I've also piggy-backed a commit that deletes an outdated comment)

@aschackmull aschackmull added the no-change-note-required This PR does not need a change note label Jan 16, 2025
Copilot AI review requested due to automatic review settings January 16, 2025 09:36
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)
  • shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qll: Language not supported

Tip: Copilot only keeps its highest confidence comments to reduce noise and keep you focused. Learn more

@aschackmull aschackmull merged commit fbf25e4 into github:main Jan 16, 2025
33 checks passed
@aschackmull aschackmull deleted the dataflow/join-fix branch January 16, 2025 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DataFlow Library no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants