avoid quadratic leaf re-adding in get_leaves_inside_matching_brackets#5242
Open
sahvx655-wq wants to merge 1 commit into
Open
avoid quadratic leaf re-adding in get_leaves_inside_matching_brackets#5242sahvx655-wq wants to merge 1 commit into
sahvx655-wq wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Profiling on inputs with deeply nested brackets kept pointing at
get_leaves_inside_matching_bracketsinbrackets.py, which runs while splittinglong lines (
should_split_funcdef_with_rhson return-type leaves, and the headcomponent of
bracket_split_build_line). It gathers the ids of leaves sitting insidematching bracket pairs, but on every matched closing bracket it re-adds the entire span
from the opening bracket to the closing one through an inner
for j in range(start, i + 1)loop. A leaf wrapped in
knested pairs is addedktimes, so the pass is O(n·depth)and turns quadratic on something like a heavily subscripted annotation. A direct
microbenchmark shows the tell-tale 4x-per-doubling curve (a 4000-deep nested sequence
takes ~0.65s), and the helper is on the pre-auth formatting path reachable through
blackd, so a small crafted input buys disproportionate work.
Instead of re-adding the whole enclosed span each time a pair closes, each leaf id is
now collected under its innermost open bracket and flushed into the result only once,
when that bracket's matching close is reached; unmatched openings simply drop the ids
they collected. Keeping the bookkeeping on the bracket stack means every id is recorded
and unioned at most once, so the helper is linear while the returned set is identical. I
checked equivalence against the previous implementation over 200k randomised bracket
sequences and the full format suite stays byte-identical; the 4000-deep input drops from
~0.65s to ~0.005s.
Checklist - did you ...
--previewstyle, following the stability policy? (n/a, no style change)CHANGES.mdif necessary?