Skip to content

avoid quadratic leaf re-adding in get_leaves_inside_matching_brackets#5242

Open
sahvx655-wq wants to merge 1 commit into
psf:mainfrom
sahvx655-wq:matching-brackets-linear
Open

avoid quadratic leaf re-adding in get_leaves_inside_matching_brackets#5242
sahvx655-wq wants to merge 1 commit into
psf:mainfrom
sahvx655-wq:matching-brackets-linear

Conversation

@sahvx655-wq

Copy link
Copy Markdown
Contributor

Description

Profiling on inputs with deeply nested brackets kept pointing at
get_leaves_inside_matching_brackets in brackets.py, which runs while splitting
long lines (should_split_funcdef_with_rhs on return-type leaves, and the head
component of bracket_split_build_line). It gathers the ids of leaves sitting inside
matching bracket pairs, but on every matched closing bracket it re-adds the entire span
from the opening bracket to the closing one through an inner for j in range(start, i + 1)
loop. A leaf wrapped in k nested pairs is added k times, so the pass is O(n·depth)
and turns quadratic on something like a heavily subscripted annotation. A direct
microbenchmark shows the tell-tale 4x-per-doubling curve (a 4000-deep nested sequence
takes ~0.65s), and the helper is on the pre-auth formatting path reachable through
blackd, so a small crafted input buys disproportionate work.

Instead of re-adding the whole enclosed span each time a pair closes, each leaf id is
now collected under its innermost open bracket and flushed into the result only once,
when that bracket's matching close is reached; unmatched openings simply drop the ids
they collected. Keeping the bookkeeping on the bracket stack means every id is recorded
and unioned at most once, so the helper is linear while the returned set is identical. I
checked equivalence against the previous implementation over 200k randomised bracket
sequences and the full format suite stays byte-identical; the 4000-deep input drops from
~0.65s to ~0.005s.

Checklist - did you ...

  • Implement any code style changes under the --preview style, following the stability policy? (n/a, no style change)
  • Add an entry in CHANGES.md if necessary?
  • Add / update tests if necessary? (byte-identical behaviour, covered by the existing format suite)
  • Add new / update outdated documentation? (n/a)

@github-actions

Copy link
Copy Markdown
Contributor

diff-shades results comparing this PR (0b00eef) to main (51abf53):

--preview style: no changes

--stable style: no changes


What is this? | Workflow run | diff-shades documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant