avoid quadratic per-leaf subtree walk in is_complex_subscript#5239
Open
sahvx655-wq wants to merge 2 commits into
Open
avoid quadratic per-leaf subtree walk in is_complex_subscript#5239sahvx655-wq wants to merge 2 commits into
sahvx655-wq wants to merge 2 commits into
Conversation
sahvx655-wq
force-pushed
the
complex-subscript-cache
branch
from
July 15, 2026 12:21
67bf189 to
9e4ac18
Compare
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Profiling a file that packs a long run of implicitly concatenated strings into a single subscript or list (
x = ["s0" "s1" ... "sn"]) showed the whole run going quadratic in the default style, withLine.is_complex_subscriptandpre_orderdominating the trace.appendcalls it once per leaf while a square bracket is open, and when the bracketed body is one node that holds no comparison or arithmetic sub-node theany(... in subscript_start.pre_order())walk never short-circuits, so each of the n string leaves re-walks the entire subtree. A comma-separated subscript dodges this becausesubscriptlistnarrows to the relevant child throughchild_towards, but a single implicitly concatenated string sits directly under the bracket and gets walked in full every time. It is reachable pre-auth through blackd, and left unfixed a 4000-string list takes several seconds where a comma list of the same size is near-instant.The walk result depends only on
subscript_start, which does not change while a line is being built, so I cache it per node on theLine. Keeping the cache at this layer rather than having callers skip the call preserves the exact spacing behaviouris_complex_subscriptdrives for genuine complex slices, and thesubscriptlistbranch still narrows per child so each distinct child is walked once. Output is byte-identical across the test data and the black source tree in stable and preview modes, and the same 4000-string input drops from ~3.4s to ~0.18s with the curve going from quadratic to linear.