Skip to content

avoid quadratic per-leaf subtree walk in is_complex_subscript#5239

Open
sahvx655-wq wants to merge 2 commits into
psf:mainfrom
sahvx655-wq:complex-subscript-cache
Open

avoid quadratic per-leaf subtree walk in is_complex_subscript#5239
sahvx655-wq wants to merge 2 commits into
psf:mainfrom
sahvx655-wq:complex-subscript-cache

Conversation

@sahvx655-wq

Copy link
Copy Markdown
Contributor

Profiling a file that packs a long run of implicitly concatenated strings into a single subscript or list (x = ["s0" "s1" ... "sn"]) showed the whole run going quadratic in the default style, with Line.is_complex_subscript and pre_order dominating the trace. append calls it once per leaf while a square bracket is open, and when the bracketed body is one node that holds no comparison or arithmetic sub-node the any(... in subscript_start.pre_order()) walk never short-circuits, so each of the n string leaves re-walks the entire subtree. A comma-separated subscript dodges this because subscriptlist narrows to the relevant child through child_towards, but a single implicitly concatenated string sits directly under the bracket and gets walked in full every time. It is reachable pre-auth through blackd, and left unfixed a 4000-string list takes several seconds where a comma list of the same size is near-instant.

The walk result depends only on subscript_start, which does not change while a line is being built, so I cache it per node on the Line. Keeping the cache at this layer rather than having callers skip the call preserves the exact spacing behaviour is_complex_subscript drives for genuine complex slices, and the subscriptlist branch still narrows per child so each distinct child is walked once. Output is byte-identical across the test data and the black source tree in stable and preview modes, and the same 4000-string input drops from ~3.4s to ~0.18s with the curve going from quadratic to linear.

@sahvx655-wq
sahvx655-wq force-pushed the complex-subscript-cache branch from 67bf189 to 9e4ac18 Compare July 15, 2026 12:21
@github-actions

Copy link
Copy Markdown
Contributor

diff-shades results comparing this PR (9cc6741) to main (51abf53):

--preview style: no changes

--stable style: no changes


What is this? | Workflow run | diff-shades documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant