Skip to content

Fix slow compilation of programs using recursive union types#4920

Draft
SeanTAllen wants to merge 1 commit intomainfrom
fix-recursive-union-subtype-perf
Draft

Fix slow compilation of programs using recursive union types#4920
SeanTAllen wants to merge 1 commit intomainfrom
fix-recursive-union-subtype-perf

Conversation

@SeanTAllen
Copy link
Copy Markdown
Member

@SeanTAllen SeanTAllen commented Mar 4, 2026

The subtype checker's assumption stack (which breaks cycles during recursive type checks) only operated on nominal types. When a recursive union type appeared as a field in its own members, the compiler would re-expand the full union membership at every level of recursion, causing exponential work in the expression pass.

Three changes fix this:

  1. Generalize exact_nominal to exact_type, which can structurally compare any type AST node (unions, intersections, tuples, arrows, type params, capabilities, ephemeral markers) — not just nominals. This lets the assumption stack recognize compound type pairs. Importantly, exact_type compares nominal typeargs structurally rather than via is_eq_typeargs (which calls is_eqtype and re-enters the subtype checker, causing infinite recursion when called from check_assume during union subtype checks).

  2. Add assumption stack check/push/pop to is_union_sub_x, so that when a union-as-subtype check recurs back to the same pair, it short-circuits instead of re-expanding.

  3. Rewrite check_assume to scan the assumption list directly instead of using a push-then-pop probe pattern, eliminating unnecessary ast_dup/ast_free allocations on every check.

Benchmarks on projects using recursive union types (json-ng):

  • http_server expr pass: 25.7s → 13.1s (-49%)
  • http_server full build: 31.7s → 15.2s (-52%)
  • No measurable regression on stdlib or other projects.

@SeanTAllen SeanTAllen added the changelog - fixed Automatically add "Fixed" CHANGELOG entry on merge label Mar 4, 2026
@ponylang-main ponylang-main added the discuss during sync Should be discussed during an upcoming sync label Mar 4, 2026
@SeanTAllen SeanTAllen marked this pull request as draft March 4, 2026 20:02
@SeanTAllen SeanTAllen force-pushed the fix-recursive-union-subtype-perf branch from f716ddb to b52d138 Compare March 4, 2026 23:20
@SeanTAllen SeanTAllen marked this pull request as ready for review March 4, 2026 23:21
@SeanTAllen SeanTAllen force-pushed the fix-recursive-union-subtype-perf branch 3 times, most recently from 2aeb937 to 8fd4acf Compare March 5, 2026 03:02
The subtype checker's assumption stack (which breaks cycles during
recursive type checks) only operated on nominal types. When a recursive
union type appeared as a field in its own members, the compiler would
re-expand the full union membership at every level of recursion, causing
exponential work in the expression pass.

Three changes fix this:

1. Generalize `exact_nominal` to `exact_type`, which can structurally
   compare any type AST node (unions, intersections, tuples, arrows,
   type params, capabilities, ephemeral markers) — not just nominals.
   This lets the assumption stack recognize compound type pairs.
   Importantly, `exact_type` compares nominal typeargs structurally
   rather than via `is_eq_typeargs` (which calls `is_eqtype` and
   re-enters the subtype checker, causing infinite recursion when
   called from `check_assume` during union subtype checks).

2. Add assumption stack check/push/pop to `is_union_sub_x`, so that
   when a union-as-subtype check recurs back to the same pair, it
   short-circuits instead of re-expanding.

3. Rewrite `check_assume` to scan the assumption list directly
   instead of using a push-then-pop probe pattern, eliminating
   unnecessary ast_dup/ast_free allocations on every check.

Benchmarks on projects using recursive union types (json-ng):
- http_server expr pass: 25.7s -> 13.1s (-49%)
- http_server full build: 31.7s -> 15.2s (-52%)
- No measurable regression on stdlib or other projects.
@SeanTAllen SeanTAllen force-pushed the fix-recursive-union-subtype-perf branch from 8fd4acf to 4f4f1db Compare March 5, 2026 03:19
@SeanTAllen
Copy link
Copy Markdown
Member Author

The numbers here are from before I had to change the approach to being more targeted as the first pass caused a crash. I will revisit this.

@SeanTAllen SeanTAllen marked this pull request as draft March 7, 2026 16:02
@SeanTAllen SeanTAllen removed the discuss during sync Should be discussed during an upcoming sync label Mar 7, 2026
@ponylang-main ponylang-main added the discuss during sync Should be discussed during an upcoming sync label Mar 7, 2026
@SeanTAllen SeanTAllen removed the discuss during sync Should be discussed during an upcoming sync label Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog - fixed Automatically add "Fixed" CHANGELOG entry on merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants