Skip to content

Fix segfault when using Generator.map with PonyCheck shrinking#5006

Open
SeanTAllen wants to merge 1 commit intomainfrom
fix-reify-typeparamref-nested-lambdas
Open

Fix segfault when using Generator.map with PonyCheck shrinking#5006
SeanTAllen wants to merge 1 commit intomainfrom
fix-reify-typeparamref-nested-lambdas

Conversation

@SeanTAllen
Copy link
Copy Markdown
Member

@SeanTAllen SeanTAllen commented Mar 10, 2026

When a lambda is nested inside an object literal inside a generic method, collect_type_params copies type parameters twice, producing an ast_data chain of length > 1. reify_typeparamref followed this chain only one level, so the doubly-copied type params never matched. This caused the lambda's apply method to not be marked as reachable during codegen, leaving a hole in the vtable and segfaulting at runtime.

The fix follows the ast_data chain to the root (the self-referential node set during the scope pass) for both sides of the comparison, handling any depth of nesting.

The same single-level pattern in reify_reference is fixed defensively — investigation showed that code path is not currently reachable in the nested lambda scenario, but the pattern is structurally vulnerable.

Closes #4838
Closes #5005

@SeanTAllen SeanTAllen added the changelog - fixed Automatically add "Fixed" CHANGELOG entry on merge label Mar 10, 2026
@ponylang-main ponylang-main added the discuss during sync Should be discussed during an upcoming sync label Mar 10, 2026
When a lambda is nested inside an object literal inside a generic
method, collect_type_params copies type parameters twice, producing
an ast_data chain of length > 1 (copy'' → copy' → original).
reify_typeparamref followed this chain only one level, so the
doubly-copied type params never matched. This caused the lambda's
apply method to not be marked as reachable during codegen, leaving
a hole in the vtable and segfaulting at runtime.

The fix follows the ast_data chain to the root (the self-referential
node set during the scope pass) for both sides of the comparison,
handling any depth of nesting.

The same single-level ast_data pattern existed in reify_reference.
While investigation showed that code path is not currently reachable
in the nested lambda scenario (TK_REFERENCE nodes are resolved by
the refer pass before any reify call encounters them), the pattern
is structurally vulnerable. Applied the same chain-following fix
defensively, guarded by a TK_TYPEPARAM check on ref_def since scope
lookup can return other node types.

Closes #4838
Closes #5005
@SeanTAllen SeanTAllen force-pushed the fix-reify-typeparamref-nested-lambdas branch from 2007d9b to 5c4a9aa Compare March 10, 2026 23:34
@SeanTAllen
Copy link
Copy Markdown
Member Author

Hit this in ponylang/corral while writing property-based tests for a DEFLATE BitReader. The test used Generators.map2 to build a 4-byte Array[U8] val and segfaulted on execution (not during shrinking — the property itself never failed, it crashed before completing the first run).

Original property test that segfaults on ponyc 0.62.0:

use "pony_check"
use "pony_test"

class _PropertyBitReaderSplit is Property1[Array[U8] val]
  fun name(): String => "inflate/property/bit-reader-split"

  fun gen(): Generator[Array[U8] val] =>
    Generators.map2[U8, U8, Array[U8] val](
      Generators.u8(),
      Generators.u8(),
      {(a, b) => recover val
        let arr = Array[U8](4)
        arr.push(a)
        arr.push(b)
        arr.push(a xor b)
        arr.push(a + b)
        arr
      end })

  fun property(sample: Array[U8] val, h: PropertyHelper) =>
    if sample.size() < 4 then return end
    // ... reads bits from sample, no assertions that would fail ...

Worked around by converting to a manual UnitTest with hardcoded byte arrays. Will convert back to a property test once this fix lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog - fixed Automatically add "Fixed" CHANGELOG entry on merge discuss during sync Should be discussed during an upcoming sync

Projects

None yet

2 participants