Conversation
Lens-based execution leaves VM record fields (contract storage maps, keccak preimages, constraints, logs) as thunk chains referencing every intermediate VM of a replayed sequence. Storing such a VM -- in EchidnaTest.vm on falsification/optimization progress and on every successful shrink step -- retains the whole execution history until the field is demanded, which reporting only does once at campaign end. The interactive UI's periodic test snapshots observe these chains continuously, where they accounted for hundreds of MB of standing heap (retainer profiling: addAliasConstraints/accessUnboundedMemory sets; forcing cut the standing Map.Bin census band from 298 MB to 63 MB on a 20-failing-test benchmark). Add forceVMData to collapse the accumulating fields when a VM is about to be stored, and force NewCoverage.transactions so queued/retained events no longer pin each sequence's VMResults and returndata through the `fst <$> results` thunk. Co-Authored-By: Claude <noreply@anthropic.com>
The Brick UI retained large amounts of heap on long campaigns; with 20 always-failing tests the same workload that holds a flat 12.9 MB in --format text reached 915 MB live (4.2 GB peak residency) and kept growing for the whole interactive session. Several compounding causes, each measured with heap censuses and cost-centre/retainer profiles: - The 200 ms ticker queued CampaignUpdated events carrying full [EchidnaTest]/[WorkerState] snapshots; with a slow consumer the bounded-at-1000 channel could pin up to a thousand snapshot generations. Publish the snapshot through an always-latest IORef and send a payload-free CampaignTick instead (stale ticks drop harmlessly). - The tests pane was rebuilt on every tick, allocating ppTx/trace pretty-printing structures per test per 200 ms which accumulate; with --shrink-limit 0 this leaked 1.6 MB/s indefinitely (527 MB after 6 min vs 35 MB after this change). Cache the pane keyed on a cheap (state, value) digest and rate-limit rebuilds to once per second. - Widget and state structures captured test records (and their VM snapshots) through unevaluated show/format thunks; force the rendered strings and store a VM-free copy of the tests in UIState. - The log pane history was an unbounded Seq, also rendered with an O(n) fold per frame; cap it at 5000 entries. - Sever the previous campaignWidget before building the new one and replace the state strictly (WHNF) so superseded generations don't chain through widget closures or lazy modify. A residual remains on the always-failing stress workload: data created during the falsification/shrink burst (~700 MB) stays reachable through closures inside brick's customMainWithVty event loop (retainer sets "(N)customMainWithVty,m..."), surviving all application-side stripping; pinning it down needs ghc-debug-grade tooling. Typical campaigns with few failing tests are unaffected by that residual. Co-Authored-By: Claude <noreply@anthropic.com>
The interactive (Brick/vty) UI retained one UIState generation per rendered
frame, unbounded: vty parks the last Picture in an IORef (lastPicRef) for
resize redraws, and Brick builds that Picture lazily -- its image/viewport
nodes are renderFinal/viewportScroll thunks closing over the UIState they
were rendered from, which through the prior render closes over the previous
UIState. The single unforced Picture is thus a thunk chain reaching back
through every frame, pinning each generation's campaigns/WorkerState/GenDict
and (on falsifying contracts) its test VM snapshots.
ghc-debug on a live process showed a single retainer path with 788 chained
renderFinal/viewportScroll thunk pairs off lastPicRef. Measured: +12 kB/s
unbounded on a trivial non-falsifying contract (interactive), and 681 MB
pinned on a 20-failing-assert stress contract.
Fixes:
- UI.hs: deep-force each Picture before handing it to vty, so lastPicRef
holds concrete images instead of a UIState-closing thunk chain (primary).
- UI.hs: force the VM-stripped test copies; `t { vm = Nothing }` was a thunk
retaining the VM-bearing original.
- Campaign.hs: bind NewCoverage.transactions strictly; `force (...)` on a
non-strict field was itself a thunk pinning the sequence's VMResults.
- Exec.hs: forceVMData now also forces vm.tx.subState, whose lazily-consed
Expr EAddr elements otherwise chain a stored VM to every intermediate VM
of its transaction.
Validated: trivial-contract interactive growth +12,071 -> +517 B/s (UIState
generation growth +4.1 MB -> +0); stress-contract live heap 681 -> 39 MB
flat; ghc-debug generation chain 788 -> 0.
updateOpenTest binds result = getResultFromVM vm' lazily; EchidnaTest.result is a lazy field, so every falsified or optimized test stored in testRefs -- and every VM-free copy pushed as a TestFalsified/TestOptimized event -- retained the unevaluated thunk and with it the full post-checkETest VM, one extra VM snapshot per test. Force the result in the two branches that store it. The binding itself must stay lazy: on the fall-through path checkETest can return a VM with no result, and forcing there crashes the worker (getResultFromVM calls error). Same hygiene bang in shrinkTest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.