sync: measure what materialization costs in resident memory - #60
Merged
Conversation
An ignored measurement test, in the style of manifest_decode_cost. It is not a guard and it asserts nothing. Run it with: cargo test --release -- --ignored --nocapture materialize_resident_cost It exists because it REFUTED the hypothesis it was written to confirm. On 2026-08-19 the daemon showed 2.52 GB RSS with 2.2 GB resident and dirty in empty large-allocation regions, and I expected the 70.2 MB re-read per pass to be what retained it. It is not. 200 passes over three 23 MB files is 14 GB of churn and RSS does not move: corpus: 3 files, 70.0 MB RSS MB: start 75, after 200 re-reading passes 75, after 200 cached passes 75 So the read is not the retention driver, and the fixes in #56 and #57 should be weighted on CPU alone. The live behaviour was allocator retention of FREED pages. It was reclaimable: RSS fell from 2.52 GB to 709 MB within 90 s when an unrelated release build created memory pressure, with no restart. Why the daemon reaches a 2.5 GB high-water mark when this reproduction plateaus at 75 MB is UNRESOLVED. The untested hypothesis is per-thread large-block caches across the daemon's 14 threads against this single-threaded loop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An ignored measurement test, in the style of
manifest_decode_cost_before_and_after_a_sweep. It is not a guard and it asserts nothing.Why it exists
It refuted the hypothesis it was written to confirm.
On 2026-08-19 the live daemon showed 2.52 GB RSS, with 2.2 GB resident and dirty
in EMPTY large-allocation regions across 242 regions. I expected the 70.2 MB
re-read per materialize pass, the cost #56 removes, to be what retained it.
It is not:
14 GB of large-allocation churn moved RSS by zero. The allocator reuses the
block cleanly.
What that established
The merged fixes in #56 and #57 should be weighted on CPU alone. They do not
remove the memory behaviour, and without this measurement we would have deployed
expecting two benefits and received one.
The live behaviour was allocator retention of FREED pages, and it was
reclaimable. RSS fell from 2.52 GB to 709 MB within 90 seconds when an unrelated
release build created memory pressure, with no restart and no code change.
What is still unresolved
Why the daemon reaches a 2.5 GB high-water mark when this reproduction plateaus
at 75 MB.
The untested hypothesis is per-thread large-block malloc caches across the
daemon's 14 threads, against this single-threaded loop. That would explain both
the 242 regions and the plateau here. It is a question, not a conclusion, and
this test does not answer it.
Scope
One ignored test and nothing else. CI does not run it, because it is
#[ignore].It uses
tempfile::tempdir, so it touches no live state.The corpus is sized to the live
st2-declarations-defaultentry as observed on2026-08-19: three files of about 23 MB, 70.2 MB total.