Add Frozen object backend for reduced memory footprint - #66
Open
anakrish wants to merge 8 commits into
Open
Conversation
Adds tests/sarif_memory.rs, a single-binary integration test that installs a custom global allocator to measure live, peak, total_alloc, and nalloc per phase of a realistic SARIF policy evaluation: - load policy - Value::from_json_str - drop the source JSON - Engine::set_input - Engine::eval_query Fixtures (tests/data/sarif_memory/): - input.json: SARIF v2.1.0 sample, ~378 KB - policy.rego: 85-line compliance check used by membench The dataset can be amplified via MULT env var (default 1 for CI speed; MULT=50 reproduces the customer workload at 6.6 MiB JSON input). On upstream/main (BTreeMap-backed Object) at MULT=50: Value::from_json_str cost: live +102.3 MiB (15.4x JSON size) peak during run: 102.3 MiB The test is gated off when the regorus 'mimalloc' feature is enabled (its custom global allocator would conflict). Provides a permanent regression guardrail for the storage abstraction and a baseline against which alternative Object representations (SmallMap, boxed-slice, etc.) can be measured apples-to-apples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replaces the single `BTreeMap<Value, Value>` backing storage with a
two-variant enum:
- `Inline`: sorted, deduplicated `SmallVec<[(Value, Value); 4]>`
- `BTree`: `BTreeMap<Value, Value>` (used after the inline buffer
is full and a new key needs to be inserted)
Inline capacity is 4. SARIF workloads (and Rego objects generally)
average ~1.65 fields per object, so the inline tier covers the vast
majority of allocations without a heap node.
All Object methods route through the repr enum; PartialEq/Eq/Ord/Debug
are defined in terms of sorted iteration so they are oblivious to the
backend variant.
Measured on tests/sarif_memory.rs (MULT=50, 6.6 MiB SARIF input):
BTreeMap SmallMap reduction
live after parse 94.3 MiB 39.2 MiB 2.4x
peak 102.3 MiB 47.2 MiB 2.2x
A separate A/B with hashbrown::HashMap as the spill backend produced
byte-identical numbers on this workload, confirming the win comes
entirely from the inline tier.
All 228 lib tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a Frozen boxed-slice Object representation and thaw mutation paths back to Inline or BTree. Box the mutable Inline builder so frozen objects do not retain the SmallVec inline-buffer size in the Object enum. SARIF MULT=50: live after parse 32.313 MiB, live after drop JSON 24.313 MiB, peak 32.315 MiB. Object variants after parse: Inline=0 Frozen=160559 BTree=0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Measurements rerun for SARIF MULT=50, corpus MULT=1, and twitter/synthea MULT=10; no source changes required. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a
Frozenrepresentation forObjectstorage that uses a compactBox<[(K, V)]>instead ofBTreeMap, reducing memory footprint substantially for fully-materialized objects.Memory impact
Live memory measurements across canonical JSON benchmark fixtures:
Design
Object::Reprbecomes an enum with four variants:Empty— zero-cost no allocation for empty objectsInline(Box<SmallVec<[(K, V); 2]>>)— for tiny objects (≤2 entries)Frozen(Box<[(K, V)]>)— compact, sorted, deduplicated; used for fully-materialized immutable-shape objectsBTree(BTreeMap<K, V>)— fallback for actively-mutated objectsFrozen is the default for objects constructed via parsing/serialization. Mutation (insert/remove) thaws to BTree on demand. Value-only mutation on existing keys preserves Frozen.
Where Frozen kicks in
freeze_recursiveruns at value-construction boundaries (JSON parse, value builder paths insrc/value/mod.rs) to convert fully-materialized subtrees into the compact representation.Behavior
tests/corpus_memory.rsandtests/sarif_memory.rsmeasure live memory across fixtures.Objectremains a black box.object_storage_variant_for_memory_diagnostics(&Object) -> &'static strfor testing/diagnostics.Commits
Gates
Follow-ups (separate PRs)
object-mem-lazy) — adds ~3-4× further reduction by deferring child materialization.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com