Skip to content

Commit 4912406

Browse files
anakrishCopilot
andcommitted
Introduce Object storage abstraction
Adds an opaque `Object` type that wraps the `BTreeMap<Value, Value>` backing `Value::Object`. The wrapper hides the inner map and exposes a curated method surface (get, insert, remove, iter, iter_sorted, cursor, serde), so future backends -- small-map inline, hash-backed, lazy, arena, FFI -- can swap in without call-site churn. This change is purely additive. `Value::Object` still wraps `Rc<BTreeMap<Value, Value>>`; the payload swap and call-site migration ship in the follow-up PR. `Object` is fully covered by its own tests but is not yet used by the engine. `docs/value/object.md` describes the design, enabled scenarios, known use cases (Azure Policy aliases and case-insensitive compare, SARIF small-object pressure, Kubernetes admission, cloud drift detection), and precedents (`serde_json::Map`, `toml::Table`, `simdjson` DOM, `indexmap`). The matching `Set` abstraction follows in a separate PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent acf7f7a commit 4912406

6 files changed

Lines changed: 837 additions & 4 deletions

File tree

docs/value/object.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Object
2+
3+
Opaque container for `Value::Object`'s key→value storage, enabling
4+
alternative backends without call-site changes.
5+
6+
## Design
7+
8+
`Object` wraps the storage for a key→value collection of `Value`s and
9+
provides a curated set of methods (`get`, `insert`, `remove`, `iter`,
10+
`iter_sorted`, `cursor`, serde). The backing store is private; callers
11+
never see or pattern-match on it, so the representation can change
12+
without rippling through call sites.
13+
14+
Multiple backends can coexist at runtime. Because the backing store is
15+
private, different `Object` instances in the same process can use
16+
different implementations — e.g., a lazy DB-backed object for `input`,
17+
inline small-map objects for SARIF location records, and a regular
18+
sorted map elsewhere — all interoperating through the same opaque
19+
type. This is stronger than the typical Cargo-feature-selected backend
20+
seen in precedent crates.
21+
22+
Iteration is split intentionally. `iter()` makes no ordering promise,
23+
which lets backends that don't keep entries sorted skip any sort work.
24+
`iter_sorted()` returns entries in `Value` order and is what
25+
serialization and `Ord` rely on for deterministic output. Cursor types
26+
add resumable, incremental traversal for the RVM iteration state
27+
without leaking iterator internals.
28+
29+
`Ord` and `PartialOrd` are defined against `iter_sorted()` rather than
30+
derived from the storage. Two `Object`s built on different backends —
31+
or with different insertion histories — compare equal whenever their
32+
sorted entries match, so changing the backend never changes observable
33+
comparison results.
34+
35+
## Precedents
36+
37+
Other crates that hide storage behind a stable API so the implementation
38+
can change without breaking callers:
39+
40+
- **`serde_json::Map`** — opaque newtype allowing cargo-feature based
41+
swap between `BTreeMap` (canonical order) and `IndexMap` (insertion
42+
order).
43+
- **`toml::Table`** — opaque newtype allowing cargo-feature based swap
44+
between `BTreeMap` and `IndexMap`.
45+
- **`simdjson` DOM** — opaque tree that lazily materializes nodes on
46+
access instead of parsing the whole document up front.
47+
48+
## Use cases
49+
50+
- **SARIF small-object pressure** — SARIF reports contain millions of
51+
small objects (location records, rule references, message arguments),
52+
most with 2-5 keys. A small-map-optimized backend (inline storage
53+
for ≤N entries, heap above) eliminates per-object BTreeMap allocation
54+
for the common case.
55+
56+
- **Kubernetes admission policies** — large, deeply-nested resource
57+
objects (Pod specs, CRDs) where policies typically touch a handful
58+
of paths. A lazy-materializing backend (`LazyObjectProvider` over
59+
the incoming JSON) parses only the accessed subtrees.
60+
61+
- **Azure Policy aliases** — ARM exposes the same logical property
62+
under multiple aliases (e.g. paths like
63+
`Microsoft.Compute/virtualMachines/storageProfile.osDisk.managedDisk.id`).
64+
An alias-aware backend resolves lookups across canonical and alias
65+
forms without rewriting every policy.
66+
67+
- **Azure Policy case-insensitive compare** — ARM property names are
68+
case-preserving but case-insensitive on lookup (`tags.Environment`
69+
and `tags.environment` resolve identically). A case-insensitive
70+
backend centralizes this once at the storage layer instead of at
71+
every comparison site.
72+
73+
- **External data sources**`input` or `data` backed by a database
74+
query, CBOR slice, REST endpoint, or other streaming source via a
75+
`LazyObjectProvider`. Entries materialize on demand; the policy
76+
only pays for what it touches.
77+
78+
- **Eval-time temporaries** — objects constructed during evaluation
79+
(comprehensions, intermediate rule results) on a bumpalo arena.
80+
The whole arena drops at query end with zero per-entry free cost.
81+
82+
- **Host-language interop** — Python dicts or JS objects accessed via
83+
FFI callbacks from the embedding application, without copying into
84+
Rust on every binding boundary.

src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -177,7 +177,7 @@ pub use utils::limits::{
177177
global_memory_limit, set_global_memory_limit, set_thread_flush_threshold_override,
178178
thread_memory_flush_threshold,
179179
};
180-
pub use value::Value;
180+
pub use value::{Object, Value};
181181

182182
/// Compiled-pattern caches for the `regex.*` and `glob.*` Rego builtins.
183183
///

src/scheduler.rs

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -569,7 +569,7 @@ impl Analyzer {
569569
}
570570
Ok(false)
571571
}
572-
Array { .. } | Object { .. } => Ok(true),
572+
Expr::Array { .. } | Expr::Object { .. } => Ok(true),
573573
_ => Ok(false),
574574
})?;
575575
Ok(true)
@@ -666,7 +666,7 @@ impl Analyzer {
666666
Ok(false)
667667
}
668668
// TODO: key vs value for object binding
669-
Array { .. } | Object { .. } => Ok(true),
669+
Expr::Array { .. } | Expr::Object { .. } => Ok(true),
670670
_ => Ok(false),
671671
})?;
672672
Ok(vars)
@@ -853,7 +853,7 @@ impl Analyzer {
853853
Ok(false)
854854
}
855855
// TODO: Object key/value
856-
Array { .. } | Object { .. } => Ok(true),
856+
Expr::Array { .. } | Expr::Object { .. } => Ok(true),
857857
_ => {
858858
non_vars.push(e.clone());
859859
Ok(false)

src/value.rs renamed to src/value/mod.rs

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,18 @@
1111
clippy::as_conversions
1212
)] // value helpers index paths directly for performance
1313

14+
mod object;
15+
16+
#[cfg(test)]
17+
mod tests;
18+
19+
#[allow(unused_imports)] // surface for downstream PRs; cursor used internally in PR1b
20+
pub use object::{IntoIter, Iter, IterMut, Object};
21+
22+
#[cfg(feature = "rvm")]
23+
#[allow(unused_imports)] // surface for downstream PRs
24+
pub use object::ObjectCursor;
25+
1426
use crate::number::Number;
1527

1628
use alloc::collections::{BTreeMap, BTreeSet};

0 commit comments

Comments
 (0)