Skip to content

Commit f321863

Browse files
anakrishCopilot
andcommitted
Introduce Object storage abstraction
Adds an opaque `Object` type that wraps the `BTreeMap<Value, Value>` backing `Value::Object`. The wrapper hides the inner map and exposes a curated method surface (get, insert, remove, iter, iter_sorted, cursor, serde), so future backends -- small-map inline, hash-backed, lazy, arena, FFI -- can swap in without call-site churn. This change is purely additive. `Value::Object` still wraps `Rc<BTreeMap<Value, Value>>`; the payload swap and call-site migration ship in the follow-up PR. `Object` is fully covered by its own tests but is not yet used by the engine. `docs/value/object.md` describes the design, enabled scenarios, known use cases (Azure Policy aliases and case-insensitive compare, SARIF small-object pressure, Kubernetes admission, cloud drift detection), and precedents (`serde_json::Map`, `toml::Table`, `simdjson` DOM, `indexmap`). The matching `Set` abstraction follows in a separate PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent acf7f7a commit f321863

6 files changed

Lines changed: 864 additions & 3 deletions

File tree

docs/value/object.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Object
2+
3+
Opaque container for `Value::Object`'s key→value storage, enabling
4+
alternative backends without call-site changes.
5+
6+
## Design
7+
8+
`Object` wraps the storage for a key→value collection of `Value`s and
9+
provides a curated set of methods (`get`, `insert`, `remove`, `iter`,
10+
`iter_sorted`, `cursor`, serde). The backing store is private; callers
11+
never see or pattern-match on it, so the representation can change
12+
without rippling through call sites.
13+
14+
Multiple backends can coexist at runtime. Because the backing store is
15+
private, different `Object` instances in the same process can use
16+
different implementations — e.g., a lazy DB-backed object for `input`,
17+
inline small-map objects for SARIF location records, and a regular
18+
sorted map elsewhere — all interoperating through the same opaque
19+
type. This is stronger than the typical Cargo-feature-selected backend
20+
seen in precedent crates.
21+
22+
Iteration is split intentionally. `iter()` makes no ordering promise,
23+
which lets backends that don't keep entries sorted skip any sort work.
24+
`iter_sorted()` returns entries in `Value` order and is what
25+
serialization and `Ord` rely on for deterministic output. Cursor types
26+
add resumable, incremental traversal for the RVM iteration state
27+
without leaking iterator internals.
28+
29+
`Ord` and `PartialOrd` are defined against `iter_sorted()` rather than
30+
derived from the storage. Two `Object`s built on different backends —
31+
or with different insertion histories — compare equal whenever their
32+
sorted entries match, so changing the backend never changes observable
33+
comparison results.
34+
35+
## Precedents
36+
37+
Other crates that hide storage behind a stable API so the implementation
38+
can change without breaking callers:
39+
40+
- **`serde_json::Map`** — opaque newtype allowing cargo-feature based
41+
swap between `BTreeMap` (canonical order) and `IndexMap` (insertion
42+
order).
43+
- **`toml::Table`** — opaque newtype allowing cargo-feature based swap
44+
between `BTreeMap` and `IndexMap`.
45+
- **`simdjson` DOM** — opaque tree that lazily materializes nodes on
46+
access instead of parsing the whole document up front.
47+
48+
## Use cases
49+
50+
- **SARIF small-object pressure** — SARIF reports contain millions of
51+
small objects (location records, rule references, message arguments),
52+
most with 2-5 keys. A small-map-optimized backend (inline storage
53+
for ≤N entries, heap above) eliminates per-object BTreeMap allocation
54+
for the common case.
55+
56+
- **Kubernetes admission policies** — large, deeply-nested resource
57+
objects (Pod specs, CRDs) where policies typically touch a handful
58+
of paths. A lazy-materializing backend (`LazyObjectProvider` over
59+
the incoming JSON) parses only the accessed subtrees.
60+
61+
- **Azure Policy aliases** — ARM exposes the same logical property
62+
under multiple aliases (e.g. paths like
63+
`Microsoft.Compute/virtualMachines/storageProfile.osDisk.managedDisk.id`).
64+
An alias-aware backend resolves lookups across canonical and alias
65+
forms without rewriting every policy.
66+
67+
- **Azure Policy case-insensitive compare** — ARM property names are
68+
case-preserving but case-insensitive on lookup (`tags.Environment`
69+
and `tags.environment` resolve identically). A case-insensitive
70+
backend centralizes this once at the storage layer instead of at
71+
every comparison site.
72+
73+
- **External data sources**`input` or `data` backed by a database
74+
query, CBOR slice, REST endpoint, or other streaming source via a
75+
`LazyObjectProvider`. Entries materialize on demand; the policy
76+
only pays for what it touches.
77+
78+
- **Eval-time temporaries** — objects constructed during evaluation
79+
(comprehensions, intermediate rule results) on a bumpalo arena.
80+
The whole arena drops at query end with zero per-entry free cost.
81+
82+
- **Host-language interop** — Python dicts or JS objects accessed via
83+
FFI callbacks from the embedding application, without copying into
84+
Rust on every binding boundary.

src/collections/mod.rs

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
// Copyright (c) Microsoft Corporation.
2+
// Licensed under the MIT License.
3+
4+
//! Storage abstraction for `Value::Object`.
5+
//!
6+
//! [`Object`] is an opaque wrapper around the current backing storage
7+
//! (`BTreeMap<Value, Value>`). Its inner field is private so that future
8+
//! storage representations (two-tier inline+hash, lazy, schema-shared,
9+
//! projection-aware) can swap in without touching call sites.
10+
//!
11+
//! ## Iteration order
12+
//!
13+
//! - [`Object::iter`] is **implementation-defined order** (mirroring
14+
//! `HashMap::iter`). Today it happens to iterate in sorted order because
15+
//! storage is BTree-backed, but callers MUST NOT depend on that.
16+
//! - [`Object::iter_sorted`] is **sorted by `Value::Ord`**. Use this whenever
17+
//! deterministic order is required (serialization, snapshots, hashing,
18+
//! `Debug`, the `object.keys` builtin, RVM↔interpreter parity).
19+
//!
20+
//! ## Resumable iteration
21+
//!
22+
//! Use [`Object::cursor`] / [`Object::next`] when callers must yield
23+
//! mid-iteration and resume later (e.g. the RVM `IterationState`). The
24+
//! cursor types are crate-internal so future storage variants can change
25+
//! their resume-state representation. Use plain [`Object::iter`] /
26+
//! [`Object::iter_sorted`] for one-shot consumption.
27+
28+
mod object;
29+
30+
#[cfg(test)]
31+
mod tests;
32+
33+
#[allow(unused_imports)] // surface for downstream PRs; cursor used internally in PR1b
34+
pub use object::{IntoIter, Iter, IterMut, Object};
35+
36+
#[cfg(feature = "rvm")]
37+
#[allow(unused_imports)] // surface for downstream PRs
38+
pub use object::ObjectCursor;

0 commit comments

Comments
 (0)