Skip to content

Commit f8888ee

Browse files
anakrishCopilot
andcommitted
Introduce Object storage abstraction
Adds an opaque `Object` type that wraps the `BTreeMap<Value, Value>` backing `Value::Object`. The wrapper hides the inner map and exposes a curated method surface (get, insert, remove, iter, iter_sorted, cursor, serde), so future backends -- small-map inline, hash-backed, lazy, arena, FFI -- can swap in without call-site churn. This change is purely additive. `Value::Object` still wraps `Rc<BTreeMap<Value, Value>>`; the payload swap and call-site migration ship in the follow-up PR. `Object` is fully covered by its own tests but is not yet used by the engine. `docs/value/object.md` describes the design, enabled scenarios, known use cases (Azure Policy aliases and case-insensitive compare, SARIF small-object pressure, Kubernetes admission, cloud drift detection), and precedents (`serde_json::Map`, `toml::Table`, `simdjson` DOM, `indexmap`). The matching `Set` abstraction follows in a separate PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent acf7f7a commit f8888ee

6 files changed

Lines changed: 863 additions & 3 deletions

File tree

docs/value/object.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Object
2+
3+
Opaque container for `Value::Object`'s key→value storage, enabling
4+
alternative backends without call-site changes.
5+
6+
## Design
7+
8+
`Object` wraps the storage for a key→value collection of `Value`s and
9+
provides a curated set of methods (`get`, `insert`, `remove`, `iter`,
10+
`iter_sorted`, `cursor`, serde). The backing store is private; callers
11+
never see or pattern-match on it, so the representation can change
12+
without rippling through call sites.
13+
14+
Iteration is split intentionally. `iter()` makes no ordering promise,
15+
which lets backends that don't keep entries sorted skip any sort work.
16+
`iter_sorted()` returns entries in `Value` order and is what
17+
serialization and `Ord` rely on for deterministic output. Cursor types
18+
add resumable, incremental traversal for the RVM iteration state
19+
without leaking iterator internals.
20+
21+
`Ord` and `PartialOrd` are defined against `iter_sorted()` rather than
22+
derived from the storage. Two `Object`s built on different backends —
23+
or with different insertion histories — compare equal whenever their
24+
sorted entries match, so changing the backend never changes observable
25+
comparison results.
26+
27+
## Precedents
28+
29+
Other crates that hide storage behind a stable API so the implementation
30+
can change without breaking callers:
31+
32+
- **`serde_json::Map`** — opaque newtype allowing cargo-feature based
33+
swap between `BTreeMap` (canonical order) and `IndexMap` (insertion
34+
order).
35+
- **`toml::Table`** — opaque newtype allowing cargo-feature based swap
36+
between `BTreeMap` and `IndexMap`.
37+
- **`simdjson` DOM** — opaque tree that lazily materializes nodes on
38+
access instead of parsing the whole document up front.
39+
40+
## Scenarios enabled
41+
42+
- **Hash-backed storage** — for policies where keys aren't compared
43+
ordinally; swap to FxHashMap-backed inner without touching call sites.
44+
- **Lazy/streaming** — wrap a `LazyObjectProvider` (DB query, CBOR slice,
45+
REST endpoint) and materialize entries on demand.
46+
- **Arena allocation** — bumpalo-backed inner for eval-time temporaries;
47+
drop the whole arena at query end with zero per-entry free cost.
48+
- **FFI-backed** — host-language callbacks (Python dict, JS object) without
49+
copying into Rust.
50+
- **Small-map optimization** — inline storage for ≤N entries, heap above;
51+
eliminates per-object BTreeMap heap allocation for the common case.
52+
53+
## Known use cases
54+
55+
- **Azure Policy aliases** — policy authors write paths like
56+
`Microsoft.Compute/virtualMachines/storageProfile.osDisk.managedDisk.id`;
57+
the same logical property is exposed under multiple aliases by ARM.
58+
An alias-aware Object backend resolves lookups across canonical and
59+
alias forms without rewriting every policy.
60+
- **Azure Policy case-insensitive compare** — resource property names in
61+
ARM are case-preserving but case-insensitive on lookup
62+
(`tags.Environment` vs `tags.environment` resolve identically). A
63+
case-insensitive Object backend implements this once at the storage
64+
layer instead of every comparison site in policies.
65+
- **SARIF small-object pressure** — SARIF reports contain millions of
66+
small objects (location records, rule references, message arguments),
67+
most with 2-5 keys. A small-map-optimized backend eliminates per-object
68+
BTreeMap heap allocation for the common case.
69+
- **Kubernetes admission policies** — large, deeply-nested resource
70+
objects (Pod specs, CRDs) where policies typically touch a handful
71+
of paths. A lazy-materializing Object backend parses only accessed
72+
subtrees from the incoming JSON.
73+
- **Cloud config drift detection** — comparing current vs desired
74+
resource state requires structural equality that's tolerant of
75+
key-order differences and provider-specific casing. Centralizing this
76+
in the Object backend keeps policies portable across cloud providers.
77+
78+
## Notes
79+
80+
Cursor types are `pub` (referenced by public `IterationState`) but not
81+
re-exported at the crate root. Future Array and String abstractions
82+
follow the same shape — see `docs/value/array.md` and `docs/value/string.md`
83+
when they land.

src/collections/mod.rs

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
// Copyright (c) Microsoft Corporation.
2+
// Licensed under the MIT License.
3+
4+
//! Storage abstraction for `Value::Object`.
5+
//!
6+
//! [`Object`] is an opaque wrapper around the current backing storage
7+
//! (`BTreeMap<Value, Value>`). Its inner field is private so that future
8+
//! storage representations (two-tier inline+hash, lazy, schema-shared,
9+
//! projection-aware) can swap in without touching call sites.
10+
//!
11+
//! ## Iteration order
12+
//!
13+
//! - [`Object::iter`] is **implementation-defined order** (mirroring
14+
//! `HashMap::iter`). Today it happens to iterate in sorted order because
15+
//! storage is BTree-backed, but callers MUST NOT depend on that.
16+
//! - [`Object::iter_sorted`] is **sorted by `Value::Ord`**. Use this whenever
17+
//! deterministic order is required (serialization, snapshots, hashing,
18+
//! `Debug`, the `object.keys` builtin, RVM↔interpreter parity).
19+
//!
20+
//! ## Resumable iteration
21+
//!
22+
//! Use [`Object::cursor`] / [`Object::next`] when callers must yield
23+
//! mid-iteration and resume later (e.g. the RVM `IterationState`). The
24+
//! cursor types are crate-internal so future storage variants can change
25+
//! their resume-state representation. Use plain [`Object::iter`] /
26+
//! [`Object::iter_sorted`] for one-shot consumption.
27+
28+
mod object;
29+
30+
#[cfg(test)]
31+
mod tests;
32+
33+
#[allow(unused_imports)] // surface for downstream PRs; cursor used internally in PR1b
34+
pub use object::{IntoIter, Iter, IterMut, Object};
35+
36+
#[cfg(feature = "rvm")]
37+
#[allow(unused_imports)] // surface for downstream PRs
38+
pub use object::ObjectCursor;

0 commit comments

Comments
 (0)