Skip to content

Commit 5faa3c3

Browse files
alexmalyshevmeta-codesync[bot]
authored andcommitted
Add an attribute inline-cache benchmark to cinderx/benchmarks
Summary: `LOAD_ATTR` and `STORE_ATTR` cost almost entirely depends on the *layout* of the receiver, where the attribute physically lives and what has to be checked to prove a cached answer is still valid. There was no benchmark isolating that, so `attr_cache.py` builds one receiver per distinct layout CPython 3.12, 3.14 and 3.15 can produce and hammers a separately generated hot function against each, one cache shape per workload. Storage layouts: `inline` (managed dict, values still inline), `split` (materialized dict sharing the type's keys), `combined` (dict detached and unshared), `exhausted` (shared keys full, name absent from them), `dictoffset` (non-managed heap type with a positive `tp_dictoffset`), `slots`, `slotsdict`, `varsize` (managed dict on a variable-sized type, which gets no inline values on 3.14+ but does on 3.12) and `afteritems` (`__slots__` on a `tuple` subclass, 3.15 only, skipped elsewhere). Lookup kinds that bypass instance storage: `classvar`, `property`, `descriptor`, `getset`, `method`, `typeattr`, `module`, `getattr`, `getattribute`. Polymorphic runs are separate workloads, `poly2` through `poly16` plus `polylayout` and `polytype`, so the megamorphic cliff past `CINDERX_JIT_ATTR_CACHE_SIZE` and the single-entry type-attribute cache both show up on their own lines. Receivers are passed in as an untyped `list`, which keeps the JIT from folding loads into raw field accesses and holds the sites on the inline-cache path being measured. Results are ns per attribute op, reported as the median of the timed runs. Three flags support verifying rather than just timing. `--describe` reports the layout each workload's receivers actually ended up with, read through the GC so the probe does not itself materialize a dict. `--specialization` reports which adaptive-interpreter opcode CPython settled on. Every result block prints whether the hot function was really JIT-compiled. `--cinderx` and `--compare` follow `inference_pipeline.py`. Also adds a `py_versions` argument to `cinderx_benchmark_binary` so this benchmark, whose only dep is click, can also generate a 3.15 variant for the 3.15-only layouts. `_cinderx` is not wired into `python_binary` for 3.15, so that variant measures the interpreter only and warns on stderr. Reviewed By: yoney Differential Revision: D114683513 fbshipit-source-id: 8967cf54df165ddd660d7a73cdb46cf09ff787d4
1 parent 8803001 commit 5faa3c3

2 files changed

Lines changed: 994 additions & 0 deletions

File tree

cinderx/benchmarks/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,38 @@ uv run python benchmarks/spectral_norm.py 3
3030

3131
The numeric argument controls the number of iterations (higher = longer run).
3232

33+
## Attribute Cache Benchmark
34+
35+
`attr-cache` hammers `LOAD_ATTR` and `STORE_ATTR` against one receiver per
36+
distinct object layout, so CinderX's inline caches can be measured against the
37+
CPython interpreter's adaptive specializations one cache shape at a time. There
38+
is a workload for each of inline values, split and combined dicts, exhausted
39+
shared keys, a fixed `tp_dictoffset`, `__slots__`, variable-sized types,
40+
properties and other descriptors, class/type/module attributes and the
41+
`__getattr__` / `__getattribute__` hooks, plus polymorphic and megamorphic
42+
variants of a single site.
43+
44+
```bash
45+
# One layout, plus which adaptive-interpreter specialization CPython chose:
46+
buck run fbcode//cinderx/benchmarks:attr-cache-314 -- --workload inline --specialization
47+
48+
# Baseline vs JIT across every workload:
49+
buck run fbcode//cinderx/benchmarks:attr-cache-314 -- --compare
50+
51+
# What layout each workload's receivers actually ended up with:
52+
buck run fbcode//cinderx/benchmarks:attr-cache-314 -- --describe
53+
```
54+
55+
Results are nanoseconds per attribute operation, reported as the median of the
56+
timed runs. Each run prints the receiver layout it measured and whether the hot
57+
function was actually JIT-compiled, so a suspicious number can be traced back to
58+
what really ran.
59+
60+
A `attr-cache-315` variant exists for the 3.15-only layouts, notably `__slots__`
61+
on a `tuple` subclass. The `_cinderx` native extension is not wired into
62+
`python_binary` for 3.15, so that variant measures the interpreter only and says
63+
so on stderr.
64+
3365
## JIT Compilation Time Benchmark
3466

3567
Measures how long the JIT takes to compile functions (not runtime performance):
@@ -130,6 +162,7 @@ CINDERJIT_DISABLE=1 uv run python benchmarks/runner.py
130162
| `richards` | Operating system task scheduler simulation (object-oriented workload) |
131163
| `spectral_norm` | Numerical computation of the spectral norm of a matrix |
132164
| `compile_time` | Measures JIT compilation speed (not runtime performance) |
165+
| `attr-cache` | LOAD_ATTR/STORE_ATTR against every receiver layout, monomorphic through megamorphic |
133166
| `fastmark` | Full pyperformance suite (~60 benchmarks) with CinderX integration |
134167
| `torchbench` | Run of a real TorchBench model (default `pyhpc_equation_of_state`), kept Python-bound for the JIT |
135168

0 commit comments

Comments
 (0)