Performance Triage

Performance Regression Triage

Use this checklist when nir-check, benchmarks, or interactive CLI runs get slower or time out without obvious semantic failures.

1. Establish scope

• Single binary vs corpus: Corpus regressions often indicate manifest-wide hotspots.
• Platform: macOS vs Linux vs Windows timers differ—confirm apples-to-apples (release builds).

2. Freeze toolchain inputs

Capture:

• rustc --version, commit SHA, feature flags if any.
• Exact CLI invocation (decomp, inventory, automation lane args).
• Binary fingerprint (SHA256) and manifest entry (benchmark/config/).

3. Separate compile-time vs runtime

Symptom	Likely bucket
CI spends minutes in cargo build	Compile/link regression—inspect dependency graph changes
CLI hangs after binary load	Analysis-phase regression—profile hot crates (fission-pcode, loaders)

Use cargo build -p fission-cli --release timings locally as a coarse compile signal.

4. Benchmark harness hygiene

Python benchmarks depend on GHIDRA_INSTALL_DIR, pyghidra, and pinned Ghidra vendor paths—misconfiguration often masquerades as slowness (timeouts vs correctness).

Review benchmark/full_benchmark/README.md when interpreting wall-clock deltas.

5. Automation lanes

nir-check exposes structured statistics aligned with NirBuildStats—compare deltas across commits rather than eyeballing pseudocode alone.

See NIR-check and CI.

6. Reporting

Include:

• Before/after commits or tags.
• Representative manifests (smoke_corpus.json snippets).
• Hardware basics (CPU generation, thermal headroom when laptop-throttled).

Related wiki pages

• Automated checks overview
• Full benchmark interpretation

Related repository docs

• benchmark/full_benchmark/README.md
• crates/fission-automation/AGENTS.md