perf: reuse verified reducer receipts within a session - #8
Open
NIU-123370 wants to merge 1 commit into
Open
Conversation
gaoanze888
approved these changes
Sep 11, 2026
gaoanze888
left a comment
There was a problem hiding this comment.
Reviewed the cache key, session isolation, LRU behavior, failure paths, archive revalidation, and fused-result reconstruction. I also ran npm run check locally: typecheck, 17 test files / 149 tests, and package dry-run all passed.
The key covers the source and reduction semantics, cache hits revalidate exact evidence against the archive, and failed/non-shrinking results are not retained. I did not find a blocking correctness issue. Nice focused optimization and clear documentation of the in-memory/session-local and concurrent-first-call trade-offs.
This was referenced Sep 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Repeated build/test runs can produce byte-identical diagnostic logs. Evidence-Preserving Reducer currently reuses the archived source file but still calls the reducer model for each occurrence, adding token usage and latency without new input evidence.
Change
Reuse completed, accepted reducer responses in a per-session, in-memory LRU cache capped at 64 entries. The key covers the source hash, command, error status, configured provider/model, output limit, receipt schema, reducer instructions, and storage root.
Cache hits still verify the source archive, revalidate every quotation, and rebuild the receipt for the current tool result (including fused write/edit output). Invalid, failed, or non-shrinking reductions are not cached. Hits record a distinct
cache_hitjournal event and zero new reducer usage, avoiding duplicate accounting of the original request.This uses existing Pi extension APIs, adds no configuration or persistent files, and does not modify Pi core. The cache is lost on process restart. Concurrent first occurrences may still issue independent requests; only completed accepted results are reused.
Reproducible efficiency check
The new
reduces three identical logs once and charges no model usage for cache hitstest runs the reduction path with and without the cache against a mocked provider:Both cache hits retain verified evidence and report zero additional reducer tokens. This is a deterministic call-count measurement, not a paid-model benchmark or a claim about overall agent cost. Real savings depend on repeated identical logs, eviction, and provider billing.
Validation
npm ci --ignore-scriptsnpm run check: TypeScript, all 17 test files / 149 tests, and package dry-run passed.node scripts/check-pi-compat.mjs: passed.npx vitest run tests/all-mechanisms.test.ts: 4/4 passed.git diff --check: passed.npm audit --audit-level=high: passed; the existing lockfile reports two moderate Vitest-related findings (GHSA-82fw-gwwq-j7x9). Dependencies were not changed.The 15 added regression cases cover repeated output, zero usage on hits, changes to source/command/status/model/provider/output limit/instructions, session/restart isolation, failed or invalid/non-shrinking receipts, archive tampering, preservation of fused results, and LRU eviction.