Skip to content

perf: reuse verified reducer receipts within a session - #8

Open
NIU-123370 wants to merge 1 commit into
NVlabs:mainfrom
NIU-123370:niu/cache-verified-reducer-receipts
Open

perf: reuse verified reducer receipts within a session#8
NIU-123370 wants to merge 1 commit into
NVlabs:mainfrom
NIU-123370:niu/cache-verified-reducer-receipts

Conversation

@NIU-123370

Copy link
Copy Markdown

Problem

Repeated build/test runs can produce byte-identical diagnostic logs. Evidence-Preserving Reducer currently reuses the archived source file but still calls the reducer model for each occurrence, adding token usage and latency without new input evidence.

Change

Reuse completed, accepted reducer responses in a per-session, in-memory LRU cache capped at 64 entries. The key covers the source hash, command, error status, configured provider/model, output limit, receipt schema, reducer instructions, and storage root.

Cache hits still verify the source archive, revalidate every quotation, and rebuild the receipt for the current tool result (including fused write/edit output). Invalid, failed, or non-shrinking reductions are not cached. Hits record a distinct cache_hit journal event and zero new reducer usage, avoiding duplicate accounting of the original request.

This uses existing Pi extension APIs, adds no configuration or persistent files, and does not modify Pi core. The cache is lost on process restart. Concurrent first occurrences may still issue independent requests; only completed accepted results are reused.

Reproducible efficiency check

The new reduces three identical logs once and charges no model usage for cache hits test runs the reduction path with and without the cache against a mocked provider:

Three sequential identical logs Reducer requests
Without receipt cache 3
With receipt cache 1

Both cache hits retain verified evidence and report zero additional reducer tokens. This is a deterministic call-count measurement, not a paid-model benchmark or a claim about overall agent cost. Real savings depend on repeated identical logs, eviction, and provider billing.

Validation

  • npm ci --ignore-scripts
  • npm run check: TypeScript, all 17 test files / 149 tests, and package dry-run passed.
  • node scripts/check-pi-compat.mjs: passed.
  • npx vitest run tests/all-mechanisms.test.ts: 4/4 passed.
  • git diff --check: passed.
  • npm audit --audit-level=high: passed; the existing lockfile reports two moderate Vitest-related findings (GHSA-82fw-gwwq-j7x9). Dependencies were not changed.

The 15 added regression cases cover repeated output, zero usage on hits, changes to source/command/status/model/provider/output limit/instructions, session/restart isolation, failed or invalid/non-shrinking receipts, archive tampering, preservation of fused results, and LRU eviction.

@gaoanze888 gaoanze888 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the cache key, session isolation, LRU behavior, failure paths, archive revalidation, and fused-result reconstruction. I also ran npm run check locally: typecheck, 17 test files / 149 tests, and package dry-run all passed.

The key covers the source and reduction semantics, cache hits revalidate exact evidence against the archive, and failed/non-shrinking results are not retained. I did not find a blocking correctness issue. Nice focused optimization and clear documentation of the in-memory/session-local and concurrent-first-call trade-offs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants