feat: add bounded compaction semantic index - #476
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a2ca3b020
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fd308bd89
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review the current PR head e34de5c. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads. After two actionable rounds I completed a whole-diff invariant audit covering responsibility boundaries, exact provenance, revision tombstones, bounded memory and telemetry, cache-prefix stability, trace locality, subagent scoping, and the disabled path. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e34de5cb8b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review Please review the current PR head f1b038e. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads. Architecture changed after the prior review: |
|
@codex review Please review the current PR head 373d326. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads. The invariant closeout moved semantic extraction into the formatter branches that already parse tool inputs, eliminating duplicate JSON parsing; UI-only activity labels now short-circuit provider analysis, derived pending/oversized text is bounded before return, and the benchmark compares equivalent bounded separate versus one-pass projection. Please focus on provider parity, source-index mapping after tool filtering/summary slicing, and disabled-path cost. |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review Please review the current PR head 6119a4b. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads. This follow-up replaces oldest-only cap behavior with fixed-memory early/recent/type-balanced producer retention and compaction-only temporal/type-balanced character-budget selection. The common <=256-entry formatter path remains the original array-push path; balanced state initializes only after overflow. Please focus on bounded CPU/memory, revision and omission-count semantics, exact chronological output ordering, and disabled/common-path latency. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6119a4bd31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact current PR head af4b0a7. This follow-up addresses the prior suppression finding with bounded, collision-safe identity tracking across all semantic entry types. Please audit revision precedence, equal-revision conflict handling, bounded memory, retention/ring eviction, output ordering, and common-path performance. Ignore superseded heads. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: af4b0a7361
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact current PR head fdda1b8. This follow-up renews advanced/conflicting revisions in the bounded global and per-type recent rings, and aligns producer identity hashing/equality with renderer whitespace normalization. Please audit ring cursor/order behavior, reference counts and eviction, normalized collision handling, revision suppression, bounded memory, and performance. Ignore superseded heads. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fdda1b865e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact current PR head fe9c705. This follow-up normalizes equal-revision text exactly like the renderer before conflict detection and rejects present malformed activity revisions while preserving the absent legacy default. Please audit overflow/non-overflow semantic parity, fail-closed validation, tombstone behavior, regression coverage, and performance. Ignore superseded heads. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe9c705243
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact current PR head ce1e5bb. This follow-up converts malformed non-identity semantic state into bounded suppressing tombstones whenever identity and revision remain trustworthy, covering producer pending/status/text and snapshot/direct-render redaction/status/text paths. Please audit redaction safety, stale-revision suppression, malformed input behavior, legacy defaults, bounded data retention, and performance. Ignore superseded heads. |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Adds the Agents SDK Compaction Semantic Index in guidance-only mode and derives it during
formatAgentMessages' existing persisted-content analysis.The formatter can return bounded, source-addressed semantic guidance beside provider messages, summary, and token metadata. Hosts forward that result into
AgentInputs; no separate payload extraction pass is needed. Raw conversation messages remain complete and authoritative, and generated labels never replace evidence.One-pass Model Context Reconstruction
intentonly when the host supplies that tool's name incompactionSemanticIndex.intentToolNames; arbitrary businessintentarguments remain ordinary tool input.Request and cache shape
splitAtRecencyBoundaryselectsmessagesToRefine.Trust, lifecycle, and privacy boundaries
_sourceInputsretains only the bounded snapshot, never a rejected caller-owned index.Timing scope
This interface captures semantic state present during Model Context Reconstruction, before AgentContext construction. Same-run label ingestion would require a separate late-bound resolver or append seam with explicit timing and failure semantics.
Not included:
Performance
npm run bench:compaction-semantic-indexmeasures rendering and formatter projection with rotating samples.Standalone median run, 9 × 300 formatter projections over 32 tool-rich persisted messages producing 192 entries:
The enabled one-pass path is within ~1.2% (about 3 µs at 192 entries) of the equivalent separate projection while removing its extra payload pass and duplicate string-argument parse. The normal disabled formatter remains allocation-free for the semantic index and measured at parity with the current pre-strip path. Appendix rendering and source validation occur only when compaction fires (~23 µs for 48 exact entries in the same run).
Coverage-balanced bounds
Performance follow-up:
Verification
npx tsc --noEmitnpm run check:circular-depsnpm run buildnpm run bench:compaction-semantic-indexgit diff --checkNew derivation coverage includes provider-message parity, single-parse stringified tool intents, host intent classification, exact source coordinates, pending lifecycle, missing identity, summary-slice exclusion, fixed entry bounds, and bounded oversized tombstones.
Architecture
ADR 0007 and
CONTEXT.mdnow place extraction inside Model Context Reconstruction, where exact provenance already exists. LibreChat retains lifecycle, enablement, tool-intent classification, forwarding, and rollout ownership. The separate host extraction module is intentionally rejected as a shallow seam that duplicates traversal and coordinate logic.