Fix critical governance bypass and LLM classifier gaps - #35
Closed
requie wants to merge 2 commits into
Closed
Conversation
… LLM channel gap
Three related fixes to the runtime integrity path.
1. Untrusted tool arguments could shadow the fields governance matches
on. The pre_tool_use buffer entry spread the agent's own arguments
over the trusted fields, and that entry becomes context["action"], so
an argument named "tool" overwrote the real tool name: a
prompt-injected agent calling payment_execute with {"tool": "noop"}
skipped GOV-001's sensitive-tool gate entirely. Under enforce=True
the dangerous tool ran; in observe-only mode the signed attestation
recorded governance:pass for a check that never happened. The same
trick overrode "type". Arguments now nest under action["arguments"],
the shape the decision-record path already used, so no
agent-controlled key can collide with a trusted field. GOV-003 reads
the nested amount.
2. The LLM classifier skipped the two multi-agent channels. The regex
scanner covers seven channels; aevaluate fed only five to the
classifier, so shared_memory and broadcast_messages reached neither
strong detector (the regex taxonomy scores ~0 on the action-oriented
injections that travel through them). That left the cross-agent
cascade path covered only by the detector that cannot see it. Both
channels are now classified, using the regex scanner's content keys
and channel labels.
3. Classification cost is bounded. Calls ran sequentially with no
ceiling while the multi-agent buffers cap at 1000 entries per
channel. Calls now run with bounded concurrency and a per-evaluation
target ceiling; truncation is logged and reported so a capped scan
never reads as full coverage. Verdicts are cached per (channel,
content hash) with oldest-first eviction, and fail-open verdicts are
never cached since they record an outage rather than a judgment.
Adds regression tests for the bypass, both channel gaps, and every cost
bound. Full suite green (640 passed), ruff and mypy --strict clean.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
…w-arbrx0 # Conflicts: # CHANGELOG.md
This was
linked to
issues
Jul 25, 2026
requie
added
bug
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR closes two security gaps and adds cost controls to the adversarial LLM classifier:
Key Changes
Security Fixes
Tool argument shadowing (governance.py, base.py)
action["arguments"]instead of spreading flat into the action dict"tool"or"type"from overwriting the trusted fields that governance rules match onaction["arguments"]["amount"]test_governance_action_shape.py) validates the nesting and confirms the bypass is closedLLM classifier now covers multi-agent channels (adversarial_llm.py)
shared_memoryandbroadcast_messagesto the targets fed to the classifierCost Controls (adversarial_llm.py)
max_concurrency(default 8) limits in-flight API callsmax_targets_per_evaluation(default 200) prevents cost amplification on attacker-writable channelsdetails["llm_classifier"]["targets_dropped"]surfaces when a scan is capped, so it never reads as full coverageTest Coverage
summary/textfor shared memory,textfor broadcast)broadcast_channelsto match regex scanner)TestCostBounds) validates concurrency limits, target caps, verdict caching, and fail-open behaviorMigration Notes
Custom
PolicyRuleconditions that read tool-call arguments from the top level ofcontext["action"]must be updated to read fromaction["arguments"]. Built-in rules have been updated; this only affects custom policy code.Implementation Details
OrderedDictwith oldest-first eviction at the size limit(channel, sha256(text))to handle the same text on different channels as distinct questions_classify_targetsensures bounded parallelism