Proposal: Pre-deployment system prompt defense audit

[ppcvote]

Context

The Agent Governance Toolkit provides runtime governance for AI agents. A complementary layer would be pre-deployment system prompt auditing — checking whether agent prompts include adequate defenses before they're deployed.

The gap

We scanned 257 production system prompts from major AI tools (ChatGPT, Claude, Grok, Cursor, v0, Copilot) and found:

Defense	Gap Rate
Indirect Injection	94.9%
Role Boundary	84.4%
Harmful Content	72.4%
Social Engineering	42.4%

Average defense score: 49/100. Full data (n=1,646 prompts from 4 public datasets): research results

Proposal

A policy evaluator that validates agent system prompts against defense patterns before deployment:

from agent_governance.evaluators import PromptDefenseEvaluator

evaluator = PromptDefenseEvaluator()
result = evaluator.evaluate(agent_config.system_prompt)
# result.score = 49, result.missing = ['indirect-injection', 'role-boundary', ...]
# result.compliance = {'OWASP_LLM01': 'FAIL', 'OWASP_LLM06': 'PASS', ...}

This complements the runtime governance by catching weak prompts before they enter production.

Implementation

The scanner is open source and on npm: prompt-defense-audit (MIT, zero deps, <5ms). The regex patterns could be ported to Python or called via subprocess.

Happy to contribute a PR if this direction aligns with the toolkit's roadmap.

[github-actions]

Welcome to the Agent Governance Toolkit! Thanks for opening your first issue.
A maintainer will review this shortly. Check our Contributing Guide.
For security issues, use private vulnerability reporting.

[github-actions]

🤖 AI Agent: contributor-guide

Hi @user! 👋

Welcome to the Agent Governance Toolkit project! 🎉 We're thrilled to have you here, and thank you for taking the time to open such a well-thought-out issue. Your proposal is both insightful and highly relevant to the mission of this toolkit — ensuring robust governance for AI agents.

🔍 Issue Analysis

Your idea of a pre-deployment system prompt defense audit is fantastic! It addresses a critical gap in the AI governance lifecycle by proactively identifying vulnerabilities in system prompts before they are deployed. This aligns well with the goals of the toolkit, especially in the areas of compliance and reliability.

Based on your description, this proposal seems most relevant to the following packages:

• agent-compliance: This package focuses on compliance frameworks and audit logging. A pre-deployment prompt defense audit would fit naturally here, as it involves validating system prompts against security and compliance standards.
• agent-sre: Since this proposal enhances the reliability of deployed agents by reducing vulnerabilities, it could also tie into the reliability and chaos testing aspects of this package.

📂 Relevant Code Areas

To get started, you might want to explore the following directories:

1. agent-compliance/: This is where compliance-related utilities and frameworks are implemented. You could add the PromptDefenseEvaluator class here or extend existing compliance checks.
2. agent-sre/: If you see a strong connection to reliability or chaos testing, you might consider adding complementary functionality here.

📚 Helpful Resources

Here are some resources to help you get started:

• CONTRIBUTING.md: This guide will walk you through the contribution process, including setting up your development environment and submitting a PR.
• QUICKSTART.md: A quick guide to get up and running with the toolkit.
• Code of Conduct: Please review this to ensure a welcoming and inclusive environment for everyone.

🚀 Next Steps

Here’s how you can move forward:

1. Discuss Alignment: Let us know if you’d like to move forward with this idea. If so, we can discuss how best to integrate it into the toolkit (e.g., as part of agent-compliance or elsewhere).
2. Explore Existing Code: Familiarize yourself with the relevant directories mentioned above to understand the current structure and where your contribution might fit.
3. Draft a Design: Share a high-level design for how you envision the PromptDefenseEvaluator working. For example:
   • Will it be a standalone module or integrated into an existing workflow?
   • How will it handle configuration for different compliance standards (e.g., OWASP)?
   • How will it report results (e.g., logs, metrics, or alerts)?
4. Open a Draft PR: Once you’re ready, feel free to open a draft pull request. This will allow us to provide feedback as you work on the implementation.

We’re excited to see where this goes and are here to support you every step of the way. Let us know if you have any questions or need guidance. Thank you again for contributing to the project — your work could make a big difference in improving AI agent governance! 🚀

Looking forward to collaborating with you! 😊

[jagmarques]

Pre-deployment prompt defense auditing is valuable. Another angle worth considering: making the audit results themselves cryptographically signed so there is a verifiable record that the defense check actually ran and passed before deployment. We do this in asqav with ML-DSA signatures on compliance check results.

[ppcvote]

@jagmarques — great point. Cryptographically signed audit results would close the gap between "we ran the check" and "we can prove we ran the check." That's especially relevant for compliance-heavy deployments where audit trails need to be tamper-evident.

The current prompt-defense-audit output is a deterministic JSON object (same input always produces same output), which makes it naturally suited for signing — the result is reproducible and verifiable. Adding an ML-DSA or Ed25519 signature envelope around the AuditResult JSON would be a clean integration point.

I'll look into asqav's signing approach. Would be interested in aligning on a shared schema for signed compliance check results if that's something the toolkit would benefit from.

[imran-siddique]

Great discussion @ppcvote @jagmarques! The cryptographically signed audit angle connects well with our existing work:

• Audit chain integrity — we recently added SHA-256 hash chain verification to DeltaEngine, FlightRecorder, and RogueAgentDetector (fix(security): resolve audit chain integrity defects #770)
• Annex IV exporter — structured compliance evidence export (feat(agent-mesh): add Annex IV technical documentation exporter #782)
• MerkleAuditChain — tamper-evident audit in agent-mesh

A pre-deployment prompt defense audit that produces signed evidence would fit naturally as an additional check in the compliance verification pipeline. Interested in contributing a proposal?

[ppcvote]

@imran-siddique — yes, interested in contributing a proposal.

The integration path you described makes sense. Our scanner (prompt-defense-audit) produces a deterministic JSON result — same input always yields the same output, which maps cleanly to AuditEntry in the MerkleAuditChain:

• event_type: prompt_defense_audit
• action: pre_deployment_check
• outcome: pass/fail based on grade threshold
• data: the full audit result (grade, score, 12 per-vector checks with evidence)

Since the output is a canonical JSON object with stable field ordering, the SHA-256 hash is reproducible — anyone can re-run the scan and verify the hash matches what's in the chain.

Where I see this fitting:

1. Pre-deployment gate in agent-compliance — a PromptDefenseEvaluator that scans system prompts before agent deployment, produces a signed AuditEntry, and appends it to the chain. Fails promotion if the prompt scores below a configurable threshold.

2. Evidence for Annex IV export — the per-vector breakdown (12 checks with confidence scores and evidence strings) could feed into Section 4 (risk management measures) of the TechnicalDocumentationExporter.

Before writing code, a few questions:

• Should this live in agent-compliance alongside integrity.py and promotion.py, or in agent-mesh/governance/ closer to the audit chain?
• Is there a preferred pattern for evaluators? I see lint_policy.py and verify.py — should this follow a similar interface?
• For the signing envelope, should I use the existing compute_hash() canonical JSON approach, or is there a move toward ML-DSA as @jagmarques suggested?

Happy to draft a design doc or go straight to a draft PR — whichever the team prefers.

[imran-siddique]

Assigned! Looking forward to the proposal @ppcvote. For the integration points you mentioned, check out:

• \�gent_compliance.verify.GovernanceVerifier\ for the compliance check pipeline
• \�gent_compliance.integrity.IntegrityVerifier\ for hash-based verification
• The \ComplianceReport\ model in agent-mesh for structured evidence output

Feel free to open a draft PR early — we can iterate on the design together.