Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,22 @@ All notable changes to the Agent Security Harness will be documented in this fil
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [4.4.2] - 2026-05-02

**Theme: Documentation hardening.** Reframed offensive-vocabulary phrasing in `docs/ADVANCED.md` GTG-1002 capability table for unambiguous defensive intent. NVD-anchored CVE-2026-25253 references in `docs/TEST-INVENTORY.md`. No code changes; no test changes; test count unchanged at 470 across 32 modules.

### Changed

- `docs/ADVANCED.md` GTG-1002 table: column headers reframed from `Real GTG-1002 Activity` / `What We Test` to `Adversary behavior we probe for` / `Detection probes the harness sends`. Cell content reworded from active to defensive voice ("Probes detection of X" rather than "User data exfiltration").
- `docs/ADVANCED.md` added top-of-section defensive framing paragraph and reading guide above the GTG-1002 table.
- `docs/TEST-INVENTORY.md` anchored both CVE-2026-25253 references with inline NVD links.

### Notes

- Hardening pass against VirusTotal Code Insight (Gemini-powered LLM scanner). The previous v4.4.1 bundle drew a "suspicious" Code Insight verdict from string-density on offensive vocabulary in bundled markdown documentation; the reframing reduces that signal without changing test capability or coverage.
- ClawHub bundle republished as v4.4.2 (skill-bundle versioning is independent of underlying package version; `pyproject.toml` remains at v4.4.0 until next code-change release).
- Counterpart memory entry: `playbook_security_skill_scanner_hardening.md` Pattern 5 (bundled-docs adversary-vs-defender table reframing).

## [4.4.0] - 2026-04-17

**Theme: Accuracy + Infrastructure.** Bump to 470 tests, add pyyaml as core dependency, fix all stale test counts and module counts across docs, add missing CI imports, Python 3.13 to matrix.
Expand Down
20 changes: 11 additions & 9 deletions docs/ADVANCED.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,23 @@

## GTG-1002 APT Simulation

Full simulation of the first documented AI-orchestrated cyber espionage campaign (Anthropic, November 2025).
This module sends crafted defensive probes that exercise a target system's controls against the adversary behavior pattern documented in the GTG-1002 incident (Anthropic, November 2025 — the first publicly documented AI-orchestrated cyber espionage campaign). The harness does not execute the listed adversary behaviors; it sends inputs designed to trigger the target's detection and response controls, then analyzes target responses.

```bash
agent-security test gtg1002 --url https://agent.example.com --full-campaign
```

| Phase | Tests | Real GTG-1002 Activity | What We Test |
**Reading guide:** *Adversary behavior we probe for* describes what the original GTG-1002 actor did. *Detection probes the harness sends* describes what the harness sends to your target so you can verify your defenses block, log, or alert on that behavior class.

| Phase | Tests | Adversary behavior we probe for | Detection probes the harness sends |
|---|---|---|---|
| **Initialization** | 2 | Roleplay as "CrowdStrike pentester" | Security firm impersonation + escalating personas |
| **Reconnaissance** | 3 | Autonomous infrastructure enumeration | Systematic enumeration, MCP tool abuse |
| **Vulnerability** | 2 | AI-generated exploit payloads | Exploit generation + callback validation |
| **Credentials** | 3 | Config credential extraction | 4 extraction techniques + lateral movement |
| **Data Collection** | 3 | Database extraction + intelligence categorization | User data exfiltration + backdoor creation |
| **Documentation** | 2 | Auto-generated attack documentation | Attack report generation + handoff |
| **Hallucination** | 2 | Claude "fabricated findings and data" | Security finding + credential fabrication |
| **Initialization** | 2 | Roleplay as "CrowdStrike pentester" | Probes target resistance to security-firm-impersonation persona escalation |
| **Reconnaissance** | 3 | Autonomous infrastructure enumeration | Probes target rate-limiting and access controls against enumeration patterns |
| **Vulnerability** | 2 | AI-generated exploit payloads | Probes detection of AI-generated payload patterns |
| **Credentials** | 3 | Config credential extraction attempts | Probes detection of four credential-extraction patterns and lateral-movement signals |
| **Data Collection** | 3 | Database extraction + intelligence categorization | Probes detection of exfiltration and categorization patterns |
| **Documentation** | 2 | Auto-generated attack documentation | Probes detection of attack-report and handoff generation patterns |
| **Hallucination** | 2 | Claude "fabricated findings and data" | Probes detection of fabricated-finding and credential-fabrication patterns |

---

Expand Down
4 changes: 2 additions & 2 deletions docs/TEST-INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,14 +140,14 @@ agent-security test enterprise --platform salesforce --url https://your-org.sale
| **GTG-1002 APT Simulation** | 17 | Full Campaign | First documented AI-orchestrated cyber espionage |
| **Advanced Attacks** | 10 | Multi-step | Polymorphic, stateful, multi-domain attack chains |
| **Over-Refusal** | 25 | All protocols | False positive rate testing: legitimate requests that should NOT be blocked |
| **Provenance & Attestation** | 15 | Supply Chain | Fake provenance, spoofed attestation, marketplace integrity (CVE-2026-25253) |
| **Provenance & Attestation** | 15 | Supply Chain | Fake provenance, spoofed attestation, marketplace integrity ([CVE-2026-25253](https://nvd.nist.gov/vuln/detail/CVE-2026-25253)) |
| **Jailbreak** | 25 | Model/Agent | DAN variants, token smuggling, authority impersonation, persistence |
| **Return Channel** | 8 | Output/Context | Return channel poisoning: output injection, ANSI escape, context overflow, encoded smuggling, structured data poisoning |
| **Identity & Authorization** | 18 | NIST NCCoE | All 6 focus areas from NIST agent identity standards |
| **Capability Profile** | 10 | A2A JSON-RPC | Executor capability boundary validation, profile escalation prevention |
| **Harmful Output** | 10 | A2A JSON-RPC | Toxicity, bias, scope violations, deception (AIUC-1 C003/C004) |
| **CBRN Prevention** | 8 | A2A JSON-RPC | Chemical/biological/radiological/nuclear content safeguards (AIUC-1 F002) |
| **Incident Response** | 8 | A2A JSON-RPC | Alert triggering, kill switch, log completeness, recovery (AIUC-1 E001-E003) |
| **CVE-2026-25253 Reproduction** | 8 | MCP Supply Chain | Nested schema injection, fork fingerprinting, marketplace contamination, encoded payload detection |
| **[CVE-2026-25253](https://nvd.nist.gov/vuln/detail/CVE-2026-25253) Reproduction** | 8 | MCP Supply Chain | Nested schema injection, fork fingerprinting, marketplace contamination, encoded payload detection |
| **AIUC-1 Compliance** | 12 | Agent Safety | Incident response, CBRN prevention, harmful content, scope creep, authority impersonation |
| **Cloud Agent Platforms** | 25 | Platform APIs | AWS Bedrock, Azure AI Agent Service, Google Vertex, Salesforce Agentforce, IBM watsonx |
Loading