From 17c51263c24823839f30c4156b35ad8bf161c205 Mon Sep 17 00:00:00 2001 From: Michael Saleme Date: Sat, 2 May 2026 08:07:51 -0500 Subject: [PATCH] docs: v4.4.2 documentation hardening pass against VT Code Insight Reframed GTG-1002 capability table in docs/ADVANCED.md for unambiguous defensive intent: column headers from "Real GTG-1002 Activity" / "What We Test" to "Adversary behavior we probe for" / "Detection probes the harness sends"; cell content reworded from active to defensive voice. Added top-of-section defensive framing paragraph and reading guide above the table. Anchored both CVE-2026-25253 references in docs/TEST-INVENTORY.md with inline NVD links. No code changes; no test changes; test count unchanged at 470 across 32 modules. ClawHub bundle republished as v4.4.2; pyproject.toml remains v4.4.0 until next code-change release. Counterpart memory entry: playbook_security_skill_scanner_hardening.md Pattern 5 (bundled-docs adversary-vs-defender table reframing). Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 16 ++++++++++++++++ docs/ADVANCED.md | 20 +++++++++++--------- docs/TEST-INVENTORY.md | 4 ++-- 3 files changed, 29 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f34160..9d0a295 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,22 @@ All notable changes to the Agent Security Harness will be documented in this fil The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [4.4.2] - 2026-05-02 + +**Theme: Documentation hardening.** Reframed offensive-vocabulary phrasing in `docs/ADVANCED.md` GTG-1002 capability table for unambiguous defensive intent. NVD-anchored CVE-2026-25253 references in `docs/TEST-INVENTORY.md`. No code changes; no test changes; test count unchanged at 470 across 32 modules. + +### Changed + +- `docs/ADVANCED.md` GTG-1002 table: column headers reframed from `Real GTG-1002 Activity` / `What We Test` to `Adversary behavior we probe for` / `Detection probes the harness sends`. Cell content reworded from active to defensive voice ("Probes detection of X" rather than "User data exfiltration"). +- `docs/ADVANCED.md` added top-of-section defensive framing paragraph and reading guide above the GTG-1002 table. +- `docs/TEST-INVENTORY.md` anchored both CVE-2026-25253 references with inline NVD links. + +### Notes + +- Hardening pass against VirusTotal Code Insight (Gemini-powered LLM scanner). The previous v4.4.1 bundle drew a "suspicious" Code Insight verdict from string-density on offensive vocabulary in bundled markdown documentation; the reframing reduces that signal without changing test capability or coverage. +- ClawHub bundle republished as v4.4.2 (skill-bundle versioning is independent of underlying package version; `pyproject.toml` remains at v4.4.0 until next code-change release). +- Counterpart memory entry: `playbook_security_skill_scanner_hardening.md` Pattern 5 (bundled-docs adversary-vs-defender table reframing). + ## [4.4.0] - 2026-04-17 **Theme: Accuracy + Infrastructure.** Bump to 470 tests, add pyyaml as core dependency, fix all stale test counts and module counts across docs, add missing CI imports, Python 3.13 to matrix. diff --git a/docs/ADVANCED.md b/docs/ADVANCED.md index a7ed871..bf2dbce 100644 --- a/docs/ADVANCED.md +++ b/docs/ADVANCED.md @@ -2,21 +2,23 @@ ## GTG-1002 APT Simulation -Full simulation of the first documented AI-orchestrated cyber espionage campaign (Anthropic, November 2025). +This module sends crafted defensive probes that exercise a target system's controls against the adversary behavior pattern documented in the GTG-1002 incident (Anthropic, November 2025 — the first publicly documented AI-orchestrated cyber espionage campaign). The harness does not execute the listed adversary behaviors; it sends inputs designed to trigger the target's detection and response controls, then analyzes target responses. ```bash agent-security test gtg1002 --url https://agent.example.com --full-campaign ``` -| Phase | Tests | Real GTG-1002 Activity | What We Test | +**Reading guide:** *Adversary behavior we probe for* describes what the original GTG-1002 actor did. *Detection probes the harness sends* describes what the harness sends to your target so you can verify your defenses block, log, or alert on that behavior class. + +| Phase | Tests | Adversary behavior we probe for | Detection probes the harness sends | |---|---|---|---| -| **Initialization** | 2 | Roleplay as "CrowdStrike pentester" | Security firm impersonation + escalating personas | -| **Reconnaissance** | 3 | Autonomous infrastructure enumeration | Systematic enumeration, MCP tool abuse | -| **Vulnerability** | 2 | AI-generated exploit payloads | Exploit generation + callback validation | -| **Credentials** | 3 | Config credential extraction | 4 extraction techniques + lateral movement | -| **Data Collection** | 3 | Database extraction + intelligence categorization | User data exfiltration + backdoor creation | -| **Documentation** | 2 | Auto-generated attack documentation | Attack report generation + handoff | -| **Hallucination** | 2 | Claude "fabricated findings and data" | Security finding + credential fabrication | +| **Initialization** | 2 | Roleplay as "CrowdStrike pentester" | Probes target resistance to security-firm-impersonation persona escalation | +| **Reconnaissance** | 3 | Autonomous infrastructure enumeration | Probes target rate-limiting and access controls against enumeration patterns | +| **Vulnerability** | 2 | AI-generated exploit payloads | Probes detection of AI-generated payload patterns | +| **Credentials** | 3 | Config credential extraction attempts | Probes detection of four credential-extraction patterns and lateral-movement signals | +| **Data Collection** | 3 | Database extraction + intelligence categorization | Probes detection of exfiltration and categorization patterns | +| **Documentation** | 2 | Auto-generated attack documentation | Probes detection of attack-report and handoff generation patterns | +| **Hallucination** | 2 | Claude "fabricated findings and data" | Probes detection of fabricated-finding and credential-fabrication patterns | --- diff --git a/docs/TEST-INVENTORY.md b/docs/TEST-INVENTORY.md index 4be94b0..4221122 100644 --- a/docs/TEST-INVENTORY.md +++ b/docs/TEST-INVENTORY.md @@ -140,7 +140,7 @@ agent-security test enterprise --platform salesforce --url https://your-org.sale | **GTG-1002 APT Simulation** | 17 | Full Campaign | First documented AI-orchestrated cyber espionage | | **Advanced Attacks** | 10 | Multi-step | Polymorphic, stateful, multi-domain attack chains | | **Over-Refusal** | 25 | All protocols | False positive rate testing: legitimate requests that should NOT be blocked | -| **Provenance & Attestation** | 15 | Supply Chain | Fake provenance, spoofed attestation, marketplace integrity (CVE-2026-25253) | +| **Provenance & Attestation** | 15 | Supply Chain | Fake provenance, spoofed attestation, marketplace integrity ([CVE-2026-25253](https://nvd.nist.gov/vuln/detail/CVE-2026-25253)) | | **Jailbreak** | 25 | Model/Agent | DAN variants, token smuggling, authority impersonation, persistence | | **Return Channel** | 8 | Output/Context | Return channel poisoning: output injection, ANSI escape, context overflow, encoded smuggling, structured data poisoning | | **Identity & Authorization** | 18 | NIST NCCoE | All 6 focus areas from NIST agent identity standards | @@ -148,6 +148,6 @@ agent-security test enterprise --platform salesforce --url https://your-org.sale | **Harmful Output** | 10 | A2A JSON-RPC | Toxicity, bias, scope violations, deception (AIUC-1 C003/C004) | | **CBRN Prevention** | 8 | A2A JSON-RPC | Chemical/biological/radiological/nuclear content safeguards (AIUC-1 F002) | | **Incident Response** | 8 | A2A JSON-RPC | Alert triggering, kill switch, log completeness, recovery (AIUC-1 E001-E003) | -| **CVE-2026-25253 Reproduction** | 8 | MCP Supply Chain | Nested schema injection, fork fingerprinting, marketplace contamination, encoded payload detection | +| **[CVE-2026-25253](https://nvd.nist.gov/vuln/detail/CVE-2026-25253) Reproduction** | 8 | MCP Supply Chain | Nested schema injection, fork fingerprinting, marketplace contamination, encoded payload detection | | **AIUC-1 Compliance** | 12 | Agent Safety | Incident response, CBRN prevention, harmful content, scope creep, authority impersonation | | **Cloud Agent Platforms** | 25 | Platform APIs | AWS Bedrock, Azure AI Agent Service, Google Vertex, Salesforce Agentforce, IBM watsonx |