- Date: 2026-07-11
- Qualified PentestGPT commit:
9e6d2c43614134a2d6d7214442b1a03bb8bf4708 - Pre-hardening checkpoint:
acaca31 - Status: historical baseline; implementation has since changed
This record keeps the reproducible benchmark facts and engineering lessons from the first hardened
pentestgpt_agent build. It is not the current runtime contract. See
docs/architecture.md and
pentestgpt_agent/README.md.
Both accepted runs used Claude Opus 4.8, fresh provider episodes, a clean PentestGPT source commit, and exact trace-backed evidence.
| Target | Result | Tasks / episodes | Provider cost | Wall time | Historical audit |
|---|---|---|---|---|---|
| XBEN-068 command injection | solved | 4 / 9 | $0.856196 | 238.9 s | 16/16 |
| XBEN-080 Jinja2 SSTI | solved | 4 / 9 | $0.834787 | 180.0 s | 16/16 |
The manifests recorded exact source commits and wheel hashes. Raw artifacts remain private because they include complete prompts, target output, provider session identifiers, and canonical state.
The 16-check audit count is historical. Audit schema 2 no longer rejects Supervisor actions or provider-side file writes; those values remain visible as diagnostics.
- exact run/task/attempt/episode/receipt identity for observations;
- one contiguous receipt slice per canonical observation;
- safe handling of nonzero command exits as negative evidence;
- narrow CRLF/LF transport normalization;
- bounded fallback for oversized or unsupported evidence quotes;
- no automatic replay after an actionful failure;
- deterministic recovery of terminal traces and one known malformed Claude result shape;
- persistent retry/decision budgets and atomic failure settlement;
- target, dependency, TEST-to-EXPLOIT, and completion-basis validation;
- bounded Supervisor and Executor state projections.
The decisive behavior improvement was episode locality: discovery, bounded testing, exploitation, and completion occurred in separate tasks instead of one drifting tool session.
| Failure class | Resulting correction |
|---|---|
| target image unavailable under a unique Compose project | build the target before up --no-build |
| provider auto-memory crossed episodes | disable Claude auto-memory and keep canonical state in SQLite |
| one task per payload | constrain each task to one hypothesis/surface |
| capability proved but Executor continued | strengthen phase boundaries and reserve a result turn |
| malformed structured result after successful actions | exact deterministic recovery, never fuzzy replay |
| CRLF receipt versus LF model quote | accept only the narrow newline transport normalization |
| unsupported long evidence rewrite | retain one exact bounded receipt and commit only progress |
- The package was renamed from
pentestgpt_agent_newtopentestgpt_agent. - Both Supervisor and Executor now intentionally receive all provider tools and
FULL_ACCESS. - The isolated deployment environment—not a PentestGPT-owned tool mediator—is the selected security boundary for the current product.
- Supervisor actions and file writes are allowed, traced, and reported.
- XBOW orchestration and result archives moved to the sibling
xbow-benchmarkrepository. - The canonical provider wrapper is
../UnifedAgentWrapper; the root copy is transitional only.
The hardening run proved durability and provenance on two local medium targets. It did not prove general controller convergence. The later HTB Enigma run preserved state correctly but revisited completed discovery branches and failed to reach exploitation. The next design work should improve the Supervisor's compact coverage/branch projection and deterministic duplicate policy before adding another agent or a RAG system.
Git history retains the former 300-line operational chronology if deeper forensic detail is needed.