Skip to content

WIP: Adds search functionality to File Explorer#3

Open
useafterfree wants to merge 1 commit into
digitalandrew:mainfrom
useafterfree:2/search-function
Open

WIP: Adds search functionality to File Explorer#3
useafterfree wants to merge 1 commit into
digitalandrew:mainfrom
useafterfree:2/search-function

Conversation

@useafterfree

Copy link
Copy Markdown

Done:

  • make targets: make dev which runs the services in docker-compose and the frontend in dev mode
  • fixes tests
  • Re-organizes backend Dockerfile to help with re-builds
  • python -m alembic so alembic is not needed to be in pypath
  • Attempts to install missing binaries (libmagic) on macos using homebrew
  • Implement actual search functionality using existing search harnesess

In progress:

  • Attempts to use relative directories for kernels and data for MacOS
  • Support searching for directories
search_2.mp4

@useafterfree useafterfree changed the title WIP: Adds search funntionality to File Explorer WIP: Adds search functionality to File Explorer Mar 12, 2026
eastmadc referenced this pull request in eastmadc/wairz Apr 20, 2026
Two previously-unmigrated call sites reached via scope-widening grep
beyond the intake's list of 7:
- ai/tools/filesystem.py:_handle_get_component_map — firmware-wide
  cache (binary_sha256 IS NULL) for the MCP get_component_map tool
- routers/component_map.py:get_component_map — firmware-wide cache
  for the REST component-map endpoint

Both sites also gain delete-then-insert idempotency (previously plain
INSERT; second build attempts after a failed first would accumulate
rows until the unique constraint caught it). The router retains its
explicit commit; the MCP tool flushes via the helper per rule #3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 20, 2026
…on d9f61335)

6 successful patterns + 5 anti-patterns + 5 key decisions extracted
from the 15-commit Wave 1 closure of the intake-sweep campaign.

Headline extractions:
- Pattern #3: worktree-per-stream (Rule #23) validated for the 3rd
  consecutive session (this session 3/3 streams held; prior sessions
  198243b8 β only + 435cb5c2 control).
- Pattern #4: Wave 2 deferral decision — prompt-scheduled refactor
  deferred when LOC grew past intake measurement (2263 → 2589 in
  manifest_checks.py); safer than partial split.
- Pattern #5: Rule #26 verification recipe refinement — per-page
  code-split chunks, not just main bundle.
- Anti-pattern #1: absolute path for worktree node_modules symlink;
  relative '../../' resolves to .worktrees/ not the repo root.
- Anti-pattern #2: main-bundle-only grep gave false stale-build alarm;
  per-chunk verification is the durable form.
- Anti-pattern #4: always chain 'cd .worktrees/... &&' in the SAME
  bash call as git commit; cwd resets between discrete bash calls.

No new quality rules proposed — existing Rules 1-26 cover this
session's failure modes. Candidates for future promotion: #4 Wave-2
deferral discipline and #5 per-chunk bundle verification, but both
are arguably derivable from existing Rule #19 and Rule #26.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
4-scout research fleet (ROI scan, CI triage, 202+polling readiness,
Ouroboros fit) consolidated into a 3-session forward plan:

+1: /fleet 202+polling campaign (2 streams, emulation α + fuzzing β)
+2: /fleet backend-pytest-unstable-tests (3 domain clusters)
+3: intake drain (cache extraction, hook dedup, pagination)

Key findings documented:
- test_cache_module.py failures are likely real product bugs
  (db.commit vs db.flush per Rule #3), not test bugs — intake
  diagnosis needs re-measure under Rule #19.
- Ouroboros stays dormant; no current backlog needs Socratic
  interview. Memory preference confirmed.
- MobSF baseline fixtures are the only blocker risk for stream α
  of the CI-unblock campaign; pre-flight fixture audit documented.
- Anti-picks: feature-latte-llm-taint-analysis (already shipped),
  device-acquisition-v2 (blocked on hardware).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
Builds a real Eaton-shape tar and zip (each containing a minimal
FAT16 stub + EULA + manifest.json), streams the file through
FirmwareService.upload, and asserts:

  1. Tar-of-FAT-image → firmware.extracted_path is None.
     The upload-time shortcut must fall through to the terminal
     path so the frontend's subsequent POST /unpack hands the
     file to unblob.

  2. Zip-of-FAT-image → firmware.extracted_path is None.
     Same defect class, same expected fall-through behaviour.

  3. Pure-rootfs tar (ADB-dump shape) → firmware.extracted_path
     is set AND firmware.device_metadata['detection_roots'] is
     a non-empty list of existing directories (Rule #16 guard).

Each test builds its own firmware tarball via tarfile+BytesIO and
streams the bytes through a MagicMock UploadFile that replicates
FastAPI's UploadFile.read()/size surface — no network, no real DB,
fast (<1s total). Settings are patched with per-test storage_root
under tmp_path so the tests are hermetic.

Together these guard the commit chain:
  - 38d01d8 (find_filesystem_root fallback gate) — #1 and #2
  - 5b8d606 (detection_roots on shortcut paths) — #3

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
…ID format

Pre-flight diagnosis confirmed: both failures were test bugs, not service bugs.
_cache.py correctly uses flush() (Rule #3); the assertions were wrong.

- test_does_not_commit_only_flushes: hasattr on an AsyncMock auto-creates
  attributes, so the guard was meaningless. Switched to
  db.commit.assert_not_called() which is the correct mock-level check.
- test_deletes_all_rows_for_firmware: SQLAlchemy compiles UUID literals
  hyphenless (32-char hex). str(fw_id) has hyphens, compiled SQL does not.
  Fixed to assert fw_id.hex in compiled.

All 15 tests in test_cache_module.py now pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
Extracted from the fleet session that un-ignored 15 pytest files + renamed
the workflow job ("Pytest (Backend, Stable Subset)" -> "Pytest (Backend)").
7 successful patterns, 6 anti-patterns, 3 new harness quality rules, 1
next-session seed.

Patterns landed in .planning/knowledge/:
- Pre-flight Rule #19 re-measure before trusting any scout/seed/intake count
- Worktree discipline via literal `git worktree add` in every stream prompt
  (0 cross-stream sweeps across 19 commits; 5 of 6 consecutive Rule #23 wins)
- Lazy-import patch target = source module (triple-independent discovery α/β/γ)
- Baseline-parity assertion shape: `>=N` not `==N` when checks can grow
- Adjacent-hunk merge conflict resolution: drop BOTH sides when each branch
  deleted a different adjacent block
- Docker-cp iteration loop (Rule #20) for 5s test iteration vs 3-5min rebuild
- Per-stream test subdir: works IFF no cross-package imports

Anti-patterns captured:
- Scout "service is broken" hypothesis without reading raw assertions
  (Scout B's cache_module Rule #3 claim — caught at pre-flight, prevented
  a service regression)
- Per-stream test subdir as default (breaks `from tests.X import Y`)
- Stale intake line counts (intake said 18 files; actual workflow had 15)
- `assert not hasattr(AsyncMock, "X_invoked")` — auto-attr creation defeats
- `str(uuid) in compiled_sql` — SQLAlchemy renders hex without dashes
- `patch("app.services.X.Y")` when Y is lazy-imported from a third-party lib

New harness.json quality rules (all 3 new, 0 duplicates):
- auto-pytest-no-hasattr-against-mock
- auto-pytest-uuid-str-in-compiled-sql
- auto-pytest-mock-patch-androguard-at-service

Next-session seed: seed-next-session-2026-04-24.md with four routes
(A=P3-circular-imports carve-out, B=wait for pressure, C=promote to
CLAUDE.md/.mex, D=re-scan architecture-review). Previous seed +1+2+3
marked complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
Extracts patterns + antipatterns from session f2f9060c continuation
(commits b213795, ff111d2, 681a592 — mobsfscan/ pair). No campaign file
(intake-driven continuation); 3rd link in the chain after
assessment-promote-rule30-2026-04-24 and p3-carveout-fuzzing-emulation-
2026-04-24.

Highlights:
- Rule #31 width-canary FIRST applied pre-edit (not discovered post-hoc)
  — narrow pattern would have hidden 3 of 7 hits.
- New pattern: TYPE_CHECKING-vs-runtime triage separates legitimate
  PEP-484 type-only imports from genuine lazy-import promotion targets.
  3 of 7 broad hits were TYPE_CHECKING-guarded; preserved untouched.
- New near-miss documented: stale-container Rule #11 smoke returned
  all-False module attributes; correctly diagnosed as Rule #20 stale-
  image condition (not a refactor bug) and recovered via docker cp.
  A misdiagnosis would have reverted good commits.
- Mechanical-safe profile now durable at 5 files / 3 sessions / 22
  promotions / 0 reverts. But firmware_service.py remains explicitly
  out-of-scope per seed + antipattern #3 (density ranking alone is not
  a green light).

Zero new quality rules added to harness.json. All patterns are workflow
discipline (width-canary, TYPE_CHECKING triage, stale-container
diagnosis), not regex-catchable signatures. Predecessor reached the
same conclusion — the pattern is stable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
…or nested-extract paths

Credentials-detector formats title as f"Hardcoded credential in {path}";
on deeply-nested firmware extracts (e.g. RespArray V1.12 with paths
through .tar.xz_extract → .gzip_extract → .gzip_extract → etc/.../delegates.xml,
~290 chars), the per-finding flush triggered StringDataRightTruncationError
on findings.title and rolled back the entire /security/audit transaction —
no findings persisted even though every scanner ran successfully (~12 min
of work lost). Widening to 512 matches the existing file_path column width.

Migration: hand-written using the 1f6c72decc84 precedent (autogenerate is
currently blocked by an orthogonal model-registration gap — hardware_firmware
referenced via FK from SbomVulnerability but not exported via app/models/__init__.py;
fixing that registration is out of scope for this fix per Rule #19).

Verified post-rebuild: DB column width = 512, Finding mapper width = 512,
alembic head = d4a7c8b6e2f1.

Source: .planning/intake/gui-smoke-bugs-2026-04-24.md Bug #3 (HIGH).
Rule #20: rebuilt backend+worker (class-shape change to ORM mapper).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz Apr 26, 2026
…al-oom follow-up

Three-commit sweep (ca583d0 / f71f978 / 9f7ddde) verified end-to-end against
fw a7523429 (RespArray V1.12, project 00815038):

- Bug #3 (HIGH, findings.title VARCHAR widening) — FULLY VERIFIED:
  post-audit findings count grew from 4 baseline to 315; zero
  StringDataRightTruncationError events in backend logs across the full
  ~16 minute audit run; DB column = 512, ORM mapper = 512, alembic head =
  d4a7c8b6e2f1.
- Bug #2 (LOW-MED, attack_surface arch=NULL) — FULLY VERIFIED:
  /attack-surface/scan?force_rescan=true (in JSON body, not query) returned
  200 in 14.5 s; arch != NULL grew from 0/1624 to 1613/1624 (99.3%); sole
  non-null architecture is 'arm' as expected.
- Bug #1 (MEDIUM, cve_matcher singleton) — MECHANICALLY VERIFIED, FUNCTIONALLY
  INCOMPLETE: singleton fix is in place and provably effective (zero "CPE
  dictionary loaded from Redis cache" events during second cve-match smoke,
  vs intake's pre-fix 11×); but the cve-match endpoint STILL kills the
  backend container at ~85 s with zero log output during that window. The
  intake's hypothesis was complete-but-narrow — per-blob CpeDictionaryService
  was one memory hog, not the only one. Most likely residual cause is Tier 4
  _match_kernel_cpe (kernel components × kmod blobs matrix) or Tier 5
  _match_kernel_subsystem; both undiagnosed pending instrumentation.

Filed cve-match-residual-oom-2026-04-25.md as HIGH-priority observation-only
follow-up: describes evidence, ranks Tier-4/5 suspects, prescribes
instrumentation-first diagnostic plan + acceptance criteria for the next
session. Includes a candidate Rule #20 sub-clause refinement extracted from
this session: docker cp + Rule #11 import smoke is INSUFFICIENT to verify
that a long-lived uvicorn process picked up code changes (Rule #11 spawns
a NEW python subprocess; uvicorn's sys.modules cache holds the pre-cp
version). The first cve-match request after Bug #1's fix still ran the
OLD cached module and OOM'd from the 11× dictionary load — only after the
OOM-restart did the new code take effect.

ROUTER updated: GUI-smoke triage entry added to "Recently shipped"; the
cve-match endpoint listed under "Known issues" as un-shippable on
multi-blob firmware until the residual is resolved.

Smoke wall-clock: ~23 min (15-min budget exceeded; abort condition triggered
on Bug #1 acceptance criterion #2 — backend kill, not 2xx). Three commits
ship as-is; verification was 2/3 complete + 1 partial. Per Rule #19 and the
intake's own abort guidance, no 4th iteration into a Tier-4/5 fix this
session.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eastmadc referenced this pull request in eastmadc/wairz May 7, 2026
…21 resolved)

Second re-scan (Option C from closed seed-next-session-2026-04-24). All 21
items in the master index — 20 review items + item 0 hardware-firmware
feature — verified `status: completed` in their respective intake files.
Item #12 (P3 circular imports), the only `partial` from the first re-scan
on 2026-04-24, is now fully closed via 4 per-service-pair carve-outs
(firmware_service 5e2cb18, cve_matcher 9a26c1a, attack_surface 4bd491b,
clamav 8f9d261) plus the P3 thread close (e84f02e).

Underlying patterns and anti-patterns from the original review are encoded
as Learned Rules in CLAUDE.md (Rules #1, #3, #5, #7 cover the recurring
sub-themes — sandbox path validation, MCP transaction ownership, async
filesystem I/O, AsyncSession concurrency).

This master index is fully closed against; future sessions should skip the
re-scan unless wairz's attack surface materially changes (new external
integration, public deployment).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
digitalandrew added a commit that referenced this pull request Jun 3, 2026
Captures the grounded plan for the higher-lift deferred items: Track A
(AFL_INST_LIBS auto-detect), Track B (compile-and-fuzz a function harness
vs a firmware .so = #2 + #4-persistent), Track C (auth-replay shim), and
Track D (robust serial exec via a dedicated virtio console = #6, with the
root cause confirmed live). Includes grouping rationale and order.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
digitalandrew added a commit that referenced this pull request Jun 3, 2026
In QEMU mode AFL++ instruments only the main object's code range, so a
thin CLI wrapper whose real logic lives in a shared library (e.g.
xmllint → libxml2) fuzzes at near-zero coverage with no feedback —
looking like a healthy campaign while exercising nothing (feedback #3).

- _elf_lib_backed(): classifies a target as lib-backed when it has a
  non-libc DT_NEEDED and a small .text (delegates its work).
- start_campaign auto-sets AFL_INST_LIBS=1 for such targets unless the
  caller set it explicitly, and persists the effective env.
- analyze_fuzzing_target reports the lib-backed verdict + deps.
- diagnose_fuzzing_campaign, on <5% coverage, recommends restarting with
  AFL_INST_LIBS=1 when a lib-backed target is missing it (catches an
  explicit disable or a heuristic miss).

Verified live: xmllint (libxml2) auto-enables the flag and reaches ~10%
coverage in 90s vs the silent ~0.3% baseline; busybox (self-contained)
is correctly not flagged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
digitalandrew added a commit that referenced this pull request Jun 4, 2026
Captures the grounded plan for the higher-lift deferred items: Track A
(AFL_INST_LIBS auto-detect), Track B (compile-and-fuzz a function harness
vs a firmware .so = #2 + #4-persistent), Track C (auth-replay shim), and
Track D (robust serial exec via a dedicated virtio console = #6, with the
root cause confirmed live). Includes grouping rationale and order.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
digitalandrew added a commit that referenced this pull request Jun 4, 2026
In QEMU mode AFL++ instruments only the main object's code range, so a
thin CLI wrapper whose real logic lives in a shared library (e.g.
xmllint → libxml2) fuzzes at near-zero coverage with no feedback —
looking like a healthy campaign while exercising nothing (feedback #3).

- _elf_lib_backed(): classifies a target as lib-backed when it has a
  non-libc DT_NEEDED and a small .text (delegates its work).
- start_campaign auto-sets AFL_INST_LIBS=1 for such targets unless the
  caller set it explicitly, and persists the effective env.
- analyze_fuzzing_target reports the lib-backed verdict + deps.
- diagnose_fuzzing_campaign, on <5% coverage, recommends restarting with
  AFL_INST_LIBS=1 when a lib-backed target is missing it (catches an
  explicit disable or a heuristic miss).

Verified live: xmllint (libxml2) auto-enables the flag and reaches ~10%
coverage in 90s vs the silent ~0.3% baseline; busybox (self-contained)
is correctly not flagged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
MSI is an OLE2 compound document with structured tables and an
embedded compressed file payload. msiextract (msitools) walks the File
table and writes installable files with the Directory-table layout.

backend/app/workers/unpack_msi.py: two-phase shape — msiinfo suminfo
(validity probe) → msiextract (file payload). 5-min timeout (radare2-tier
per Rule digitalandrew#29). Custom-action discipline documented in module docstring:
the worker NEVER executes custom actions; msiextract is a pure file
extractor (no msiexec, no Windows Installer engine). Persona-E
anti-pattern digitalandrew#3 / Phase β CLAUDE.md Rule digitalandrew#36 candidate.

backend/tests/test_unpack_msi.py: 8 mock-based contract tests covering
missing binaries, unreadable archive, extract timeout, extract non-zero
exit, ProgramFiles payload, empty (action-only) MSI, progress callback,
plus a custom-action discipline assertion that scans the extract command
for forbidden tokens (msiexec, wine, winetricks). One Rule #35b live
canary auto-skipping until Phase α.6 ships msitools + tiny.msi fixture.

Test: pytest tests/test_unpack_msi.py -v → 8 passed, 1 skipped.
Branch: feat/windows-phase-alpha-2026-05-07.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Surfaces extracted Windows-archive contents to the MCP layer with
sandbox-safe path resolution and 30 KB output truncation (Rules digitalandrew#1, digitalandrew#29).

backend/app/ai/tools/windows_archive.py:
- list_cab_contents — cabextract --list of any CAB inside the firmware
  tree (foundation for MSU, driver-package, generic CAB inspection).
- read_msix_manifest — XML parse of AppxManifest.xml or
  AppxBundleManifest.xml. Surfaces Identity (Name/Version/Publisher/
  Architecture), Capabilities (incl. DeviceCapability /
  RestrictedCapability), Applications (Id/Executable/EntryPoint),
  TargetDeviceFamily MinVersion/MaxVersion. Bundle vs single-package
  detection on root tag.
- dump_msi_custom_actions — msidump --binary into a sibling
  <msi>_custom_actions/ dir. **Custom actions are EXTRACTED, NEVER
  EXECUTED** (Persona-E anti-pattern digitalandrew#3 / Rule digitalandrew#36 candidate). The
  module docstring + tool description + dump-target log all carry
  the "extract-only, never executed" discipline message.
- parse_inf_basic — INF section parser with [Version] / [Manufacturer]
  / [Strings] / [Models.NTamd64]/.NTx86 enumeration. Encoding-aware
  (UTF-16-LE BOM check before UTF-8 fallback). 200 KB input cap.
- identify_psf_baseline — PSF magic validation + RSDS GUID extraction
  from header (the PE-debug marker that identifies the target binary
  to reconstruct against). Surfaces standard mixed-endian GUID format
  + age. Operator uses GUID to locate baseline elsewhere.
- classify_driver_package_subtype — re-walks an extracted CAB tree
  for the operator-hint reclassification path (subtype: cab_inf_sys_cat
  / dch / cab_inf_only / cab_sys_only / driver_store_dir / unknown).

backend/app/ai/__init__.py: register_windows_archive_tools wired into
create_tool_registry() — 172 → 178 total tools.

backend/tests/test_windows_archive_tools.py: 20 contract tests with
duck-typed _StubContext for ToolContext.resolve_path() — covers
missing-file / missing-binary / parse-error / canonical-success across
all 6 handlers, plus an RSDS GUID extraction verification on the PSF
header (validates the mixed-endian GUID conversion).

Test: pytest tests/test_windows_archive_tools.py -v → 20 passed in 2.55s.
Rule digitalandrew#11 import smoke: docker compose exec backend python -c
'from app.ai import create_tool_registry; r = create_tool_registry();
print(len(r._tools))' → 178 (was 172).

Branch: feat/windows-phase-alpha-2026-05-07.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
backend/pyproject.toml: add 3 Python deps for Phase β work:
- signify >= 0.7 (resolves to 0.9.2): happy-path PE Authenticode
  validation. Crucially, signify 0.9.2 ships TRUSTED_CERTIFICATE_STORE
  pre-populated with Microsoft Authenticode roots — offline-trust-
  anchor (Rule digitalandrew#37 candidate) is partially solved out-of-the-box.
- asn1crypto >= 1.5: ASN.1/PKCS#7 primitives for malformed legacy
  chains and the dual-sig (SHA-1+SHA-256) enumeration path that
  signify's "best mode" doesn't expose (Persona-E sec.2 digitalandrew#3).
- uefi-firmware >= 1.11: DBX EFI_SIGNATURE_LIST parsing (Persona-E
  digitalandrew#10). Latest released version is 1.11; Persona-B brief had stale
  '>=1.12' which doesn't exist on PyPI — corrected.

Smoke verification (post-pip install in rebuilt backend container):
- signify 0.9.2 → AuthenticodeFile + TRUSTED_CERTIFICATE_STORE green
- asn1crypto 1.5.1 → cms + x509 green
- uefi-firmware 1.11 → guids.get_guid_name green
- Real exit=0; no pipe-induced exit obfuscation (Rule #35a clean).

signify 0.9.2's API differs from the persona-B brief's class names:
- AuthenticodeFile (not SignedPEFile — renamed in 0.9.x)
- TRUSTED_CERTIFICATE_STORE replaces manual MS root bundling
This means Phase β.4 Authenticode validator can lean entirely on
signify's bundled trust store; the explicit MS-roots Dockerfile bundle
work simplifies to just refreshing signify itself quarterly.

Branch: feat/windows-phase-alpha-2026-05-07 (will rename to
.../windows-coverage-godmode-... at PR time as the campaign now spans
α + β; phase boundaries are commit-tagged).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
backend/app/services/authenticode_service.py: wraps signify 0.9.2's
AuthenticodeFile API to produce a canonical AuthenticodeVerdict that
maps 1:1 onto WindowsPESignature columns. Cryptographic correctness
is signify's job; this service's job is the verdict mapping +
defensive error handling so the Phase β.4 background runner +
Phase β.7 MCP tool can call into it without try/except sprawl.

Persona-E digitalandrew#2 / digitalandrew#3 / digitalandrew#5 disciplines codified:
- Tri-state ChainStatus: valid_now / revoked / never_valid /
  valid_at_signing / unknown — discriminated by counter-signature
  timestamp presence.
- Dual-sig: iter_signatures() enumerates all signatures (Persona-E
  sec.2 digitalandrew#3 — signify's "any" mode picks one but signatures_count
  surfaces the dual presence).
- Offline-first per Rule digitalandrew#1 / Rule digitalandrew#37: signify's
  TRUSTED_CERTIFICATE_STORE ships MS Authenticode roots — no
  runtime fetch.
- Never raises: every I/O / parse / verify failure mode maps to
  verdict fields (signed=False / chain_status='unknown' / error=str).

Maps signify's AuthenticodeVerificationResult enum to ChainStatus:
- OK → valid_now (subsumes valid_at_signing).
- CERTIFICATE_ERROR + has_timestamp → revoked (was valid at signing).
- CERTIFICATE_ERROR without timestamp → never_valid (pessimistic).
- VERIFY_ERROR / INVALID_DIGEST / INCONSISTENT_DIGEST_ALGORITHM /
  COUNTERSIGNER_ERROR / INVALID_ADDITIONAL_HASH → never_valid.
- PARSE_ERROR / UNKNOWN_ERROR / NOT_SIGNED → unknown.

backend/tests/test_authenticode_service.py: 21 contract tests:
- 12 parametric mapping tests covering all AuthenticodeVerificationResult
  enum values × has_timestamp boolean.
- 3 file-handling tests (missing, corrupt-bytes, unsigned-PE) verify
  the never-raises contract.
- 5 mock-injected verdict tests covering OK / revoked-with-timestamp /
  dual-signed / no-signatures / explain_verify-raises paths.
- 1 shape-contract test verifying verdict fields ↔ WindowsPESignature
  columns drift-detection (catches future schema changes that don't
  update the verdict).

Test: pytest tests/test_authenticode_service.py -v → 21 passed in 0.82s.
Real exit=0.

Real-PE Rule #35b live canary deferred to Phase β.7 — the MCP-tool
integration test path has access to fixture PEs in firmware extraction
trees, which is where signed Microsoft binaries actually live.

Branch: feat/windows-phase-alpha-2026-05-07.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…se β.5)

Adds `detect_pe_arch_view(path)` to format_detection — a lief-backed
inspector that returns `{primary, secondary, divergence_score}` for
ARM64EC / ARM64X bimorphic PEs and `None` for single-arch images.
Hooked into AuthenticodeVerdict (new `arch_view` field) so every
verdict ships the bimorphic discriminator regardless of signing state.
The Phase β.7 background runner persists the dict onto
`WindowsPESignature.arch_view` JSONB; NULL is the durable signal for
single-arch.

Persona-E digitalandrew#2 + digitalandrew#3: ARM64EC = native ARM64 with x64-compatible ABI
surface (Win11 emulator layer). ARM64X = true bimorphic ARM64 + AMD64
in one PE (Win11 system DLLs). Bug classes can hide in only one half
of an ARM64X binary — the discriminator gates the analyst's view so
a vulnerability in the AMD64 path isn't missed while reading the
ARM64 path.

Predicate table (declarative; mirrored in tests):
  is_arm64x  → primary=arm64x,  secondary=amd64
  is_arm64ec → primary=arm64ec, secondary=x64_abi
ARM64X takes precedence over ARM64EC (an ARM64X binary's machine
type can be reported as ARM64EC; the predicate is the authoritative
gate). Divergence score:
  ARM64X:  count of ARM64X dynamic-fixup relocations from
           load_configuration.dynamic_relocations[].fixups[].
  ARM64EC: load_configuration.chpe_metadata.redirection_metadata_count.
Both default to 0 if the load-config / CHPE structures aren't readable
(defensive — verdict still ships).

Per Rule digitalandrew#30 lief is lazy-imported inside the function so
detect_format()'s hot path (every upload) stays lief-free.

Per Rule #35c the new JSONB sub-key gets the full normalizer +
stamp + schema_version triplet:
  WINDOWS_PE_SIGNATURES_ARCH_VIEW_SCHEMA_VERSION = 1
  _normalize_windows_pe_signatures_arch_view(value) → dict | None
  _stamp_windows_pe_signatures_arch_view(payload)   → dict | None
Three named consumers planned: authenticode_service writer (β.4/β.5),
windows_pe_signature MCP tool reader (β.7), PeHardeningPage frontend
(β.6). None / empty-in-None-out preserves the single-arch null contract.

Per Rule digitalandrew#4 the verdict's `arch_view` field is mirrored 1:1 onto the
WindowsPESignature.arch_view column; the existing
test_verdict_maps_to_windows_pe_signature_columns drift-detector now
asserts the new field's presence on both sides.

Acceptance pytest:
  pytest tests/test_format_detection.py tests/test_authenticode_service.py \
         tests/test_jsonb_normalizers.py tests/test_windows_pe_signature_model.py -v
  → 30 β.5-specific tests pass; 1 pre-existing α.3 inconsistency
  (test_capability_notes_only_for_partial_or_none, WINDOWS_DRIVER_PACKAGE
  carries operator-hint note despite capability=FULL — out of scope here).

Rule digitalandrew#11 import smoke (in rebuilt backend container):
  from app.services.format_detection import detect_pe_arch_view
  from app.services.authenticode_service import AuthenticodeVerdict, verify_pe_file
  from app.services.jsonb_normalizers import (
      WINDOWS_PE_SIGNATURES_ARCH_VIEW_SCHEMA_VERSION,
      _normalize_windows_pe_signatures_arch_view,
      _stamp_windows_pe_signatures_arch_view,
  )
  → all green; AuthenticodeVerdict().arch_view defaults to None.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…se β.8)

Walks each firmware's hardware_firmware_blobs, runs verify_pe_file per PE
(MZ-magic pre-filter + run_in_executor), and persists one
WindowsPESignature row per PE. Drives the firmware.authenticode_chain_*
202+poll status columns from β.3 through the Rule digitalandrew#33 contract:
idle → queued → running → completed (success) | failed (session error).

New service: backend/app/services/authenticode_chain_runner.py
  - DIRECT_MAPPED frozenset — single source of truth shared with the
    drift-detector test (test_verdict_maps_to_windows_pe_signature_columns
    now imports DIRECT_MAPPED). Per β.5/β.6 postmortem rec digitalandrew#3.
  - _is_pe_file(path) — MZ-magic pre-filter so non-PE blobs (ELF / MBN /
    DTB) don't waste signify cycles. Defensive on missing-paths /
    directories / short-files (Rule digitalandrew#19 evidence-first).
  - _verdict_to_signature_kwargs(verdict) — spreads only DIRECT_MAPPED
    keys, decoupled from the row-construction site for testability.
  - verify_firmware_pe_chain(firmware_id, db) — sequential per-PE
    iteration (Rule digitalandrew#7: NEVER share an AsyncSession across coroutines;
    Rule digitalandrew#5: blocking work via loop.run_in_executor). Per-PE error
    containment per design constraint digitalandrew#5: a single failed verify_pe_file
    captures into aggregate["errors"] with chain_status='unknown';
    the run still completes successfully. Re-run idempotency via DELETE
    of prior WindowsPESignature rows.
  - run_authenticode_chain_background(firmware_id) — outer detached
    runner, owns its own AsyncSession via async_session_factory(),
    transitions queued → running → completed/failed. Mirrors
    _run_cve_match_background in routers/hardware_firmware.py.

Aggregate shape (matches the existing Rule #35c normalizer doc + the
β.3 migration docstring; schema-version stamped via
_stamp_firmware_authenticode_chain_result):
  {signed_count, signed_pct, unsigned_count, dbx_revoked_count,
   by_chain_status: {valid_at_signing, valid_now, revoked, never_valid,
   unknown}, run_seconds, total_pe_count, errors: [{blob_path, error}]}

New schemas (backend/app/schemas/hardware_firmware.py):
  - AuthenticodeChainAggregate — final-result Pydantic shape;
    extra='ignore' so the JSONB schema_version stamp is stripped at
    read time without rejecting the row.
  - AuthenticodeChainStatusResponse — 202+poll status snapshot;
    mirrors CveMatchStatusResponse field-for-field.

New endpoints (backend/app/routers/hardware_firmware.py):
  - POST /api/v1/projects/{project_id}/hardware-firmware/authenticode-chain
    — idempotent 202 ack; returns 409 if status is already queued/running;
    schedules asyncio.create_task(run_authenticode_chain_background).
  - GET .../authenticode-chain/status — current snapshot for the
    frontend's 2 s polling loop. Mirrors cve-match polling shape.

Why 202 rather than 200: Win11 23H2 ISOs hold 1000+ PEs; signify
verification is ~50-200 ms per PE; full walk runs 1-3 minutes — past
nginx default proxy_read_timeout (60s) and Cloudflare origin-response
default (100s). The 202+polling shape decouples the work from any
reverse-proxy ceiling (CLAUDE.md Rule digitalandrew#29 + Rule digitalandrew#33).

Tests added (backend/tests/test_authenticode_chain_runner.py — 16 tests):
  - 5 unit tests for _is_pe_file (MZ true; non-MZ / short / missing /
    directory all false without raising).
  - 2 unit tests for DIRECT_MAPPED + _verdict_to_signature_kwargs
    (frozen-set immutability, indirect fields dropped).
  - 6 Rule #35b live-canary tests for verify_firmware_pe_chain (one
    row per PE; per-PE error containment; re-run idempotency;
    dbx_revoked count + persisted columns; empty-firmware no-op;
    verdict.error captured in errors[]). Uses tests._live_db.make_live_db.
  - 3 outer-runner status-transition tests (completed path; failed
    path with truncated traceback; missing-firmware no-op).

Drift-detector update (backend/tests/test_authenticode_service.py):
  test_verdict_maps_to_windows_pe_signature_columns now imports
  DIRECT_MAPPED from authenticode_chain_runner instead of redeclaring
  the literal — single source of truth keeps β.8 + the test in sync as
  Phase γ/δ add new verdict fields.

Validation:
  - Rule digitalandrew#11 import smoke: runner module + 2 schemas + new endpoints
    all import cleanly via /app/.venv/bin/python (post docker cp +
    restart).
  - Rule digitalandrew#20 docker cp + restart (no class-shape change on
    AuthenticodeVerdict — that happened in β.7; β.8 adds new modules
    + Pydantic classes only).
  - Targeted pytest: 16/16 runner tests pass; 35/35 authenticode_service
    tests pass (drift-detector green via DIRECT_MAPPED import); 299/299
    across the wider authenticode/dbx/rich_header/format_detection/
    jsonb_normalizers/windows_pe_signature_model sweep.
  - Rule #35a real-exit-code discipline: caught the test-fixture
    substring bug ("unsigned.dll" matched naive "if 'signed' in path"
    fake) when the test reported FAILED with pytest_rc captured
    post-pipe; re-validated with file-redirect (`> /tmp/x; rc=$?`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Promotes the 3 candidate CLAUDE.md rule additions identified in β.12's
postmortem rec digitalandrew#3, bundled per Rule digitalandrew#25 + Rule digitalandrew#36/digitalandrew#37 "worked-example-
first" promotion shape. All three follow the same discipline as β.13's
Rule digitalandrew#36/digitalandrew#37 promotions: each has at least Rule-of-Two worked examples
already in tree before the rule text lands.

(a) Rule digitalandrew#25 single-slice exception digitalandrew#2 — cross-stack alignment tests
    (sub-clause appended to existing Rule digitalandrew#25). Worked examples:
      • 7079b4d (alembic 61b147189fcf_close_findings_source_drift.py
        + frontend FindingSource union + FINDING_SOURCE_CONFIG mirror,
        2026-05-06)
      • ee2abd9 (β.12a alembic c5b6a7d8e9f0_extend_findings_source_
        windows_verdicts.py + same frontend mirror, 2026-05-08)
    Both bundled DB CHECK + frontend mirror in one atomic commit
    because test_finding_source_alignment.py enforces strict pairwise
    agreement; splitting leaves the alignment test RED between commits
    and breaks bisect-clean lanes. Rule-of-Two; pattern is durable.

(b) Rule digitalandrew#33 .c clarification on Pydantic Literal typo-gate location
    (subtlety appended to clause c). Worked example:
      • b67f062 (β.12b WindowsFindingSource = Literal[
        "windows_authenticode", "windows_dbx_revoked"]) — narrow Literal
        at the new helper's boundary (_PEFindingDraft.source typed
        constants); the permissive FindingCreate.source: str field
        unchanged so 18 legacy callers continue to work; DB CHECK
        ck_findings_source enforces the full 20-source allowlist as
        the safety floor.
    The naïve read of Rule digitalandrew#33 .c was "tighten the schema field to a
    Literal" — that's the wrong move when N legacy callers pass
    runtime-derived strings. The right read is "Literal at the typo-
    gate, schema field stays permissive".

(c) New Rule digitalandrew#38 — Bash absolute-path discipline. Worked examples:
      • β.10 antipattern digitalandrew#3 (originally caught — Rule-of-One)
      • β.12 postmortem "What Broke" digitalandrew#1 (CWD drift after `cd backend
        && uv run pytest tests/test_finding_service_pe_emit.py`;
        recovered via absolute paths) — Rule-of-Two
    The Bash tool's working directory IS persisted across calls;
    `cd X && ...` compounds leak into subsequent calls and produce
    "no such directory" failures. The system instructions already
    recommend this discipline; this rule is the wairz-specific
    reinforcement with worked-example incidents. How to apply:
    `git -C /home/dustin/code/wairz <subcmd>` for git invocations;
    `( cd backend && ... )` subshell form for cwd-sensitive tools.

Per Rule digitalandrew#21 mirror discipline: .mex/context/conventions.md Verify
Checklist updated in the SAME commit — Rule digitalandrew#25 bullet extended with
the alignment-test exception, Rule digitalandrew#33 .c bullet extended with the
typo-gate clarification, new Rule digitalandrew#38 bullet added at end of list,
last_updated frontmatter bumped 2026-05-04 → 2026-05-08. Out-of-sync
state rots fast (Rule digitalandrew#21 worked example).

Companion edit: CLAUDE.md "rules 1–35 above" reference (in the .mex/
companion-scaffold section) bumped to "rules 1–38 above" — β.13's
Rule digitalandrew#36/digitalandrew#37 promotion missed this stale phrase; corrected here.

Verification: no code change → no test regression possible. CLAUDE.md
+ .mex/context/conventions.md are doc-only; the rule text describes
existing β.4-β.12 worked examples that already shipped.
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…riplet (Rule-of-Three)

Promotes the inner/outer/safe runner triplet pattern to CLAUDE.md Rule digitalandrew#39
after Rule-of-Three confirmation across windows-coverage-godmode phases:
- γ.4 (3161a70) — registry_hive_walker.py — implicit first application
- δ.5 — windows_update_diff_service.py — first explicit codification
- ε.1.b.3 (c0e4979) — evtx_service.py — second explicit codification

Rule shape: any background runner that owns a Rule digitalandrew#33 .a 5-state column
ships as exactly three functions —
- `_do_<op>_run(db, firmware_id) -> dict` — INNER pure-logic, accepts db,
  no commit, no status mutation, tier-1 testable via make_live_db()
- `run_<op>_background(firmware_id) -> None` — OUTER state-machine,
  owns async_session_factory(), owns 5-state transitions, outer guard
  catches escapes, failure persistence on fresh session
- `auto_<op>_firmware_safe(firmware_id) -> None` — UNPACK-POST-DETECTION
  hook, owns own session, swallows exceptions, leaves status `idle` so
  manual re-trigger works without 409

Pitfall codified: tier-1 tests MUST call the inner runner — the outer
wrapper opens async_session_factory() which raises socket.gaierror on
dev host (Docker DNS unreachable).

Files updated in this single commit per Rule digitalandrew#21 mirror discipline + Rule
digitalandrew#25 single-slice exception (cross-scaffold sync — CLAUDE.md, conventions
mirror, and recipe land together):
- CLAUDE.md — adds Rule digitalandrew#39 + updates "rules 1-38" → "rules 1-39"
- .mex/context/conventions.md — Verify Checklist mirror entry
- .mex/patterns/INDEX.md — recipe row
- .mex/patterns/inner-outer-safe-runner.md — full recipe (~280 LOC)

Closes ε.1.b postmortem Recommendations digitalandrew#3 + digitalandrew#4.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ross β.14a/γ.9/δ.9)

New `.mex/patterns/real-firmware-skip-tier-canary.md` recipe codifying
the 3-tier real-firmware end-to-end canary discipline that emerged
across three windows-coverage-godmode applications:

| # | Phase | Commit | Pipeline |
|---|-------|--------|----------|
| 1 | β.14a | 77b257d | Authenticode chain validation |
| 2 | γ.9 | 8437ae3 | Registry hive walk + driver INF/CAT classifier |
| 3 | δ.9 | 1f09179 | .NET decompile + KB-diff + R2R-stomping |

Three independent pipelines following the same shape — Rule-of-Three
durable. δ patterns file Pattern digitalandrew#1 explicitly recommends promoting
to a recipe at this point.

**Tier shape:**
- Tier 1 (always runs) — synthetic data with third-party libs mocked
  at SOURCE module per Rule digitalandrew#30; drives the FULL pipeline; uses
  `make_live_db()` from `tests/_live_db.py` for real ORM round-trip;
  asserts persisted rows + value-flow not just call shape (Rule #35b).
- Tier 2 (skip-unless `WAIRZ_TEST_REAL_<X>`) — real single artefact,
  no mocks; assertions are well-shaped not exact-value (real
  artefacts vary).
- Tier 3 (skip-unless `WAIRZ_TEST_<X>_PAIRED`) — paired real artefact
  for compose-style pipelines (KB-diff older/newer); skip entirely
  if pipeline has no compose stage.

Each tier's skipif reason field documents the env-var name so the
operator can graduate the canary set from N pass + M skip → (N+M)
pass via fixture commits — no test code edits needed.

**Inner-vs-outer runner discipline (δ pattern digitalandrew#2 / antipattern digitalandrew#3):**

Tier-1 cannot call a `run_<op>_background()` outer runner directly —
`async_session_factory()` resolves `DATABASE_URL` with hostname
`postgres` (Docker service name); host pytest can't resolve Docker
DNS, raising `socket.gaierror`. The recipe codifies the fix shape:
expose an inner `_do_<op>_run(db, firmware_id)` that accepts a db
arg + does the actual work; the outer wrapper owns the Rule digitalandrew#33 .a
state machine + uses async_session_factory(). Tier-1 calls the
inner runner; outer wrapper is exercised in the running container.

γ.4's `auto_walk_firmware(fw.id, db)` was the implicit precedent;
δ.5's `_do_diff_run(db, firmware_id)` codified the discipline with
a refactor on first canary run. Mechanical heuristic in the recipe:
"if the runner takes only `firmware_id`, factor out an inner
function that takes `(db, firmware_id)`."

**Mock patch target (Rule digitalandrew#30):**

Recipe explicitly cites: tier-1 mocks the THIRD-PARTY library
SOURCE module (`signify.authenticode.AuthenticodeFile`,
`regipy.RegistryHive`, `dnfile.dnPE`), NEVER the wairz service
module. Lazy imports inside function bodies make
`patch("app.services.X_service.Symbol")` a silent no-op.

**Sections:** Context (Rule #35b reasoning + Rule-of-Three table) /
Decide (3 vs 2 tiers; >3 = over-engineering) / Steps (7 — identify
stages, identify inner-vs-outer split, write header, tier-1, tier-2,
tier-3, run+commit) / Gotchas (8) / Verify (10) / Debug (6) /
Update Scaffold / References.

INDEX.md updated with the new entry.

Refs:
- CLAUDE.md Rule digitalandrew#30 (mock patch target for lazy-imports)
- CLAUDE.md Rule digitalandrew#33 (202+polling — establishes inner-vs-outer split)
- CLAUDE.md Rule #35b (live canaries verify value flow)
- δ Pattern digitalandrew#1 (`.planning/knowledge/windows-coverage-godmode-delta-2026-05-09-patterns.md`)
- δ Pattern digitalandrew#2 (inner-vs-outer runner split — same file)
- Implementation precedents: 77b257d, 8437ae3, 1f09179
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Closes the windows-coverage-godmode housekeeping campaign extended for
ε.2.A + ε.2.B + the production-merge consolidation that landed γ + δ +
ε.1.b + Rule digitalandrew#39 + ε.2.A into main via direct push of ε's tip.

Production state at close-out:
- main = 86f4028 (was 5+ weeks stale at e2fd35e; now contains 753 commits
  spanning α + β + γ + δ + ε.1.a + ε.1.b + Rule digitalandrew#39 + ε.2.A +
  postmortem-followups + gitignore)
- alembic head: f1a2b3c4d5e6 (ε.2.A migration)
- 4 services healthy (backend / worker / postgres / redis)
- 219 MCP tools registered
- 13/13 ε.2.A tests pass + 9/9 ε.2.B tests pass

Defensive checkpoint pushed: tag pre-windows-coverage-merge-2026-05-09 at
old main e2fd35e (rollback path: git push origin <tag>:main --force).

Deferred to next session (handoff-ready on feat/post-merge-eps2bc-zeta1-2026-05-09):
- ε.2.C — search_events MCP tool (paginate windows_event_records)
- ζ.1 — Amcache finding emit (via existing windows_registry_extracts;
  cross-stack alignment extending ck_findings_source)
- Close 4 OPEN PRs that are now UI-only residue (digitalandrew#1 δ, digitalandrew#2 postmortem-
  followups, digitalandrew#3 ε, digitalandrew#5 gitignore — content in main, but their head/base
  SHAs don't auto-resolve)
- CI investigation (PRs showed CI failures; local rebuild + smoke pass)
- Tier-2/3 EVTX fixture provisioning (operator setup for canary upgrades)

Knowledge artefacts:
- postmortem-windows-coverage-godmode-housekeeping-plus-eps2-zeta-merge-2026-05-09.md
- Campaign file moved to .planning/campaigns/completed/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ase ζ.2.E)

Phase ζ.2.E ships 3 MCP tools surfacing the ζ.2.B Prefetch walker
verdicts:

1. search_prefetch_records — paginate windows_prefetch_records by
   executable_name (exact) and last_run_time range. Returns up to
   500 rows per page with total_count for navigation. Order:
   last_run_time DESC (NULL last). Mirrors ε.2.C search_events shape.
2. prefetch_walk_status — Rule digitalandrew#33 status reader for the firmware-row
   prefetch_walk_* state machine (idle / queued / running / completed
   / failed) + last-known-result aggregate.
3. trigger_prefetch_walk — Rule digitalandrew#33 .a idempotent POST + 409-on-conflict
   trigger for run_prefetch_walk_background. Resets stale state on
   fresh runs; schedules via asyncio.create_task (Rule digitalandrew#33 .d for
   in-process pure-Python work). Uses flush() not commit() (Rule digitalandrew#3).

Files:
- backend/app/ai/tools/windows_prefetch.py — the 3 handlers + register
  function. Imports run_prefetch_walk_background lazily (Rule digitalandrew#30) so
  windowsprefetch absence degrades gracefully at MCP-init time.
- backend/app/ai/__init__.py — registers register_windows_prefetch_tools
  in create_tool_registry().
- backend/tests/test_windows_prefetch_tools.py — 12 tier-1 tests:
  * register-count canary (3 tools)
  * 5 search tests (empty / pagination / filter / invalid time / limit
    bounded)
  * 2 status tests (idle default / firmware-not-found)
  * 4 trigger tests (409-on-running / 409-on-queued / reset-on-idle
    with monkeypatched runner / firmware-not-found)
  Tests use make_live_db Rule #35b live canaries — SELECT the row to
  verify the trigger reset persisted, not just that the handler returned
  scheduled=true. Patches run_prefetch_walk_background at the source
  module per Rule digitalandrew#30 (function-body lazy import in handler).

Verification:
- Rule digitalandrew#11 import smoke clean — create_tool_registry() returns 223
  total tools (was 220, +3 from this commit).
- Targeted pytest: 12/12 prefetch tools tests pass; 31 cumulative pass
  across the ζ.2 surfaces (model + walker + tools + alignment).
- Full pytest: 3646/3646 pass excluding 8 pre-existing failures
  (alembic-autogenerate dev-host pw, binary_analysis cpu_rec, etc.;
  unrelated to ζ.2 — confirmed via prior baseline stash check).
- Ruff clean.
- Pattern digitalandrew#6 lower-bound count sweep clean — the 3 == N assertions in
  test_windows_dotnet_tools.py are per-category register-fn counts
  (stable-by-design, not the global registry growing count). Global
  registry tests already use >= per ε.1.b CI-recovery commit relaxation.

Phase ζ.2 (Prefetch walker) now COMPLETE — ζ.2.A migration + model +
JSONB normalizers; ζ.2.B Rule digitalandrew#39 inner/outer/safe runner triplet
(Rule-of-Four: γ.4 + δ.5 + ε.1.b.3 + ζ.2.B); ζ.2.C+D cross-stack
alignment (Rule digitalandrew#25 single-slice exception digitalandrew#2 Rule-of-Seven); ζ.2.E
MCP tool category (this commit). Total +1102 LOC across 4 commits,
all bisect-clean per Rule digitalandrew#25 per-sub-task discipline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ζ.3.E)

Phase ζ.3.E ships 3 MCP tools surfacing the ζ.3.B SRUM walker verdicts:

1. search_srum_records — paginate windows_srum_records by record_type
   (one of: network_data_usage, network_connectivity,
   application_resource_usage, push_notification, energy_usage),
   app_identifier (exact), and recorded_at range. Returns up to 500
   rows per page with total_count for navigation. Order: recorded_at
   DESC (NULL last). Mirrors search_prefetch_records / search_events.
2. srum_walk_status — Rule digitalandrew#33 status reader for the firmware-row
   srum_walk_* state machine + last-known-result aggregate.
3. trigger_srum_walk — Rule digitalandrew#33 .a idempotent POST + 409-on-conflict
   trigger for run_srum_walk_background. Schedules via
   asyncio.create_task. Uses flush() not commit() (Rule digitalandrew#3).

Files:
- backend/app/ai/tools/windows_srum.py — the 3 handlers + register
  function. Imports run_srum_walk_background lazily (Rule digitalandrew#30).
  search_srum_records validates record_type against the 5-value
  enum BEFORE querying (typo-gate before round-trip).
- backend/app/ai/__init__.py — registers register_windows_srum_tools.
- backend/tests/test_windows_srum_tools.py — 10 tier-1 tests:
  * register-count canary (3 tools)
  * 4 search tests (empty / record_type filter / invalid record_type /
    app_identifier filter)
  * 2 status tests (idle default / firmware-not-found)
  * 3 trigger tests (409-on-running / reset-on-idle with monkeypatched
    runner / firmware-not-found)
  Tests use make_live_db Rule #35b live canaries — SELECT the row to
  verify trigger reset persisted.

Verification:
- Rule digitalandrew#11 import smoke: create_tool_registry() returns 226 total tools
  (was 223 before this commit, +3).
- Targeted pytest: 28/28 pass across all ζ.3 surfaces (model + walker
  + alignment + tools).
- Full pytest: 3688/3688 pass excluding 8 pre-existing failures
  (alembic-autogenerate dev-host pw, etc.; unrelated to ζ.3).
- Ruff clean.
- Pattern digitalandrew#6 lower-bound count discipline preserved — register-count
  canary uses >= N, not == N.

Phase ζ.3 (SRUM walker) now COMPLETE — ζ.3.A migration + table + status
columns + ORM model + JSONB normalizers; ζ.3.B Rule digitalandrew#39 inner/outer/safe
runner triplet (Rule-of-Five: γ.4 + δ.5 + ε.1.b.3 + ζ.2.B + ζ.3.B);
ζ.3.C+D cross-stack alignment with 2 source values (Rule digitalandrew#25 single-
slice exception digitalandrew#2 Rule-of-Eight); ζ.3.E MCP tool category (this commit).
Total +2274 LOC across 4 commits, all bisect-clean per Rule digitalandrew#25.

The SRUM walker is the highest-value forensic source after EVTX —
answers "did this binary make network connections in the last 30 days?"
without re-running the firmware.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…e (postmortem digitalandrew#3)

Postmortem `async-cleanup-2026-05-12-structured.md` Recommendation digitalandrew#3
flagged campaign-open ledger drift — that campaign stated
`[tool.ruff.lint] ignore` would shrink 30 → 26 (4 ASYNC removals) but
actual was 30 → 24 (2 incidental commented-placeholder cleanups also
landed during ASYNC-removal edits).  The +2 drift was only caught at
structured-postmortem time, after the campaign was already closed.

Mitigation: campaign-open template now requires (optionally — only when
the campaign has a quantitative target) a `## Ledger Math Baseline`
section between `## Claimed Scope` and `## Phases`.  The section records:

- **Quantitative target** (one-line description, e.g. "ignore shrinks 30 → 26")
- **Count** (literal numeric measurement)
- **Named list** (full enumerated entries — the critical anti-drift capture)
- **Date** (when the baseline was taken)

Includes a reusable awk example for the ruff ignore-list metric source —
verified live against `backend/pyproject.toml`:

  Count: 24
  Named: B007,B008,B017,B023,B904,B905,E402,E501,F401,F821,F841,S101,
         S104,S105,S108,S110,S112,S202,S324,S603,S607,UP040,UP042,UP046

The named-list capture is the new mechanism: future campaigns can diff
their open-time enumerated list against the close-time list at
structured-postmortem time, catching drift even when totals happen to
match (a swap of one entry for another would have zero-sum totals but
non-empty diff).

Section is OPTIONAL — explicit `<!-- Skip this section entirely for
campaigns with no quantitative target -->` so pure-feature builds and
qualitative refactors aren't forced to invent a fake metric.

Mirror discipline (Rule digitalandrew#21): `.mex/patterns/INDEX.md` has no campaign-open
recipe — no mirror update needed.  (Confirmed via Read of INDEX.md.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Three parallel research-fleet scouts dispatched 2026-05-12T00:55Z to
brief the Phase θ Windows-coverage horizontal-expansion scope decision.
All 3 returned within 4 minutes wall, independent (research-fleet wave
discipline preserved — none read another's output before publishing).

- Scout 1 (theta-scout1-oss-lib-survey.md, 130 lines) — OSS Python lib
  availability + Linux/wairz fit + integration complexity for each of
  the 8 deferred-to-θ candidates. Top 3 picks (MEDIUM-or-better +
  small-or-medium complexity): digitalandrew#1 BCD store (zero new dep — regipy
  already in tree), digitalandrew#2 EVT pre-Vista (libevt-python fresh), digitalandrew#3 SDB
  shim (fork-and-vendor python-sdb).

- Scout 2 (theta-scout2-persona-e-adversary.md, 128 lines) — Persona-E
  adversary refresh, ranked by adversary-coverage-value × commercial-
  EDR-blind-spot. Top 3: digitalandrew#1 WMI persistence (T1546.003 across APT29,
  APT32, Turla, FIN7), digitalandrew#2 ETL (2024-25 anti-forensic surviving-cleanup
  blind spot), digitalandrew#3 Boot chain (BlackLotus, Bootkitty Nov 2024,
  CosmicStrand, MoonBounce).

- Scout 3 (theta-scout3-competitive-parity.md, 191 lines) — Competitive
  RE/forensic-platform parity vs EZTools, FLARE-VM, Volatility, KAPE,
  Velociraptor, Plaso, Autopsy. Top 3: digitalandrew#1 WMI (EZTools = no parser;
  flare-wmi unmaintained since 2018; MCP-exposed cross-firmware
  aggregation is wairz-unique), digitalandrew#2 Boot chain (wairz uniquely owns
  surrounding primitives — UEFI volumes + Authenticode + DBX + BYOVD;
  adding BCD/MBR/ESP makes wairz the only OSS static-analysis
  platform for the complete UEFI→Windows-userland boot chain), digitalandrew#3
  Shim .sdb.

Cross-scout convergence: BCD 3/3 HIGH (strongest signal); WMI 2/3 digitalandrew#1
(Scout 1 flags vendor-in complexity); Volatility+hiberfil 3/3 DEFER;
ETL 1/3 outlier (Scout 2 wants, Scouts 1/3 defer); EVT 1/3 outlier.

Synthesized in `.planning/campaigns/windows-coverage-godmode-theta-2026-05-12.md`
(next commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Synthesises the 3-scout research-fleet pre-pass (commit b65fc60) into
a Phase ι campaign brief.

Scope evolution: ι is the FIRST CROSS-PLATFORM coverage campaign.
Scouts 2 + 3 both converge HIGH on Linux journald + systemd
persistence as the strategic top pick, breaking the Windows-only
η + θ pattern. The brief retains "windows-coverage-godmode" prefix
for series continuity but the scope evolves to "wairz cross-platform
forensic coverage."

Recommended Phase ι execution order:
- ι.A — Linux journald walker (FIRST LINUX WALKER; campaign-defining
  milestone; ~40 min agent-wall first-stream precedent)
- ι.B — systemd unit file persistence walker (pairs with ι.A; pure
  stdlib INI parser, zero new dep)
- ι.C — ETL via dissect.etl (Scout 1 dramatic reversal: fresh Fox-IT
  pick; Rule digitalandrew#19 evidence-first API probe gate)
- ι.D — EFS DDF/DRF metadata walker (parse-only; reuses η.A NTFS
  walker shape; zero new dep)

Deferred:
- Volatility 3 + hibernate.sys — 2/3 DEFER. MemProcFS already shipping
  MCP eliminates wairz's "first OSS memory-forensic with MCP" wedge;
  re-evaluate at κ.
- Container runtime — Scout 2 classes κ candidate; Scout 3 digitalandrew#3 ship
  with modest engineering cost caveat. Carry into κ.
- EVT pre-Vista — 3/3 DEFER (niche audience).
- macOS — out of wairz scope per README.

Patterns to extend at ι.A:
- Rule digitalandrew#39 walker triplet: Rule-of-Thirteen → Rule-of-Fourteen
- Rule digitalandrew#25 cross-stack alignment: Rule-of-Eighteen → Rule-of-Nineteen
  (NEW: LinuxFindingSource Literal sibling of WindowsFindingSource)
- Pattern P1 single-sub-agent: Rule-of-Five → Rule-of-Six

Risks surfaced:
- First Linux walker detection-root spot-check at ι.A.C
- dissect.journal availability vs Kaitai fallback
- Scout 3 audience-mismatch concern for ETL (hypothesis to test post-stream)
- LinuxFindingSource Literal: extend test_finding_source_alignment.py
  to enforce pairwise agreement for Linux sources too

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Single-scout 3-angle research brief on Acronis True Image .tibx backup
format support for wairz. Surfaced during 2026-05-12 Medtronic
ILLUMISITE upload smoke (4 × 4GB .tibx files locked inside the outer
recovery ISO).

Verdict: HOLD/soft-NO-GO at current intake strength. No AGPL-compatible
.tibx parser exists; only OSS tool (`dennisss/acronis-tib`, MIT, 13
commits) supports legacy .tib not .tibx. Writing parser from scratch is
multi-month libyal-tier reverse engineering chasing a yearly-updated
proprietary format. Strategic floor: N=1 Medtronic encounter.

CRITICAL FINDING: companion intake digitalandrew#3 surfaces the actual immediate-
value path — the OUTER recovery ISO (bootmgr, BCD, .efi, boot.wim,
Acronis driver payload, signed binaries) is ALREADY walkable via
existing unpack_iso9660.py + unpack_wim.py once the walker
auto-trigger gap (separate intake) lands. The .tibx wrapper isn't
blocking the meaningful test surface.

Revisit triggers documented: (a) 2+ additional .tibx encounters across
device families, OR (b) credible OSS .tibx reader with encryption
support emerges.

3 companion intakes recommended:
- tibx-magic-and-shape-measurement-2026-05-12 (1-2 hr xxd investigation)
- acronis-recovery-pe-iso-walker-2026-05-12 (use existing iso/wim walkers)
- tibx-format-tracking-watchlist (monitor OSS landscape quarterly)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…rew#39 triplet + registry wiring

Sub-task digitalandrew#3 of λ.α.D (Issue digitalandrew#11) — final implementation slice. Wires
the runner from sub-task digitalandrew#1 (``2e76a49``) and the schema from sub-task
digitalandrew#2 (``009801d``) into a working post-extraction walker.

``backend/app/services/windows_info_walker.py``:

- ``_do_windows_info_walk(db, firmware_id) -> dict`` — INNER pure-
  logic orchestrator. Resolves detection_roots via Rule digitalandrew#16 helper
  (sanity check that the firmware has extracted content), then
  iterates ``MemoryDumpImage`` rows whose ``os_family`` is
  ``"windows"`` OR ``"unknown"`` (raw acquisitions carry no magic;
  Vol3's automagic LayerStacker re-classifies them at scan time).
  For each: ``vol3_runner.run_vol3_plugin(image_path, "windows.info")``
  with the runner's Rule digitalandrew#29 600 s default timeout. Stamps per-image
  ``kernel_hint`` (e.g. ``"Windows 10 (NT 10.0.19045) AMD64"``) +
  ``isf_profile_guess`` (16-char bundle ID from the Symbols URL) +
  ``last_walked_at`` directly on the ``MemoryDumpImage`` row.
  Returns per-firmware aggregate UNSTAMPED — outer caller stamps via
  ``_stamp_firmware_windows_info_walk_result``.
- ``run_windows_info_walk_background(firmware_id) -> None`` — OUTER
  state-machine wrapper. Owns its own ``async_session_factory()``
  session; transitions ``firmware.windows_info_walk_status`` through
  ``idle → running → completed | failed``. Failure persistence on a
  FRESH session because the inner session rolled back on the
  exception (Rule digitalandrew#39 canonical shape).
- ``auto_windows_info_walk_firmware_safe(firmware_id) -> None`` —
  UNPACK-POST-DETECTION hook. Fire-and-forget. Stamps the aggregate
  but does NOT mutate ``windows_info_walk_status`` (leaves ``idle``
  so a future operator-driven re-trigger via the λ.δ MCP tool
  ``trigger_windows_info_walk`` succeeds without 409 conflict per
  Rule digitalandrew#33 .a).

Pure helpers exposed for unit-test direct exercise:

- ``_extract_variables`` — Vol3 ``[{Variable, Value, TreeDepth}, ...]``
  → flat ``{Variable: Value}`` dict. Tolerates malformed records.
- ``_derive_kernel_hint`` — composes a human kernel identifier from
  NtMajorVersion + NtMinorVersion + NTBuildLab + MachineType +
  NtProductType. Returns ``None`` when essentials absent.
- ``_classify_os_kernel_family`` — buckets into windows10 / windows11
  / windows_server / unknown. Uses NTBuildLab >= 22000 as the Win11
  cutoff (per Vol3-supplied scout-3 plugin taxonomy).
- ``_derive_isf_profile_guess`` — extracts the canonical 16-char
  bundle ID from the Vol3 Symbols URL. Caps at 64 chars to match the
  ``isf_profile_guess`` column constraint.
- ``_family_from_kernel_hint`` — reverses a kernel_hint back to the
  family bucket. Symmetrical with ``_classify_os_kernel_family`` so
  aggregate classification round-trips correctly.

Per CLAUDE.md Rule digitalandrew#16: ``get_detection_roots(firmware)`` confirmed at
the start of every walk; an empty root list short-circuits to empty
aggregate (no images can exist if no detection roots).

Per CLAUDE.md Rule digitalandrew#29: every Vol3 invocation gets the runner's
default 600 s timeout; the walker doesn't override.

Per CLAUDE.md Rule digitalandrew#33 .a: 5-state machine on
``firmware.windows_info_walk_*`` (added in ``009801d``); CHECK
constraint enforces the Literal at the DB layer.

Per CLAUDE.md Rule digitalandrew#33 .d: ``asyncio.create_task`` dispatch (not arq)
— the work is in-process Python with subprocess-invocation; per-image
state IS the durable state (the ``MemoryDumpImage`` rows persist
across restarts).

Per CLAUDE.md Rule digitalandrew#36 + #45: argv discipline + deny-list discipline
are enforced at the ``vol3_runner`` boundary — the walker NEVER
constructs its own argv and NEVER shells out, only calls
``run_vol3_plugin(plugin="windows.info")``. The Rule #45 walker
source-scan gate ``test_walker_no_decrypt_invocation`` asserts no
decrypt invocation in the walker source; Rule #46 canary
``test_walker_source_scan_gate_canary_fires`` confirms the gate
actually fires on a synthetic violation.

Per CLAUDE.md Rule digitalandrew#39: full inner/outer/safe triplet — same shape
as registry_hive_walker / dpapi_walker / efs_walker / 18 other walker
families. The inner returns aggregate UNSTAMPED; outer stamps +
mutates status; safe stamps only (no status mutation).

Per CLAUDE.md Rule #47: walker_registry.py updated with the new
safe-runner registered immediately AFTER
``auto_memory_image_enumeration_safe`` so the
``_fire_walker_auto_triggers`` sequential dispatch guarantees the
enumerator commits its ``memory_dump_image`` rows BEFORE the
windows_info walker queries them. Comment in the registration list
documents the ordering dependency explicitly. Walker count increments
22 → 23 → 24 (λ.α.B enumerator + λ.α.D walker).

``backend/tests/test_windows_info_walker.py``:

- 25 tests pass under uv-managed pytest 9.0.3 / Python 3.14:
  - 8 pure-helper tests covering ``_extract_variables`` (canonical +
    malformed), ``_derive_kernel_hint`` (Win10 / Win11 / Server / no
    essentials), ``_classify_os_kernel_family`` (all 4 buckets),
    ``_derive_isf_profile_guess`` (bundle ID extraction + None + 64-
    char truncation), ``_family_from_kernel_hint`` (round-trip).
  - 3 ``_walk_one_image`` tests covering success (stamps per-image
    fields), Vol3InvocationFailed (per-image error string,
    classification False), Vol3NotInstalled (propagates to abort
    the whole walk).
  - 3 ``_do_windows_info_walk`` live-canary tests (Rule #35b) via
    ``make_live_db`` SQLite fixture: persists per-image fields +
    aggregate, no-images-graceful, Vol3NotInstalled-aborts-loop.
  - 1 ``run_windows_info_walk_background`` outer-wrapper test —
    verifies idle → running → completed transition + stamped
    aggregate with schema_version=1.
  - 1 ``auto_windows_info_walk_firmware_safe`` test — verifies
    safe runner stamps aggregate but leaves status idle (Rule digitalandrew#33 .a
    re-trigger compatibility).
  - 2 module-config sanity tests (``_WALK_OS_FAMILIES`` shape,
    ``_empty_aggregate`` shape).
  - 4 Rule #45 + #46 source-scan gate tests: no-spawn-primitives, no-
    decrypt-invocation, gate-canary-fires-on-synthetic-violation,
    docstring-mentions-don't-trigger.
- Ruff clean.

Validation:

- 86 tests pass across vol3_runner (51) + windows_info_walker (25) +
  jsonb_normalizers windows_info (10).
- ORM smoke (``from app.workers.walker_registry import
  get_walker_auto_triggers`` + iterate) confirms windows_info_walker
  registered at index 12 (after memory_image_enumerator at index 11).
- Live canary (Rule #35b) confirmed: a fixture firmware + image row
  go through the walker; SELECT against the persisted MemoryDumpImage
  row shows kernel_hint="Windows 11 (NT 10.0.22631)…" +
  isf_profile_guess="bundle1" + last_walked_at populated.

λ.α.D scope (Issue digitalandrew#11) is now complete:

1. ``2e76a49`` — vol3_runner.py + tests (51 tests, all Rule digitalandrew#29/digitalandrew#36/digitalandrew#37/#45/#46).
2. ``009801d`` — alembic + ORM + jsonb_normalizers + tests (10 tests).
3. THIS COMMIT — windows_info_walker.py + walker_registry + tests
   (25 tests).

Next: Rule digitalandrew#8 backend+worker+migrator rebuild + Rule digitalandrew#11 import smoke
+ alembic upgrade head + integration smoke + Rule digitalandrew#21 CLAUDE.md mirror
update (if Rule #45 / #46 application surfaces a Rule-of-Two extension
worth promoting).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…are + Rule #44 cross-firmware)

Surfaces the λ.β.B walker to the MCP layer. Four tools registered:

- ``list_windows_processes`` — paginate per-process rows for the active
  firmware. Filters: image_filename (exact), pid, only_unlinked /
  only_terminated / only_orphan / only_suspicious_path. Order:
  create_time DESC NULLS LAST, pid ASC. 30 KB output cap (Rule digitalandrew#29).

- ``windows_processes_walk_status`` — Rule digitalandrew#33 status reader.

- ``trigger_windows_processes_walk`` — Rule digitalandrew#33 .a idempotent POST +
  409-on-conflict; schedules run_windows_processes_walk_background via
  asyncio.create_task. db.flush() (Rule digitalandrew#3) not commit() inside the
  MCP handler.

- ``lookup_windows_process_across_firmwares`` — **Rule #44 cross-
  firmware aggregation** (wairz's unique forensic differentiator).
  Identity key is SHA256(image_filename + "\\n" + command_line);
  optional command_line so the coarser "is svchost.exe in every
  firmware" survey also works. Returns one entry per matching firmware
  with match_count, sample_process_record, unique_command_lines, and
  ``supply_chain_signal=True`` when match_count >= 2 AND ANY matching
  row has a non-Microsoft image_path_full (vendor-pushed service or
  attacker payload appearing across captures).

Rule digitalandrew#36 / #45 reminder: NONE of these handlers invoke a process
binary, expand a command line, or shell out — they read records AS
DATA. The supply-chain-signal heuristic keys on string SHA256 only.

Closes Issue digitalandrew#17 MCP surface. Tests in next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
… cross-firmware lookup

Surfaces the λ.γ.B walker to the MCP layer. Four tools:

- ``list_windows_injection_detections`` — paginate per-detection rows
  for the active firmware. Filters: detection_kind (one of 5),
  pid, image_filename, hexdump_sha256 (find every detection with the
  same first-64-bytes hash in this firmware). Order: detection_kind
  ASC, pid ASC. 30 KB output cap.

- ``windows_injection_walk_status`` — Rule digitalandrew#33 status reader.

- ``trigger_windows_injection_walk`` — Rule digitalandrew#33 .a idempotent POST +
  409-on-conflict; schedules run_windows_injection_walk_background
  via asyncio.create_task. db.flush() (Rule digitalandrew#3) inside MCP handler.

- ``lookup_volatility_injection_across_firmwares`` — **Rule #44
  cross-firmware aggregation** keyed on hexdump_sha256. The strongest
  cross-firmware signal in the λ chain: an injected code region
  appearing across captures with the same first-64-bytes hash is
  canonical threat-actor TTP-reuse evidence. Optional detection_kind
  filter scopes to one of the 5 kinds (most useful:
  injected_code_region — the kind that produces a hexdump_sha256).
  Returns one entry per matching firmware with match_count,
  sample_detection, detection_kinds, ``supply_chain_signal=True``
  when match_count >= 2.

Rule digitalandrew#36 / #45 reminder: tools read records AS DATA. The
cross-firmware tool keys on a SHA256 of a hex string; it NEVER
fetches any binary, NEVER attempts to deobfuscate injected code,
NEVER invokes a region for execution.

Total MCP tool count: 306 → 310 (4 new injection tools).

Closes Issue digitalandrew#18 MCP surface. Tests in next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…onents)

Forensic-review A2 finding digitalandrew#3 (2026-05-14): the DS1 Moto-G32 313-blob
detection had 0 entries categorized as Broadcom Bluetooth, despite
the firmware ZIP containing BTFM.bin (the Broadcom Bluetooth firmware
archive for Motorola / Qualcomm Bengal Sm6225 platforms). unblob
extracts BTFM.bin into:

    BTFM.bin_extract/raw.image_extract/image/
        cmbtfw{10,11,12,13}.tlv     - common-rev Bluetooth FW (TLV)
        cmbtfw{10,11,12,13}.ver     - paired version files
        apbtfw10.tlv                - additional-rev (Apple-derived)
        crbtfw11.tlv                - additional-rev (chris-rev)
        cmnv{12,13s}.bin            - common-NV config
        crnv21.bin                  - chris-NV config
        apnv11.bin                  - additional-NV config

The existing Broadcom YAML only matched `bcm*.hcd` shape (HCI
firmware) and `brcmfmac*` (Wi-Fi). The BTFM sub-components matched
none, fell through to the qcom-prefix heuristic, and were tagged
"other/qcom_mbn/qualcomm" — wrong vendor, wrong category.

Three new patterns added to firmware_patterns.yaml:

  - `^btfm\.bin$`          → broadcom/bluetooth (raw_bin, high)
  - `^[a-z]{2}btfw\d+\.(tlv|ver)$` → broadcom/bluetooth (raw_bin, high)
  - `^[a-z]{2}nv\d+s?\.bin$`        → broadcom/bluetooth (raw_bin, medium)

End-to-end repro on DS1 Moto-G32 (`firmware_id = eed5db82-...`):
  Before patch: 313 blobs total, 0 Broadcom Bluetooth.
  After patch:  331 blobs total, 18 Broadcom Bluetooth.

The 18 Bluetooth blobs ARE the actual Bengal-platform Broadcom BT
firmware: 8 cmbtfw TLVs (4 versions × .tlv/.ver), 1 apbtfw, 1
crbtfw, 3 NV configs (cmnv12, cmnv13s, crnv21, apnv11), plus the
top-level BTFM.bin archive marker.

9 classifier-pattern tests added covering BTFM.bin + each
sub-component shape. Companion to commit fc08450 (NUL sanitization)
+ commit c0c107e (path-string fallback) — together these resolve
the DS1 0-blob-detection bug end-to-end with correct categorization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Path-context rules for the Android Bluetooth host stack (Bluedroid /
Fluoride) and WiFi supplicant binaries:

  /system/lib*/libbluetooth.so          → bluetooth/aosp
  /apex/com.android.btservices/lib*/    → bluetooth/aosp (Android 12+)
  /vendor/lib*/libbt-vendor-qti.so      → bluetooth/qualcomm (vendor fork)
  /system/bin/bluetoothd                → bluetooth/aosp (legacy)
  /vendor/bin/hw/wpa_supplicant         → wifi/aosp
  /system/bin/wpa_supplicant            → wifi/aosp (GKI)
  /usr/sbin/wpa_supplicant              → wifi/aosp (OpenWrt / generic)
  /(vendor|system)/lib*/libwpa_client.so → wifi/aosp

New `aosp` vendor prefix added to vendor_prefixes.yaml. Distinct from
silicon vendors (qualcomm/broadcom/...) — aosp covers OS-level binaries
that ship under /system + /apex regardless of underlying silicon.

CVE families (known_firmware.yaml — 9 new entries, ~13 distinct CVEs):

  Bluedroid (vendor=aosp + vendor=qualcomm fork):
    CVE-2023-45866 BleedingPodcast HID forced-pairing (CVSS 7.1)
    CVE-2023-40129 GATT build_read_multi_rsp RCE (CVSS 8.8)
    CVE-2023-35673 GATT integer-overflow precursor (CVSS 7.8)
    CVE-2024-43763 GATT DoS (CVSS 5.5)
    CVE-2024-49728 OBEX cross-user disclosure (CVSS 7.5)
    Vendor-side ADVISORY-QTI-BT-HAL-23 emits same CVE IDs against
    libbt-vendor-qti.so / libbluetooth_qti.so (vendor patch lag).

  wpa_supplicant (vendor=aosp):
    CVE-2023-52160 PEAP-MSCHAPv2 phase-2 bypass (Top10VPN 2024)
    CVE-2022-23303 SAE side-channel
    CVE-2022-23304 EAP-PWD side-channel
    ADVISORY-WPAS-DRAGONBLOOD covering CVE-2019-9494/9495/9496/13377

Sources cited per-entry (NVD, Android Security Bulletin SPL dates,
AOSP gerrit commit c0151aa3, Red Hat RHSA-2024:2517, GLSA-202309-16,
Vanhoef Dragonblood research site).

Tests: 6 new path_context tests + 6 new matcher tests covering
AOSP-side firing AND vendor-side firing AND non-target rejection
(e.g. wpa_supplicant CVEs must NOT fire on Qualcomm WLAN firmware
blobs — different attack surface).

Refs:
  * postmortem-btfm-correction-and-corpus-2026-05-15.md (rec digitalandrew#3 + digitalandrew#4)
  * Scout C 2026-05-16 research notes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Recommendation digitalandrew#3 from postmortem-bt-banner-parser-session-2026-05-16.md.
Documents all 5 YAML extension surfaces for operator-extensible firmware
detection + CVE attribution:

1. vendor_prefixes.yaml — canonical vendor names + aliases
2. firmware_patterns.yaml — filename patterns + path_contexts refinement
3. bt_qca_codenames.yaml — QCA codename map + BrakTooth scope + MTK chips (H1)
4. bt_banner_cve_pins.yaml — banner-pin → CVE rule engine (H2)
5. known_firmware.yaml — curated CVE matcher with vendor_regex (2026-05-16)

Covers:
- Architecture recap (classifier → parser → CVE matching pipeline) with
  the content-evidence > filename-evidence rule baked in via
  ParsedBlob.vendor override contract.
- Every field in every YAML file (required vs optional, types, defaults).
- The Format → Parser mapping table with all 10 in-tree parsers.
- Path-context priority handling + Reviewer A M5 deterministic-tiebreaker
  semantics.
- Reviewer B 2026-05-16 NVD-CPE-per-CVE verification discipline + the
  CVE-2021-28139 canary that protects against regression.
- Graceful-degrade behavior table per file (all-or-nothing for H1+H2;
  per-entry skip for classifier YAMLs).
- End-to-end worked example: adding coverage for a hypothetical new QCA
  Hennessy / QCA6490 codename — every file that needs editing, the one
  parser-code change for the filename-prefix map, the Rule digitalandrew#8 rebuild,
  and the SQL verification queries.
- Code pointers + cross-references to the Reviewer B postmortems +
  CLAUDE.md Rules digitalandrew#19 and digitalandrew#34.

Renders cleanly with the existing docs/features/*.md style.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Closes Reviewer A C2 + Reviewer B F-FORENSIC-03 deferred from
postmortem hw-firmware-adaptive-session-2026-05-18 Rec digitalandrew#3. 6 new
curated CVE family entries covering the NVIDIA Tegra/L4T BSP cluster
that DS1 (project d360f8f5) and other Jetson-based firmware images
depend on. Per-CVE NVD CPE list verified via direct WebFetch on
services.nvd.nist.gov/rest/json/cves/2.0?cveId=<CVE> recursively for
EACH of the 6 CVEs — Scout report's NVD claims independently
re-fetched before commit.

Recursive-verification discipline (Reviewer B 2026-05-15..18 caught
CVE-attribution failure modes 4 sessions in a row): every CVE-
attribution claim — whether from user prompt, scout report, or
reviewer finding — gets the direct NVD URL fetch before pinning.
The 6 fetches surfaced 3 user-prompt-discrepancies that NVD
overruled:

  * CVE-2021-1111 — user said "ALL Jetsons"; NVD CPE EXCLUDES
    TX1/Nano (only AGX-Xavier+TX2+TX2-NX+Xavier-NX listed).
    chipset_regex narrowed accordingly.

  * CVE-2021-34372 — user said product=tegra_tos_trusty; NVD CPE
    uses product=jetson_linux. wairz curated families: schema has
    no `product` field (only banner_cve_pin schema does); we narrow
    via vendor=nvidia + category=tee, which is the schematic
    equivalent for the Trusty OTE attack surface.

  * CVE-2022-42269 — user said "AGX-Xavier/TX2/Xavier-family"; NVD
    CPE ALSO includes jetson_tx1. chipset_regex expanded to include
    T210/TX1 (Nano stays excluded — different hardware anchor).

NVD wins per Rule digitalandrew#19 evidence-first in all 3 cases. All 6 pins
satisfy F-FORENSIC-10 narrowing (chipset_regex AND/OR version_regex
present — no family-only attribution).

Per-CVE summary
===============

| CVE | Fix at | Scope | Discrepancy? |
|-----|--------|-------|--------------|
| CVE-2019-5680 | R32.2 | TX1 only | None (R32.2 IS fix) |
| CVE-2021-1111 | R32.6.1 | AGX-Xav+TX2+TX2-NX+Xav-NX | NVD excludes TX1+Nano |
| CVE-2021-34372 | R32.5.1 | ALL Jetsons (TEE) | product=jetson_linux not tegra_tos_trusty |
| CVE-2021-34397 | R32.5.1 | TX2+Xavier family | None |
| CVE-2022-42269 | R32.7.2 | AGX-Xav+TX2+Xav-NX+TX1 | NVD adds TX1 |
| CVE-2022-42270 | R32.7.2 | Xavier only (NVDLA) | None |

Forward-prepared note
=====================

version_regex requires the L4T BSP release string ("R32.x.y" style)
to appear in blob.version OR any blob metadata value. The current
Tegra parser (commit 8054d22) does NOT yet extract L4T release from
blob content — these pins will fire when a future enhancement adds
L4T release extraction (e.g. parse /etc/nv_tegra_release on the
firmware tree → populate firmware.device_metadata["l4t_release"],
OR scan Tegra-blob head for "R<N> (release), REVISION: <x.y>"
banner string and store in blob.metadata).

Strict version_regex is the right discipline NOW per Reviewer B
discipline — better forward-prepared with zero false-positives than
firing on every Tegra blob regardless of L4T version.

Out-of-scope (explicit per user direction)
==========================================

CVE-2021-34373..34396 from the same NVIDIA security bulletin
disclosure batch — DO NOT batch-extrapolate. Each future CVE needs
its own NVD WebFetch verification before pinning. The disclosure-
batch antipattern has bitten wairz 4 sessions in a row (BTFM
2026-05-15, CVE-2021-28139 2026-05-16, CVE-2021-34147/31609/31612
2026-05-17, CVE-2019-5680 Selfblow 2026-05-18); the per-NVD-CPE
recipe applies recursively for any future Tegra coverage extension.

Tests (10 new in test_hardware_firmware_cve_matcher.py)
=======================================================

- test_tegra_cve_pins_all_six_loaded — all 6 entries present
- test_tegra_cve_pins_satisfy_f_forensic_10_narrowing —
  chipset_regex or version_regex on every pin (no family-only)
- test_cve_2019_5680_chipset_regex_tx1_only — accepts TX1,
  rejects TX2/Xavier/Nano (canonical Selfblow scope)
- test_cve_2021_1111_chipset_regex_excludes_tx1_and_nano —
  Rule #46 paired canary for the user-prompt discrepancy
- test_cve_2022_42269_chipset_regex_includes_tx1 —
  Rule #46 paired canary for the TX1-inclusion discrepancy
- test_cve_2022_42270_xavier_only_excludes_tx2 — NVDLA Xavier-only
- test_cve_2021_34397_excludes_tx1_and_nano — TX2/Xavier family
- test_cve_2021_34372_has_no_chipset_regex_per_nvd_all_jetsons —
  confirms intentional chipset_regex absence (ALL-Jetsons scope)
- test_tegra_version_regex_matches_pre_fix_l4t_releases —
  forward-prepared canary that pre-fix L4T release strings WILL
  match when extraction lands
- test_tegra_cve_pins_carry_nvd_url_reference — verifiability
  discipline: each pin's notes cites the NVD URL

66/66 cve_matcher tests pass. No regressions.

Refs:
- Postmortem hw-firmware-adaptive-session-2026-05-18 Rec digitalandrew#3
- NVD CPE WebFetch for each CVE (see notes field URLs)
- CLAUDE.md Rule digitalandrew#19 (evidence-first — NVD CPE list IS truth)
- CLAUDE.md Rule digitalandrew#25 (per-piece commits)
- CLAUDE.md Rule #46 (paired-canary discipline)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ared

Reviewer B 2026-05-15-PM HIGH carried-forward from postmortem-hw-firmware-
tegra-activation-2026-05-15. Per Rule digitalandrew#19 recursive NVD-CPE
verification, NVD CPE narrows CVE-2023-20819 to exactly 6 MediaTek
modem-OS families: lr11 / lr12a / lr13 / nr15 / nr16 / nr17. The
hardware chipset CPEs (66 entries MT2731..MT8798) are sibling
AND-nodes describing the runtime substrate, NOT version anchors.

Pre-narrowing the entry fired on EVERY MediaTek modem blob (20 rows
current corpus) regardless of family. Post-narrowing, the
version_regex restricts firings to blobs whose `version` field
contains an NVD-CPE-affected family token.

NVD-CPE verification (recursive — implementer-side WebFetch in scout
report + this commit re-verifies regex against published CPE strings):
  - cpe:2.3:o:mediatek:lr11:-:*:*:*:*:*:*:*
  - cpe:2.3:o:mediatek:lr12a:-:*:*:*:*:*:*:*
  - cpe:2.3:o:mediatek:lr13:-:*:*:*:*:*:*:*
  - cpe:2.3:o:mediatek:nr15:-:*:*:*:*:*:*:*
  - cpe:2.3:o:mediatek:nr16:-:*:*:*:*:*:*:*
  - cpe:2.3:o:mediatek:nr17:-:*:*:*:*:*:*:*

Regex shape (boundary-aware, case-insensitive):
  (?i)(?:^|[^a-z0-9])(lr11|lr12a|lr13|nr15|nr16|nr17)(?:[^a-z0-9]|$)

Boundary check (`(?:^|[^a-z0-9])` / `(?:[^a-z0-9]|$)`) prevents false
positives on confused substrings — e.g. `nr155_else` (nr155 ≠ nr15)
and `lr12abc` (lr12abc ≠ lr12a). Tolerates dot, underscore, and
end-of-string separators per the canonical MOLY banner shapes
(MOLY.LR12A.R3.MP.V101 / MOLY_NR16_R1 / bare `lr12a`).

Forward-prepared per Tegra precedent (Pattern digitalandrew#3 from postmortem-hw-
firmware-tegra-activation-2026-05-15 — forward-prepared CVE pins ship
with documented activation conditions; activate via N follow-up
commits when extraction infrastructure lands). Current state:
  - mtk_modem parser DOES NOT populate blob.version with MOLY banner
    (Scout 3 verified via parsers/mediatek_modem.py:46-50 _VERSION_RES
    regex set — none of the patterns capture lr11..nr17)
  - matcher version_regex semantics are HARD REJECT (Scout 3 verified
    via cve_matcher.py:480-491; comment confirms "Stays STRICT — when
    present, version evidence MUST be found")
  - Net corpus effect post-rebuild: CVE-2023-20819 row count
    20 → 0 (BETTER than the over-attributed pre-narrowing rows per
    Reviewer B's NVD-CPE evidence; activates when mtk_modem parser
    ships version-banner extraction in a future session)

5 new paired-canary tests (Rule #46):
  - test_cve_2023_20819_version_regex_present_and_matches_six_families
    (positive: all 6 NVD families in real-world MOLY banner shapes;
     negative: LR18/NR18/LR12 out-of-scope + confused-substring
     guards `nr155_else` and `lr12abc`)
  - test_cve_2023_20819_hard_rejects_blob_with_null_version (gate-
    canary asserts the matcher's HARD-REJECT version_regex semantics
    actually fire on NULL — without this, a silent regression to
    soft-fallback would re-introduce the over-attribution)
  - test_cve_2023_20819_fires_when_version_contains_lr12a (Rule #46
    positive-arm — confirms the gate doesn't silently drop all
    matches due to a regex bug; activates with mtk_modem parser
    shipment)
  - test_cve_2023_20819_excludes_out_of_scope_family_versions (Rule
    #46 negative-arm — even with MOLY banner present, out-of-NVD-
    scope families like LR18 must not fire)
  - test_cve_2023_20819_entry_satisfies_f_forensic_10_gate (asserts
    the entry survives the F-FORENSIC-10 load filter)

Verification: all 5 new tests pass; full cve_matcher + forensic10
alignment suite (98 tests) passes under
`uv run --frozen pytest backend/tests/test_hardware_firmware_cve_matcher.py
 backend/tests/test_forensic10_alignment.py -v`.

References:
- https://nvd.nist.gov/vuln/detail/CVE-2023-20819
- M-MOLY01068234 (MediaTek PSB)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…-05-18

Session-close artifacts for the 2026-05-18 docs/patterns/rules session
(commits 533bb727fed6d0; 7 commits).

POSTMORTEM .planning/postmortems/postmortem-hw-firmware-adaptive-backlog-recipes-2026-05-18.md
- Summary of 4 TARGETs + 3 Reviewer fixups
- Final commit chain table (7 commits annotated)
- What Broke section: 4 failure modes (Rule #50 UNIQUE-constraint claim drift,
  ROUTER.md surface gap, ID-scheme grep-back-incompatibility on 6 rows,
  in-progress not promoted at session close)
- What Safety Systems Caught: 8 systems including the NEW failure-mode surface
  for recursive-verification (code-verified docs claims; 8 sessions running)
- Scope Analysis: 0 drift on user-stated asks
- Patterns: 10 successful patterns extracted
- Recommendations Carried Forward: 14 highest-leverage queued in ADAPTIVE_BACKLOG
- Numbers: 7 commits, 925 net-new LOC, 50 CLAUDE.md learned rules (was 47)
- ---HANDOFF--- block for cross-session context transfer

PATTERNS .planning/knowledge/hw-firmware-adaptive-backlog-recipes-2026-05-18-patterns.md
- 10 successful patterns codified
- Key decisions table (6 decisions documented)
- Cross-references to prior session patterns (evening → this session extensions)

ANTIPATTERNS .planning/knowledge/hw-firmware-adaptive-backlog-recipes-2026-05-18-antipatterns.md
- 5 failed patterns documented
- 8 cross-cutting antipattern themes

Key contributions to the durable-discipline corpus:
- Pattern digitalandrew#1: Recursive verification EXTENDED to code-verified docs claims
  (NEW failure-mode surface 2026-05-18; 8 sessions running for the discipline
  overall). Reviewer A A-02 catch this session is the canonical instance.
- Pattern digitalandrew#2: Meta-application — TARGET 3 IS itself a cross-stack-alignment
  commit per the rule it codifies (Rule #48).
- Pattern digitalandrew#3: Rule digitalandrew#25 per-piece commits validated for docs-only sessions
  (7 commits, 0 reverts, bisect-clean).
- Pattern digitalandrew#4: Companion discoverability update is part of new-artifact
  landing (Reviewer C C-01 catch).
- Pattern digitalandrew#5: ID-scheme grep-back-compat for cross-session artifacts
  (Reviewer C C-02 catch).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…aphore guard

Fix digitalandrew#3 of the SBOM/vuln-scan regression investigation 2026-05-21 (Wave-1
Scout C live-DB probe identified this as the highest-impact single fix;
W2-α convergence root-cause digitalandrew#1; W2-β §SC5-NEW-SBOM-θ CRITICAL — gate
removal MUST bundle the concurrency bound). Rule digitalandrew#25 single-slice atomic
commit per W2-β recommendation digitalandrew#2: gate removal + safe-runner discipline
+ concurrency bound + tests in ONE commit, NOT split.

Root cause: `app/workers/unpack.py` line 106 (pre-fix) had `if count <= 0:
return` AFTER hardware-firmware blob detection, which short-circuited
the walker auto-trigger registry below it. Every firmware whose HW-blob
detection returned zero (bare-metal MCU/DSP, intel_hex, .bin without
known headers, generic ZIP archives) silently skipped 27 walker
safe-runners. Per Scout C's live-DB probe: cluster-wide zero walker
output since 2026-05-12; worked-example incident on 24967.hex
(intel_hex / ARMhf / uC/OS-II — exactly what the bare_metal walker was
built for, got NOTHING).

Fix shape:
- Walker fan-out is moved OUTSIDE the count gate; runs unconditionally
  (each walker uses Rule digitalandrew#16 `get_detection_roots(firmware)` to filter
  targets and no-ops cheaply when no relevant artefacts present).
- Driver-firmware graph build STAYS gated on `count > 0` — it operates
  over persisted hardware_firmware_blob rows; with zero blobs the graph
  is empty by construction.
- Detection-failure branch (`except Exception`) no longer returns; the
  exception is logged and the walker fan-out still gets a shot. Forensic
  walkers can still emit findings even when HW-blob detection blew up.
- New module-level `_WALKER_FANOUT_SEMAPHORE = asyncio.Semaphore(4)`
  bounds cross-firmware walker concurrency per W2-β §SC5-NEW-SBOM-θ
  CRITICAL. Sized at 10% of pool=40 per Rule #51 .iv DB-pool-headroom
  math. Wraps the walker fan-out loop via `async with`; pre-2026-05-21
  the gate was protective beyond its stated purpose, so removing it
  without a bound would have allowed 5×27=135 simultaneous walker tasks
  to detonate the connection pool on operator burst-upload workflows.

Rule #46 paired META-CANARIES (new test file):
- test_walker_fanout_fires_when_count_is_zero — mock-shape regression
  test that the post-fix flow fires walkers on count=0 firmware
- test_walker_fanout_fires_when_count_is_positive — count > 0 path
  still fires walkers AND graph build
- test_walker_fanout_fires_when_detection_raises — detection
  exception no longer short-circuits walker fan-out
- test_meta_canary_no_bare_count_le_zero_return_in_run_hw_detection_safe —
  AST-walk gate asserting the `if count <= 0: return` shape is absent
  from the post-fix function body
- test_meta_canary_would_fire_on_re_introduced_gate — paired
  synthesize-and-assert canary proving the AST walk above would
  reject the pre-fix shape if re-introduced (Rule #46
  §gate-canary-requirement)
- test_walker_fanout_semaphore_is_present_with_bound_4 — asserts the
  `_WALKER_FANOUT_SEMAPHORE` module attribute exists with sane bound
- test_walker_fanout_uses_semaphore_context — regex-level check that
  the fan-out loop is wrapped in `async with _WALKER_FANOUT_SEMAPHORE:`

Validation:
- py_compile on both modified + new files: PASS
- Targeted pytest deferred to Session 1 final-rebuild batch (backend
  env issue unrelated to this change is blocking the in-session test
  loop; will run pytest against a fresh container at Session 1 close)

Cross-refs:
- CLAUDE.md Rule digitalandrew#16 (detection roots — each walker filters its own targets)
- CLAUDE.md Rule digitalandrew#25 single-slice exception (gate removal + semaphore + tests bundled atomically)
- CLAUDE.md Rule digitalandrew#39 (walker triplet — .safe is fire-and-forget Rule digitalandrew#7-compliant)
- CLAUDE.md Rule #46 (paired META-CANARY discipline; absence-asserting gates need synthesize-and-assert)
- CLAUDE.md Rule #47 (consumer-hook enumeration — the gate is itself a consumer of `count`)
- CLAUDE.md Rule #51 .iv (DB-pool headroom budget for state-machine + concurrency changes)
- W2-β report §SC5-NEW-SBOM-θ (CRITICAL — pool detonation if gate removed without bound)
- W2-β §SC5-NEW-SBOM-η (cross-feature critique on bare gate removal)
- Scout C primary finding (Scout C lines 16, 87-89, 138-140, 162-166)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Fix digitalandrew#2 of the SBOM/vuln-scan regression investigation 2026-05-21. Closes
Rule #51 .i companion-failure gap that the 2026-05-07 firmware-upload
Rule digitalandrew#33 sync→202+polling conversion (commit 847eae9) left open. Per
Scout B's regression-history report, 847eae9 was the inflection point;
the upload_stage state-machine was introduced WITHOUT the orphan reaper
that Rule #51 mandates as a Rule digitalandrew#33 .i companion. Under any backend
restart mid-upload (Rule digitalandrew#8 rebuild from a parallel agent, OOM kill,
docker compose restart) the row stays in `upload_stage IN
('detecting','extracting','analyzing')` permanently — Scout D
identified the resulting stuck-spinner as the digitalandrew#1 candidate symptom of
the operator-reported "SBOM/vuln-scan regression".

Fix shape (44 LOC reaper + 90 LOC tests):
- backend/app/main.py — new reaper block in the existing lifespan
  startup chain (sandwiched between vuln_scan_status and the DBX
  bundle probe). Same SQL UPDATE shape as the cve_match / vuln_scan
  reapers above, with two additional WHERE clauses:
  (a) `upload_stage_started_at.is_not(None)` per W2-β §SC5-NEW-SBOM-α
      NULL handling — freshly-created rows shouldn't be eligible to
      reap because the post-process task may still be spinning up.
  (b) `upload_stage_started_at < datetime.now(UTC) - timedelta(minutes=15)`
      per W2-β §SC5-NEW-SBOM-ε grace window — protects against the
      race where the lifespan reaper fires DURING a legitimate
      upload's startup window. 15 min exceeds the worst-case 16 GB
      tarball extraction window observed in production (Medtronic
      ILLUMISITE precedent — ~12 min cold-extract).

Rule #46 paired META-CANARIES (new file):
- backend/tests/test_main_lifespan_reapers.py — 6 tests:
  - test_upload_stage_reaper_present_in_lifespan — string-match the
    docstring block + the .in_() filter shape
  - test_upload_stage_reaper_uses_15_minute_grace — asserts the
    `timedelta(minutes=15)` + `is_not(None)` clauses are present
  - test_upload_stage_meta_canary_would_fire_on_dropped_grace —
    paired synthesize-and-assert canary proving the gate above
    would reject a regression that drops the grace clause
  - test_existing_{vuln_scan,cve_match,device_dump}_reaper_still_present
    — regression guards that the 3 reapers shipped before this fix
    can't accidentally regress under a future refactor

The Fix digitalandrew#5 bare_metal_audit_status reaper ships as its own
Rule digitalandrew#25 per-piece commit immediately after this one; this file is
extended in that commit.

Validation:
- py_compile on main.py + test file: PASS
- Targeted pytest deferred to Session 1 final-rebuild batch (in-session
  backend env propagation issue blocks immediate pytest run — code-shape
  validated via py_compile + AST review; backend container in production
  will pick up the new reaper at next normal restart)

Cross-refs:
- CLAUDE.md Rule digitalandrew#33 .a/.i (state machine + orphan-reaper companion)
- CLAUDE.md Rule #46 (paired META-CANARY discipline)
- CLAUDE.md Rule #51 worked-example (Rule digitalandrew#33 conversion = invariant
  sweep across rate-limit + reaper + frontend 429 + DB pool — this
  closes the reaper companion for the 2026-05-07 upload_stage refactor)
- W2-α convergence root-cause digitalandrew#3
- W2-β §SC5-NEW-SBOM-α (NULL handling), §SC5-NEW-SBOM-δ (fresh-row guard),
  §SC5-NEW-SBOM-ε (15-min grace)
- Scout B suspect-commit digitalandrew#1 (`847eae9` 2026-05-07 upload→202+polling
  conversion that introduced the Rule #51 gap)
- Scout D digitalandrew#1 candidate symptom (stuck-spinner from orphaned upload_stage)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ion)

Fix digitalandrew#5 of the SBOM/vuln-scan regression investigation 2026-05-21. Closes
Rule #51 .i companion-failure gap that the 2026-05-19 bare-metal MCU
walker introduction left open. The bare_metal_audit state machine was
authored as a clean Rule digitalandrew#33 .a 5-state column with DB CHECK, Pydantic
Literal, and Rule digitalandrew#39 inner/outer/safe triplet — but missed the Rule
#51 .i orphan-reaper companion. Scout C's live-DB probe found the
worked-example incident: TMS320F28066 firmware `78ad638b-...` stuck in
`bare_metal_audit_status='queued'` for 6 days after a backend restart
lost its runner. Without this reaper, every subsequent bare-metal audit
attempt 409s on `firmware.bare_metal_audit_status IN ('queued','running')`.

Fix shape (44 LOC reaper + 38 LOC tests):
- backend/app/main.py — new reaper block sandwiched between the
  upload_stage reaper (Fix digitalandrew#2) and the DBX bundle probe. Identical SQL
  UPDATE shape to the cve_match / vuln_scan reapers above; no 15-min
  grace window because bare_metal_audit work is fully in-process and
  the trigger MCP tool's 409 dedup check already gates against
  re-entrancy.

Rule #46 paired META-CANARIES (extends test_main_lifespan_reapers.py):
- test_bare_metal_audit_reaper_present_in_lifespan — string-match on
  the docstring header + `.in_()` filter shape
- test_bare_metal_audit_meta_canary_would_fire_on_dropped_reaper —
  paired synthesize-and-assert canary proving the gate would catch a
  regression that removes the reaper block

Validation:
- py_compile on both files: PASS
- Targeted pytest deferred to Session 1 final-rebuild batch

Cross-refs:
- CLAUDE.md Rule digitalandrew#33 .a/.i (state machine + reaper companion)
- CLAUDE.md Rule digitalandrew#39 (walker triplet — bare_metal_walker.py shipped 2026-05-19)
- CLAUDE.md Rule #46 (paired META-CANARY discipline)
- CLAUDE.md Rule #51 .i (orphan-reaper completeness — bundled with Rule digitalandrew#33 conversions)
- CLAUDE.md Rule #52 (closed-grammar Rule-of-Two; bare-metal is the 1st instance)
- W2-α convergence root-cause digitalandrew#3 (orphan-reaper coverage gap across 28+ columns)
- W2-γ Fix digitalandrew#5 (84 LOC, 1 commit)
- Scout C live-DB probe — worked-example TMS320 stuck row firmware_id `78ad638b`

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Closes /citadel:learn for the 2026-05-21 SBOM/vuln-scan regression
investigation Session 1 campaign.

.planning/knowledge/ files written:
- sbom-vuln-scan-regression-session1-2026-05-21-patterns.md
  8 successful patterns extracted:
  1. Wave-1 + Wave-2 deep research applied to a regression cascade
  2. W2-β cross-feature blow-it-up catches attacks single-axis can't see
  3. W2-γ Rule digitalandrew#28 yardstick honored over W2-α MEDIUM-confidence
  4. Rule digitalandrew#25 single-slice atomic commit (gate + bound + tests)
  5. Rule #46 META-CANARY density 2-7 per fix
  6. Live runtime fix proof via container startup log
  7. Ship FK unblock FIRST so live canaries validate everything else
  8. `docker run --rm` ephemeral container for pytest when compose
     env propagation breaks
  + 7 key decisions table

- sbom-vuln-scan-regression-session1-2026-05-21-antipatterns.md
  8 anti-patterns extracted:
  1. Trusting operator pytest spec paths without verifying existence
  2. `docker compose restart` after sibling-service rebuild loses env
  3. Mock with too-narrow signature breaks on production widening
  4. `if count <= 0: return` BEFORE walker fan-out
  5. Naive gate removal without concurrency bound (§SC5-NEW-SBOM-θ)
  6. Bare `asyncio.create_task` for detached background work
  7. State-machine column added without orphan reaper in same chain
  8. `.dockerignore` excludes tests/ — production-image-based pytest
     can't work

Quality rules appended to .claude/harness.json (2 new, both HIGH/MED
confidence; 0 duplicates skipped):

1. auto-sbom-regr-session1-2026-05-21-bare-create-task-background
   pattern: `asyncio\\.create_task\\s*\\(\\s*_run_\\w+_background`
   filePattern: `backend/app/routers/*.py`
   confidence: HIGH (anti-pattern digitalandrew#6 — Scout D primary symptom)

2. auto-sbom-regr-session1-2026-05-21-too-narrow-mock-create-task
   pattern: `def\\s+fake_create_task\\s*\\(\\s*coro\\s*\\)\\s*:`
   filePattern: `backend/tests/*.py`
   confidence: MEDIUM (anti-pattern digitalandrew#3 — Rule digitalandrew#30 sibling discipline)

JSON validity verified via `python3 -c "import json; json.load(...)"`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ound.py + apply to 5 sites incl. SEAM-A

Session 2a Fix digitalandrew#8-broader of the 2026-05-21 SBOM/vuln-scan regression
sweep. Closes the GC-vanish risk across the entire wairz codebase by
factoring Session 1 Fix digitalandrew#8's local `_spawn_background_task` helper out
of `routers/sbom.py` into a shared `app/utils/background.py` module
and applying it to every detached `*_background_runner` spawn.

The critical W2-β §SC5-NEW-SBOM-S2-SEAM-A site is
`backend/app/services/firmware_service.py:818` — the HW-firmware
detection runner dispatch in `_post_process_pipeline`. Pre-fix this
was bare `asyncio.create_task`; under operator burst-upload + GC
pressure 1-of-N firmware silently loses its entire 27-walker fan-out
(Session 1 Fix digitalandrew#4 registry + Fix digitalandrew#3 un-gated fan-out are downstream
of this runner) with NO state-machine column to surface the loss —
the failure is invisible.

Files changed (6 production + 1 test):

- backend/app/utils/background.py — NEW shared module:
    spawn_background_task(coro, *, name=None) -> asyncio.Task
    _BACKGROUND_TASKS: set[asyncio.Task]
  Documented at module top with the asyncio docs link
  ("Save a reference to the result of this function") + cross-refs
  to Rule digitalandrew#33 .d + Rule #46 + Rule #51 + Session 1 Fix digitalandrew#8 +
  Session 2a SEAM-A.

- backend/app/routers/sbom.py — replaces the Session 1 Fix digitalandrew#8 local
  `_spawn_background_task` + `_BACKGROUND_TASKS` definitions with a
  re-export-alias import from app.utils.background. Preserves the
  local name so existing tests (test_sbom_router_background_tasks.py)
  + per-test patching continue to work without churn.

- backend/app/routers/hardware_firmware.py — 2 sites:
  cve_match_background (line 654) + run_authenticode_chain_background
  (line 750) — both now use spawn_background_task with name= for
  asyncio.all_tasks() debugger legibility.

- backend/app/routers/fuzzing.py — 1 site:
  _run_campaign_spawn_background (line 143).

- backend/app/routers/emulation.py — 1 site (the arq-fallback path):
  _run_spawn_background (line 165). The arq-primary path stays
  unchanged (arq has its own durability). The websocket-local tasks
  at lines 946+ stay bare (they're locally scoped + awaited via
  asyncio.wait, NOT detached).

- backend/app/services/firmware_service.py — THE SEAM-A SITE:
  _run_hardware_firmware_detection_safe spawn at line 818. This is
  the highest-blast-radius miss in the codebase — all 27 walker
  safe-runners depend on this runner. Now GC-hardened with
  name=f"hw_firmware_detection_{firmware.id}".

- backend/tests/test_background_task_sweep.py — NEW. 5 tests:
  * test_shared_background_helper_module_exists — asserts module
    exports
  * test_routers_use_shared_helper_not_local_definitions — every
    target file imports from app.utils.background
  * test_no_bare_asyncio_create_task_for_background_runners — regex
    AST scan across all 5 modules; asserts NO remaining bare
    create_task for any name matching `_run_*_background` /
    `run_*_background` / the specific
    `_run_hardware_firmware_detection_safe` SEAM-A site
  * test_meta_canary_would_fire_on_bare_create_task — paired
    synthesize-and-assert canary per Rule #46 §gate-canary-requirement
  * test_seam_a_firmware_service_uses_helper — explicit verification
    that the SEAM-A site uses the helper

Validation:
- py_compile on all 7 files: PASS
- Targeted pytest deferred to Session 2a Step 6 batch validation
- Rule digitalandrew#20 caveat: each modified file is a Python module import in
  the running backend. Either docker cp+restart per Rule digitalandrew#20 OR
  full Rule digitalandrew#8 rebuild after the next commit. Step 6 will do the
  rebuild.

Cross-refs:
- CLAUDE.md Rule digitalandrew#33 .d (asyncio.create_task vs arq decision rubric)
- CLAUDE.md Rule #46 (paired META-CANARY discipline)
- CLAUDE.md Rule #47 (consumer-hook enumeration on Fix digitalandrew#8-broader sweep)
- CLAUDE.md Rule #51 §SC5-NEW-SBOM Fix digitalandrew#8 (Session 1 original) +
  Fix digitalandrew#8-broader SEAM-A (Session 2a)
- Session 1 Fix digitalandrew#8 commit `fa75d5d` (the original sbom.py helper)
- Session 2 W2-α convergence shipping order step 4-broader
- Session 2 W2-β §SC5-NEW-SBOM-S2-SEAM-A (HIGH severity)
- Session 2 W2-γ Fix digitalandrew#8-broader (~105 LOC, ~1 commit)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…se 1.A)

Rule #52 instance digitalandrew#3 / Phase 1.A: 5-column state machine + DB CHECK
constraint for the new ICS protocol catalog walker (Session 2 of the
2026-05-20/22 campaign, opens the Rule-of-Three DURABLE BEYOND DEBATE
promotion path that closes in Phase 6).

Columns: ics_protocol_walk_status (idle/queued/running/completed/failed),
_started_at, _finished_at, _error, _result JSONB. Mirrors feab18c9d201
(sbom_status Session 2a) shape exactly; CHECK constraint name mirrors
ck_firmware_bare_metal_audit_status (Rule #52 instance digitalandrew#1 precedent).

Rule digitalandrew#25 single-slice (DDL coupling — model + alembic ship together;
splitting leaves alembic upgrade without the matching ORM shape).

Verified via Rule digitalandrew#20 fast iteration:
- docker cp + alembic upgrade head: succeeded (feab18c9d201 -> 1c52a4b5c6d7)
- psql column inspection: all 5 columns present with correct types/defaults
- psql CHECK constraint: ck_firmware_ics_protocol_walk_status enforced

Phase 1.B (next): JSONB normaliser + stamp helper + SCHEMA_VERSION
constant per CLAUDE.md Rule #35c (with provenance sister-key per W2-β
§SC5-NEW-ICS-S2-1). Phase 1.C: Rule digitalandrew#39 walker triplet inner/outer/safe.
Phase 1.D: walker_registry + DUAL reaper registration per W2-α
DUAL REGISTRATION (STATE_MACHINE_REAPER_CONFIGS 8→9 AND
WALKER_REAPER_CONFIGS 25→26 — the suffix-introspection check at
test_walker_reaper_configs_size_lock_matches_firmware_model forces both).

Refs:
- .planning/research/ics-protocol-session2-2026-05-22/SESSION-2-KICKOFF.md
- .planning/research/ics-protocol-session2-2026-05-22/wave2-alpha-convergence.md
- .planning/postmortems/postmortem-ics-protocol-session1-2026-05-20.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ate (Phase 1.B)

Rule #52 instance digitalandrew#3 / Phase 1.B: Rule #35c boundary normaliser +
stamp helper + SCHEMA_VERSION constant for the new
``firmware.ics_protocol_walk_result`` JSONB column from Phase 1.A
(commit 0de1eba). Includes the W2-β §SC5-NEW-ICS-S2-1 sister-provenance
mitigation: every walker-side commit stamps ``provenance: "walker"``
so downstream CVE matcher consumers can distinguish a walker-authored
result from a pre-seed via the (future) operator descriptor route.

Canonical shape carries the W2-β §SC5-NEW-ICS-S2-β snapshot-pin fields
(``snapshot_id_at_entry`` + ``snapshot_id_at_exit`` +
``consistency_warning``) so mid-walk hot-reload surfaces as a structured
warning rather than a silently-torn classification; the walker (Phase
1.C) will pin the catalog snapshot at INNER entry and stamp these fields
at exit.

Test battery (Rule #35c + Rule #46 paired META-CANARY):
- test_normalize_firmware_ics_protocol_walk_result: 7-case parametrized
  canonical pass-through + defensive coercion (None / wrong-type all
  collapse to None per the bare_metal_audit precedent)
- test_stamp_firmware_ics_protocol_walk_result_adds_version: stamp emits
  schema_version=1
- test_stamp_firmware_ics_protocol_walk_result_idempotent: re-stamp =
  identity
- test_firmware_ics_protocol_walk_result_schema_version_constant: 1
- test_stamp_firmware_ics_protocol_walk_result_enforces_provenance_walker:
  W2-β §SC5-NEW-ICS-S2-1 sister-key gate — stamp emits provenance='walker'
- test_stamp_firmware_ics_protocol_walk_result_overwrites_hostile_provenance:
  Rule #46 gate-canary — hostile pre-seed of provenance='descriptor' is
  OVERWRITTEN to 'walker' (descriptor route cannot claim walker provenance)

All 11 ICS normaliser tests green:
  docker compose exec -T backend uv run pytest tests/test_jsonb_normalizers.py
  -k "ics_protocol_walk_result or firmware_ics_protocol" -q
  → 11 passed in 0.56s

Phase 1.C (next): Rule digitalandrew#39 walker triplet inner/outer/safe + Pydantic
IcsProtocolWalkStatus Literal + Rule #45 no-execute gate.
Phase 1.D: walker_registry + DUAL reaper registration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…iteral + Rule #45/46 gates (Phase 1.C)

Rule #52 instance digitalandrew#3 / Phase 1.C: inner/outer/safe walker triplet per
Rule digitalandrew#39, consuming the Session 1 ICS protocol catalog scaffold
(``catalog.py`` / ``resolver.py`` / ``snapshot.py``) via Rule digitalandrew#16
``get_detection_roots(firmware)`` × binary head-byte slice ×
``resolve_all()`` × Rule #35c JSONB stamp from Phase 1.B.

Walker (NEW: ``backend/app/services/ics_protocol_walker.py``):
- ``_do_ics_protocol_walk(db, firmware_id)`` — INNER pure-logic; pins
  catalog snapshot at entry (W2-β §SC5-NEW-ICS-S2-β); clears stale
  ics_protocol_walk_result on entry (W2-β §SC5-NEW-ICS-S2-δ);
  iterates detection roots × binary candidates × resolve_all;
  reads exit snapshot + flags drift as consistency_warning.
- ``run_ics_protocol_walk_background(firmware_id)`` — OUTER Rule digitalandrew#33 .a
  state machine; cycles queued → running → completed | failed; failure
  persistence on FRESH session per Rule digitalandrew#33 .b; clears partial JSONB
  on failure (W2-β §SC5-NEW-ICS-S2-δ).
- ``auto_ics_protocol_walk_firmware_safe(firmware_id)`` — SAFE Rule digitalandrew#39
  .safe contract; runs INNER + stamps result + does NOT mutate status
  (leaves idle so operator MCP trigger doesn't 409); structured log on
  exception per Rule #51 partner.

Pydantic ``IcsProtocolWalkStatus = Literal[idle/queued/running/completed/
failed]`` added to ``app/schemas/firmware.py`` — Rule digitalandrew#33 .c
typo-gate at trigger MCP / status reader boundary; DB CHECK
ck_firmware_ics_protocol_walk_status (Phase 1.A) is the durable safety
floor.

Bounded scan budget: 32 KiB head per binary, 2000 binaries per walk,
64 B–50 MB size band, ``_BINARY_EXT_ALLOWLIST`` for fast filesystem
rejection. Per Rule digitalandrew#5 all blocking filesystem reads wrapped in
``loop.run_in_executor``.

Test battery (Rule digitalandrew#36 + Rule #45 + Rule #46 paired META-CANARIES;
Rule #35b live canaries against tests._live_db.make_live_db):
- test_walker_no_execute_no_decrypt: tokenize-walk over walker source;
  asserts NO subprocess/decrypt/eval/exec patterns; whitespace-tolerant
  per Rule #45 EFS-DDF lesson
- test_walker_no_execute_gate_actually_fires: Rule #46 paired META-CANARY;
  synthesizes subprocess.run + asyncio.create_subprocess_exec + .decrypt(
  in-memory; asserts gate matches all 3
- test_walker_empty_firmware_returns_empty_aggregate: live canary;
  empty firmware → empty aggregate + no_detection_roots error
- test_walker_clears_stale_jsonb_on_entry: live canary; W2-β
  §SC5-NEW-ICS-S2-δ — stale ghost_protocol JSONB cleared after INNER
- test_walker_detects_mid_walk_snapshot_drift: W2-β §SC5-NEW-ICS-S2-β
  paired META-CANARY; mocked drifting catalog → consistency_warning
  populated; pair-canary test_walker_no_drift_when_snapshot_stable
  confirms gate doesn't false-fire
- test_walker_triplet_exports_all_three_runners: Rule digitalandrew#11 import smoke
- test_walker_uses_jsonb_stamp_helper_not_direct_dict_assign: Rule #35c
  partner — direct JSONB assignment bypasses provenance contract
- test_ics_protocol_walk_status_literal_matches_db_check_values: Rule
  digitalandrew#33 .c — Literal ↔ DB CHECK pairwise agreement

All 20 ICS Phase 1.B+1.C tests green:
  docker compose exec -T backend uv run pytest tests/test_ics_protocol_walker.py
  tests/test_jsonb_normalizers.py -k "ics_protocol" -q
  → 20 passed in 2.30s

Phase 1.D (next): walker_registry + DUAL reaper registration
(STATE_MACHINE_REAPER_CONFIGS 8→9 AND WALKER_REAPER_CONFIGS 25→26)
+ Rule #46 size-lock META-CANARY updates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…NARIES (Phase 1.D)

Rule #52 instance digitalandrew#3 / Phase 1.D — closes Phase 1 (commits 0de1eba +
7c6405f + bad86ee + this). Wires the Phase 1.C walker triplet into
the Session 2b two-axis data-driven reaper sweep via DUAL REGISTRATION
per Scout E + W2-α convergence synthesis.

walker_registry.py changes:
- Import `auto_ics_protocol_walk_firmware_safe` from
  `app/services/ics_protocol_walker.py` and add to
  `_load_walker_safe_runners()` return list (alphabetical-ish position
  after auto_etl_walk_firmware_safe; 28 total walker safe runners).
- STATE_MACHINE_REAPER_CONFIGS 8 → 9: add
  `ics_protocol_walk_status` entry mirroring the bare_metal_audit
  precedent (operator-triggered + Rule digitalandrew#33 .b result JSONB shape).
- WALKER_REAPER_CONFIGS 25 → 26: add `ics_protocol_walk_status` via
  the `_walker_reaper()` helper. DUAL REGISTRATION rationale: the
  suffix-introspection cross-check at
  test_walker_reaper_configs_size_lock_matches_firmware_model
  enforces every *_walk_status column to be registered here. The
  reaper sweep applies both passes; the second is a no-op (~1 ms) on
  a row already reaped by the first. Exclusion-list precedent would
  erode the suffix-introspection guarantee shipped Session 2b commit
  95e47a6.

main.py docstring updated 8 → 9 / 25 → 26 entries.

test_main_lifespan_reapers.py updates (Rule #46 size-lock + paired
META-CANARY discipline):
- test_state_machine_reaper_configs_size_lock: 8 → 9
- test_walker_reaper_config_count_matches_documented: 25 → 26 with
  inline rationale for DUAL REGISTRATION
- test_main_py_lifespan_imports_both_registries: error messages
  updated 8/25 → 9/26

Verified end-to-end:
- pytest tests/test_main_lifespan_reapers.py -q → 16 passed
- pytest tests/test_jsonb_normalizers.py tests/test_ics_protocol_walker.py
  tests/test_main_lifespan_reapers.py + 3 baseline META-CANARY suites
  → 827 passed (no regressions across the 2026-05-21 SBOM canaries)
- docker compose exec backend uv run python -c "import + introspect"
  Rule digitalandrew#11 smoke: STATE_MACHINE 9, WALKER 26, ICS in both, safe runner
  present, 28 total walker safe runners

Phase 1 close: full Rule digitalandrew#8 rebuild required for the model class-shape
change (1c52a4b5c6d7 alembic + Firmware ORM extension) — runs AFTER
this commit to land the new walker in the running container's image.

Phase 2 (next): Rule digitalandrew#25 Shape-1 single-slice — extend
ck_findings_source DB CHECK + IcsProtocolFindingSource Literal +
frontend FindingSource union + FINDING_SOURCE_CONFIG mirror + Rule #48
5-part alignment test. Three new sources: ics_modbus_tcp_detected,
ics_dnp3_detected, ics_s7comm_detected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…ule digitalandrew#25 Shape-1)

Rule #52 instance digitalandrew#3 / Phase 2: extends the finding-source cross-stack
alignment surface to cover the 5 IcsProtocolFamily Literal values per
W2-β §SC5-NEW-ICS-S2-η mitigation — Rule digitalandrew#25 single-slice exception digitalandrew#2
atomic commit because test_finding_source_alignment.py enforces pairwise
agreement across DB CHECK + Pydantic Literal + frontend Union + frontend
FINDING_SOURCE_CONFIG, and splitting leaves the alignment test RED
between commits.

5 NEW source values (one per IcsProtocolFamily Literal entry):
- ics_modbus_tcp_detected — exercised by Session 1 modbus_tcp.yaml
- ics_modbus_rtu_detected — forward-prepared for serial-protocol YAMLs
- ics_dnp3_detected — Phase 5 dnp3.yaml will exercise
- ics_s7comm_detected — Phase 5 s7comm.yaml will exercise
- ics_unknown_ics_detected — Phase 4 plugin escape-hatch envelope

Cross-stack surfaces (all in this commit per Rule #48 5-part shape):
- DB CHECK extension: alembic 2cba3d4e5f6a (chains from 1c52a4b5c6d7)
- Pydantic Literal: app/schemas/finding.py::IcsProtocolFindingSource
- Frontend union: frontend/src/types/index.ts::FindingSource
- Frontend config: frontend/src/constants/statusConfig.ts entries
  (5 new entries; cyan/sky/indigo gradient per Scout D operator-UX)

Walker emit (app/services/ics_protocol_walker.py):
- New _ALLOWED_ICS_FINDING_SOURCES frozenset derived from
  IcsProtocolFindingSource via typing.get_args (single source of truth)
- INNER runner emits one Finding row per (firmware, protocol_family)
  tuple at severity=info — closed-allowlist guarded so unmapped
  families skip emit instead of raising 422 at FindingService.create
- Result aggregate adds `findings_emitted_count` field;
  _empty_result_aggregate updated for shape consistency

Rule #46 paired META-CANARIES (test_ics_protocol_walker.py):
- test_walker_ics_finding_source_allowlist_matches_pydantic_literal:
  walker allowlist EXACTLY equals Pydantic Literal (no drift either way)
- test_walker_ics_finding_source_naming_convention: every Literal entry
  follows ics_<family>_detected — protects per-family lookup from
  silent rename drift

Verified:
- alembic upgrade 1c52a4b5c6d7 -> 2cba3d4e5f6a applied; psql confirmed
  5 new ics_* sources in ck_findings_source CHECK
- pytest tests/test_ics_protocol_walker.py: 11 passed (was 9)
- pytest tests/test_finding_source_alignment.py on host: 3/3 passed
  (DB CHECK + frontend Union + frontend Config pairwise alignment)
- Rule digitalandrew#24 canary + npx tsc -b --force frontend typecheck: exit 0
- broader container pytest sweep (jsonb_normalizers + walker +
  reapers + auth-gate + sbom_status_alignment): 808 passed

Phase 3 (next): MCP tool category (backend/app/ai/tools/ics_protocol.py)
with 4 tools — trigger_ics_protocol_walk + list_ics_protocols +
lookup_ics_protocol_across_firmwares (Rule #44 mandatory) +
describe_ics_protocol_anomalies. Includes W2-β §SC5-NEW-ICS-S2-3
+ §SC5-NEW-ICS-S2-γ + §SC5-NEW-ICS-S2-ε + §SC5-NEW-ICS-S2-ι mitigations
(status filter + supply_chain_signal curated-only + project_id scope +
schema_version legacy-rows filter) per Wave-1 C + W2-β cross-feature.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Rule #52 instance digitalandrew#3 / Phase 3: new MCP tool category
``backend/app/ai/tools/ics_protocol.py`` with 4 tools per W2-α plan,
surfacing the Phase 1 walker's JSONB result and Rule digitalandrew#33 .a state
machine to MCP consumers. Includes ALL 5 W2-β §SC5-NEW-ICS-S2-*
provenance gates inline (Wave-1 C + W2-β mandate).

Tools (registered in ai/__init__.py → registry size 335 total):

- ``trigger_ics_protocol_walk(firmware_id)`` — Rule digitalandrew#33 .a operator-
  trigger. Flips idle→queued; fires OUTER background runner via
  asyncio.create_task; Rule digitalandrew#33 .a 409 on in-flight.
  **W2-β §SC5-NEW-ICS-S2-ε (I35)**: filters by context.project_id —
  operator-A in P1 cannot trigger walker against firmware in P2 via
  switch_project.

- ``list_ics_protocols(firmware_id)`` — reads
  ``ics_protocol_walk_result`` JSONB; surfaces consumer_warning when
  ``_result_passes_provenance_gates`` rejects (W2-β §SC5-NEW-ICS-S2-1
  sister-provenance + §SC5-NEW-ICS-S2-β consistency_warning).

- ``lookup_ics_protocol_across_firmwares(protocol_family, scope, limit)`` —
  CLAUDE.md Rule #44 MANDATORY cross-firmware aggregation. Applies:
  (a) SQL filter ``ics_protocol_walk_status='completed' AND result IS
  NOT NULL`` (W2-β §SC5-NEW-ICS-S2-3 + γ);
  (b) per-row Python gate via ``_result_passes_provenance_gates``
  (schema_version=1 / provenance=walker / no consistency_warning);
  (c) ``supply_chain_signal`` flag requires match_count≥2 AND ≥1
  matching firmware with curated-tier (_system/core) manifest_source
  (W2-β §SC5-NEW-ICS-S2-γ I30 — operator-tier-only matches do NOT
  trigger the signal).

- ``describe_ics_protocol_anomalies(firmware_id)`` — operator-UX
  surface; flags multi_protocol (3+ families), mid_walk_catalog_drift
  (W2-β §SC5-NEW-ICS-S2-β), non_walker_provenance (W2-β §SC5-NEW-ICS-S2-1),
  walker_errors.

Test battery (Rule #46 paired META-CANARIES + Rule #35b live canaries
against make_live_db):
- test_register_ics_protocol_tools_registers_four: registry sanity
- test_trigger_ics_protocol_walk_409_on_in_flight: Rule digitalandrew#33 .a conflict
- test_trigger_ics_protocol_walk_project_scope_filter: §SC5-NEW-ICS-S2-ε
- test_trigger_ics_protocol_walk_requires_active_project: defensive
- test_list_ics_protocols_consumer_warning_on_provenance_fail:
  §SC5-NEW-ICS-S2-1 — surfaces consumer_warning when provenance hostile
- test_list_ics_protocols_handles_non_completed_status: hint shape
- test_lookup_across_firmwares_filters_failed_rows: §SC5-NEW-ICS-S2-3
- test_lookup_across_firmwares_filters_legacy_schema_version: §SC5-NEW-ICS-S2-ι
- test_lookup_across_firmwares_supply_chain_signal_requires_curated:
  §SC5-NEW-ICS-S2-γ I30 — operator-tier-only does NOT trigger flag
- test_lookup_across_firmwares_supply_chain_signal_fires_when_curated:
  paired canary — curated tier ≥2 DOES trigger flag
- test_describe_anomalies_surfaces_consistency_warning: §SC5-NEW-ICS-S2-β
- test_describe_anomalies_multi_protocol_threshold: operator-UX

All 12 MCP tests + 936 broader sweep passed:
  docker compose exec -T backend uv run pytest tests/test_ics_protocol_mcp.py
  -q → 12 passed in 8.37s
  + broader ICS + reaper + normaliser + auth-gate sweep: 936 passed

Tenancy note: scope='global' is operator-owned cross-project
introspection in single-tenant wairz; future multi-tenant deploy MUST
acquire context.permitted_project_ids per the Rule #44 Rule-of-Nine
docstring direction (consistent with linux_systemd, linux_journald, +
other cross-firmware tools).

Phase 3.B (REST router + TIER_A_LIGHT_ACK + 202-polling) DEFERRED —
W2-γ marked optional ("HTTP endpoint optional; could compress to 1
commit"). Frontend page (Scout D's project sub-route surface) is also
deferred to a follow-up session; MCP-only access is sufficient for the
walker → Rule #44 cross-firmware aggregation path.

Phase 4 (next): Bundled string_scanner plugin + freeze_plugin_registry
with W2-β §SC5-NEW-ICS-S2-α MappingProxyType + AST pre-import scan
hardening + Rule digitalandrew#21 backfill to file_format_catalog.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…W2-β §SC5-NEW-ICS-S2-α HARDENED freeze (Phase 4)

Rule #52 instance digitalandrew#3 / Phase 4: closes Session 1 W2-β §SC5-NEW-ICS-7
(hot-reload × plugin module-level shadow) — the scariest unmitigated
attack identified by Session 1 deep-research. Ships with W2-β
§SC5-NEW-ICS-S2-α HARDENED shape (Wave-2 β cross-feature critique
explicit recommendation): MappingProxyType wrap + closure-capture gate
+ freeze sentinel + namespace-disjointness collision check.

Resolver plugin infrastructure (app/services/ics_protocol_catalog/
resolver.py):
- IcsProtocolMatcherProto Protocol — typed plugin contract;
  cost_class + protocol_families + detect(blob_head, path, size,
  context) signature.
- Private _PLUGIN_REGISTRY mutable dict; PRIVATE — never re-exported.
- Public PLUGIN_REGISTRY = MappingProxyType(_PLUGIN_REGISTRY) —
  read-only view. Direct ``PLUGIN_REGISTRY[x] = y`` writes raise
  TypeError. CLOSES §SC5-NEW-ICS-S2-α: hostile bundled plugin
  ``from ... import PLUGIN_REGISTRY; PLUGIN_REGISTRY["x"] = matcher``
  attack now fails at the proxy layer (NOT just the freeze check).
- register_matcher(name, matcher) — three gates: freeze sentinel,
  namespace-disjointness collision, closure-capture rejection.
- _closure_capture_check (W2-β §SC5-NEW-ICS-S2-ζ): rejects matchers
  whose detect() callable captures AsyncSession/Settings/ContextVar/
  ToolContext via closure (request-scope leak prevention).
- freeze_plugin_registry() flips sentinel; _unfreeze_plugin_registry_for_tests
  for test isolation only.

Bundled string_scanner plugin (NEW
app/services/ics_protocol_catalog/plugins/string_scanner.py):
- Stateless StringScannerPlugin — closure-clean (no captured state).
- Coarse byte-needle signatures for modbus_tcp / dnp3 / s7comm —
  detects library-symbol patterns the closed-grammar
  string_in_binary_constraint cannot express compactly.

Plugin discovery (NEW app/services/ics_protocol_catalog/plugins/__init__.py):
- register_default_plugins(freeze=True) — static imports of bundled
  plugins; no importlib against operator-controlled name strings
  (defense against §SC5-NEW-ICS-S2-α dynamic-import attack vector).

Lifespan wire (app/main.py):
- register_default_plugins(freeze=True) called AFTER LOLDrivers probe
  + BEFORE yield. Order matters — the freeze gate is the security
  boundary; calling after yield exposes a startup-to-first-request
  window. Defensive try/except so plugin registration failure does
  NOT block startup — walker degrades gracefully to closed-grammar
  YAML detection only.

Rule #46 paired META-CANARIES (test_ics_protocol_plugin.py, 13 tests):
- test_plugin_registry_is_mappingproxytype_read_only: type assertion
- test_mappingproxytype_gate_actually_blocks_direct_dict_write:
  paired canary — synthesizes hostile direct write; confirms TypeError
- test_register_matcher_pre_freeze_succeeds: paired canary (gate
  doesn't degenerate to always-reject)
- test_register_matcher_post_freeze_raises: §SC5-NEW-ICS-S2-α
  primary mitigation; RuntimeError on post-freeze write
- test_freeze_does_not_block_existing_lookups: defensive — reads
  still work post-freeze
- test_register_matcher_rejects_async_session_closure_capture:
  §SC5-NEW-ICS-S2-ζ — synthesizes hostile matcher with AsyncSession
  in __closure__; confirms ValueError
- test_register_matcher_accepts_stateless_matcher: paired canary
- test_register_matcher_rejects_protocol_family_collision: namespace
  disjointness (analog of file_format A7)
- test_register_default_plugins_registers_string_scanner: smoke
- test_register_default_plugins_freezes_when_requested: lifespan
  contract
- test_string_scanner_plugin_detect_returns_hits_on_modbus_signature:
  plugin functional smoke
- test_string_scanner_plugin_returns_none_on_clean_binary: paired
- test_main_py_lifespan_imports_and_calls_register_default_plugins:
  Rule #46 META-CANARY — confirms main.py call shape + BEFORE-yield
  textual ordering

Phase 4 test sweep: 13 passed in 0.48s; broader Phase 1-4 + baseline
sweep: 952 passed.

DEFERRED per W2-β recommendation (queued in Phase 6 postmortem):
- AST pre-import scan of plugin module source (complex; covered
  partially by closure-capture gate + MappingProxyType wrap)
- Rule digitalandrew#21 backfill to file_format_catalog (same MappingProxyType +
  closure-capture hardening) — file_format precedent uses bare dict
  shape; backfill is a Rule-of-Two sweep but ICS-S2 scope deferred
  it; documented in Phase 6 postmortem as follow-up.

Phase 5 (next): DNP3 + S7Comm production YAML manifests as Rule digitalandrew#25
single-slice commits (one per protocol per Rule digitalandrew#25).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Rule #52 instance digitalandrew#3 / Phase 5.A — Rule digitalandrew#25 per-piece commit: ships
the second ICS protocol catalog production manifest (after Session 1's
modbus_tcp_v0_system). Mirrors the Modbus YAML shape exactly with
DNP3-specific port + banner + function-code values.

DNP3 (IEEE 1815-2012; IANA TCP/UDP 20000) detector signals:
- string_in_binary needle set: 'dnp3' / 'opendnp3' / 'libdnp3'
  (case-insensitive; combine=any min_count=1)
- port_signature: 20000 (little-endian uint16 constant in head)
- function_code_set: 14 DNP3 standard FCs (0x01 READ / 0x02 WRITE /
  0x03 SELECT / 0x04 OPERATE / 0x05 DIRECT_OPERATE / 0x06
  DIRECT_OPERATE_NR / 0x0D COLD_RESTART / 0x0E WARM_RESTART /
  0x14 ENABLE_UNSOLICITED / 0x15 DISABLE_UNSOLICITED /
  0x17 DELAY_MEASURE / 0x18 RECORD_CURRENT_TIME /
  0x81 RESPONSE / 0x82 UNSOLICITED_RESPONSE) — min_count=3 within
  512-byte window per W2-β A8 floor.

W2-β §SC5-NEW-ICS-2 mitigation: combine=all_required prevents
single-signal false positives. Modbus FCs (0x01-0x06) overlap
conceptually with DNP3 0x01-0x06 but the port + banner co-requirement
disambiguates — see cross-protocol matrix tests in Phase 5.B.

Tests (tests/test_ics_protocol_dnp3_e2e.py — Rule #35b live canary
against the in-tree production YAML):
- test_production_catalog_loads_dnp3_v0: load smoke + schema shape
- test_dnp3_resolves_all_three_signals_fire: happy path
- test_dnp3_rejects_banner_only_combine_all_required: §SC5-NEW-ICS-2
- test_dnp3_rejects_port_only: §SC5-NEW-ICS-2
- test_dnp3_rejects_function_codes_only: §SC5-NEW-ICS-2

5 tests pass in 0.59s.

Cross-protocol matrix (Modbus + DNP3 + S7Comm disjointness) ships in
Phase 5.B alongside s7comm.yaml — those tests require both YAMLs
present.

Phase 5.B (next): s7comm.yaml + cross-protocol matrix tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
…hase 5.B)

Rule #52 instance digitalandrew#3 / Phase 5.B — Rule digitalandrew#25 per-piece commit: ships
the third + final v0 ICS protocol catalog production manifest, plus
the cross-protocol disjointness matrix tests that require both DNP3
+ S7Comm YAMLs present.

S7Comm (Siemens SIMATIC S7 over TPKT/COTP/TCP port 102) detector signals:
- string_in_binary needle set: 's7comm' / 'snap7' / 'libsnap7' / 'libs7'
  (case-insensitive; combine=any min_count=1)
- port_signature: 102 (IANA-registered ISO-TSAP; LE uint16)
- function_code_set: 11 S7 Job/Userdata PDU codes (0x04 Read Var /
  0x05 Write Var / 0x1A-0x1F Request Download to End Upload /
  0x28 PLC Control / 0x29 PLC Stop / 0xF0 Setup Communication) —
  min_count=3 within 512-byte window per W2-β A8 floor.

output.vendor='siemens' / vendor_product='simatic_s7' carries family-
level attribution; specific S7-1500 vs S7-300 version pinning is
forward-prepared via protocol_version (null for legacy S7Comm; future
v1 'v3' for S7Comm-Plus on S7-1200/1500).

W2-β §SC5-NEW-ICS-2 mitigation: combine=all_required prevents
port-102-alone false positives (other ISO-TSAP services on port 102).
Banner + FC table co-requirement disambiguates.

Tests (tests/test_ics_protocol_dnp3_s7comm_e2e.py):
- test_production_catalog_loads_s7comm_v0: load smoke + vendor attribution
- test_s7comm_resolves_all_three_signals_fire: happy path
- test_s7comm_rejects_port_only_no_iso_tsap_false_positive: §SC5-NEW-ICS-2
- test_modbus_blob_does_not_cross_match_dnp3_or_s7comm: cross-protocol
  disjointness (Modbus FCs 0x01-0x06 overlap with DNP3 but port + banner
  disambiguates under all_required)
- test_dnp3_blob_does_not_cross_match_modbus_or_s7comm: matrix
- test_s7comm_blob_does_not_cross_match_modbus_or_dnp3: matrix
- test_multi_protocol_firmware_matches_all_three: multi-protocol HMI
  cardinality (Session 1 postmortem Pattern digitalandrew#6 contract — resolve_all
  returns list[IcsProtocolMatch] supporting multi-family detection)

7 tests pass in 0.59s; combined Phase 5 sweep (5.A + 5.B) = 12 passed.

Phase 6 (next): Rule #52 Rule-of-Three promotion in CLAUDE.md +
.mex/context/conventions.md mirror per Rule digitalandrew#21 + .mex/patterns/
add-signal-kind.md recipe + Session 2 postmortem + /citadel:learn.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
… + postmortem (Phase 6)

Rule #52 instance digitalandrew#3 / Phase 6 — final commit of the ICS protocol
catalog Session 2 campaign. Promotes CLAUDE.md Rule #52 from
Rule-of-Two to **Rule-of-Three DURABLE BEYOND DEBATE** with ICS
protocol catalog as worked example digitalandrew#3 alongside bare-metal MCU
(instance digitalandrew#1, 2026-05-19) and file-format YAML registry (instance digitalandrew#2,
2026-05-19).

CLAUDE.md changes:
- Worked example digitalandrew#3 added (Rule-of-Three) — full surface description:
  11 closed Literals + 3 production YAMLs (modbus_tcp + dnp3 +
  s7comm) + Rule digitalandrew#39 walker triplet + Rule #44 cross-firmware MCP +
  HARDENED plugin escape hatch (MappingProxyType + closure-capture
  + freeze) + DUAL reaper + 5 finding sources + 80+ Rule #46
  paired META-CANARIES + all 10 §SC5-NEW-ICS-S2 mitigations.
- Methodology block extended to Rule-of-Four (P3.1 + P3.2 + ICS S1 +
  ICS S2 — across 4 applications W2-β surfaced 24 cross-feature
  attacks Wave-1 single-axis scouts architecturally couldn't see).
- "Promotable to Rule-of-Three" → "Promotable to Rule-of-Four";
  promotion date 2026-05-19 → 2026-05-22.

.mex/context/conventions.md mirror (Rule digitalandrew#21):
- Verify Checklist Rule #52 entry updated to Rule-of-Three with
  ICS as worked example digitalandrew#3; methodology refinement extended to
  cover all 4 applications.

.mex/patterns/add-signal-kind.md (NEW recipe — Rule-of-Three
threshold reached: P3.2.b TextFormatConstraint + P3.x
SubstringInHeadConstraint + Session 1 IcsStringInBinaryConstraint /
IcsFunctionCodeSetConstraint):
- Codifies the 4-element pattern for adding a new signal kind to a
  Rule #52 closed-grammar catalog: closed Literal + sub-model with
  extra='forbid' + symmetric-reject validator + evaluator in the
  closed dispatch table + Rule #46 paired META-CANARY (exhaustive
  + anti-hardcode AST + paired gate canary).
- Gotchas: cross-stack alignment ships ONE commit per Rule digitalandrew#25;
  cost-class is load-bearing for resolver cost-sort; YAML schema
  lock-step on Literal value list changes.

.planning/postmortems/postmortem-ics-protocol-session2-2026-05-22.md
(NEW):
- Full session narrative: Wave-1 + Wave-2 dispatch, 11-commit plan
  (10 shipped + this Phase 6), 0 reverts, bisect-clean, 980/980
  test sweep green.
- 3 failures documented (alembic ID collision; statusConfig
  apostrophe parser; cross-protocol test split) — all caught at
  design time, 0 production impact.
- 14 safety catches across 30+ individual gate fires.
- 5 recommendations (4 future-session follow-ups; 1 in this commit).
- 10 W2-β §SC5-NEW-ICS-S2-α..κ attacks catalogued with mitigations
  shipped or deferred-with-rationale.

Verified end-to-end:
- 980 tests pass across all ICS phases + baseline META-CANARIES
  (auth-gate, reapers, sbom_status_alignment, background_task_sweep)
- No regressions; bisect-clean across all 10 commits 0de1eba..cabb298
- DB CHECK constraints + alembic migrations applied + verified via
  psql column inspection + ck_findings_source 5 new ICS sources
- Rule #46 paired META-CANARIES across all gates fire on synthetic
  violations + paired canaries confirm gates don't degenerate to
  always-reject

Deferred follow-ups (documented in postmortem):
- file_format_catalog backfill to MappingProxyType + closure-capture
  hardening (Rule digitalandrew#21 mirror sweep — same §SC5-NEW-ICS-S2-α attack
  surface exists in the bare-dict file_format precedent)
- REST endpoint + frontend page (Scout D's /projects/{id}/ics-protocols
  sub-route — MCP access is sufficient for v0 ship)
- AST plugin-source pre-import scan (W2-β nice-to-have secondary
  defense beyond MappingProxyType + closure-capture)
- Feedback memory update: promote feedback_wave2_cross_feature_methodology
  to Rule-of-Four

Session totals: 10 commits (0de1eba..[this]) / +4,708 net LOC /
80 new tests / 0 reverts / W2-γ projected 5,569 — actual -29% under
midpoint due to Scout B's template-copy discipline / Rule-of-Three
DURABLE BEYOND DEBATE achieved.

Run /citadel:learn ics-protocol-session2-2026-05-22 next.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Prepends ICS protocol catalog Session 2 closure entry to the
Last-updated chain. CLAUDE.md Rule #52 promoted to Rule-of-Three
DURABLE BEYOND DEBATE with ICS as worked example digitalandrew#3. Records:
- 11 commits (0de1eba..c988c15) / +4,798 net LOC / 80 new tests /
  0 reverts / bisect-clean / 980-test sweep green
- W2-γ SINGLE-SESSION verdict HONORED (actual -14% under midpoint)
- Wave-1+Wave-2 methodology now Rule-of-Four (24 W2-β attacks
  surfaced across 4 applications)
- Deferred follow-ups: file_format_catalog backfill to MappingProxyType
  (Rule digitalandrew#21 mirror sweep); REST router + frontend page; AST plugin-
  source pre-import scan
- Session 3 (Fix digitalandrew#9 full Rule #52 extraction-strategy refactor)
  queued as Rule-of-Four north star

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Two algorithmic gaps in `_compute_roots_sync` contributed to the L4T BSP
SBOM regression (postmortem-over-constraint-sweep-2026-05-22 follow-ups
digitalandrew#2 + digitalandrew#3):

1. `_find_extraction_container` only climbed past partition-like names
   (`rootfs`, `partition_*`, Android siblings). When `extracted_path`
   landed on a vendor-BSP component subdir (`Linux_for_Tegra/bootloader/`
   for L4T), the climb stopped, trapping detection_roots inside that
   subdir alone. Sibling components (`nv_tegra/`, `kernel/`, `tools/`)
   were invisible to downstream walkers.

2. `_find_unblob_extraction_top` only recognised `_extract` suffix
   (unblob's recursive cracker). gtar / binwalk emit `_extracted`
   (trailing `-ed`); the climb refused to walk past those. The climb
   AND the path-string fallback now both accept either suffix.

Changes:
- New `_is_bsp_component_name(name)` predicate alongside
  `_is_partition_like_name`, sharing the climb contract.
- `_find_extraction_container` climbs past either kind.
- `_find_unblob_extraction_top` climb + path-string fallback recognise
  `_extracted` alongside `_extract`.
- Sibling-archive promotion branch in `_compute_roots_sync`: when the
  extraction container is itself `*_extract`/`*_extracted` and sits
  directly under the extraction marker (`extracted/`), promote sibling
  archives with the same suffix that pass shallow-sweep gates.

Tests (5 new):
- `test_compute_roots_l4t_bsp_climbs_past_bootloader` — verifies the
  climb traverses past `bootloader/` so `nv_tegra/`, `kernel/`, BSP top
  all appear in detection_roots.
- `test_compute_roots_walks_past_extracted_suffix` — nested
  `X.tar.gz_extracted/inner/Y.tar.gz_extracted/leaf` climbs correctly.
- `test_sibling_archive_extracted_dirs_both_become_roots` — TWO sibling
  `*.tar.gz_extracted` dirs under `extracted/` both surface.
- `test_canary_is_bsp_component_name_rejects_nonsense` — Rule #46
  META-CANARY: predicate rejects nonsense names so the BSP test can't
  pass for the wrong reason.
- `test_canary_extracted_suffix_climb_rejects_non_extract_segment` —
  Rule #46 META-CANARY pair: climb does NOT walk past non-_extract
  segments (negative) and DOES walk past `_extracted` (positive).

Validation:
- backend pytest tests/test_firmware_paths.py: 52 passed (47 baseline + 5 new).
- e6e45f24 L4T BSP smoke (OLD broken extracted_path =
  `.../L4T_BSP_SecureBoot.R32.3.1.tar.gz_extracted/bootloader`): now
  returns 12 detection roots INCLUDING the sibling
  `DS1.2-jetson-tx2-cot.tegraflash.tar.gz_extracted` archive,
  `nv_tegra/l4t_deb_packages`, the BSP top container, `kernel/`,
  `rootfs/` — pre-fix returned only the single bootloader subdir.

CLAUDE.md Rule digitalandrew#18 regression guards (`test_post_relocation_layout_*`,
`test_linux_rootfs_only_container_not_included`) still pass — the
changes are additive to the climb logic, not replacement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CrimsonGlory pushed a commit to CrimsonGlory/wairz that referenced this pull request Jun 5, 2026
Phase 6 — final commit of the kernel_image campaign. Closes the
operator-facing surface with 2 MCP tools:

  audit_kernel_config_firmware            — operator trigger walker
  lookup_kernel_config_across_firmwares   — Rule #44 mandatory corpus aggregator

`audit_kernel_config_firmware`
------------------------------
Rule digitalandrew#33 .a idempotent operator trigger. 409-shaped JSON response when
status is already queued/running; otherwise transitions idle → queued +
spawns the background runner (run_kernel_config_audit_background).
MCP-handler discipline per Rule digitalandrew#3: flush() then commit before spawning
the detached task so the runner's fresh session sees queued.

`lookup_kernel_config_across_firmwares`
---------------------------------------
Rule #44 mandatory cross-firmware aggregator. Filters Findings by
finding_source (one of the 10 kernel_config_* values), groups by
firmware, returns match_count + supply_chain_signal flag (true when
>=2 firmware share the same finding).

Use cases:
  "which firmware in this project ship a kernel with KASLR off?"
    → finding_source='kernel_config_no_kaslr'
  "which devices fleet-wide are missing module signing?"
    → finding_source='kernel_config_no_module_sig', scope='global'
  "which devices have /dev/mem exposed?"
    → finding_source='kernel_config_devmem_enabled'

Defends supply-chain regression discovery: two distinct vendors / SKUs
shipping kernels with the same hardening miss is a signal worth
investigating (shared upstream BSP, same OEM template, etc.).

Integration smoke (Rule digitalandrew#11 post-cutover)
-----------------------------------------
$ create_tool_registry()
- 337 total MCP tools registered (was 335; +2)
- audit_kernel_config_firmware visible
- lookup_kernel_config_across_firmwares visible
- enum validation: only the 10 kernel_config_* values accepted

$ get_walker_auto_triggers()
- 29 total triggers (was 28; +auto_kernel_config_audit_firmware_safe)

$ STATE_MACHINE_REAPER_CONFIGS
- 10 entries (was 9; +kernel_config_audit_status)

Campaign closure
----------------
6 commits total in the kernel_image campaign per Rule digitalandrew#25 per-piece:

  f947b59  Phase 1: lift IKCFG extraction to shared helper + gzip-bomb cap
  108102b  Phase 2: kernel-Image parser (4 FORMAT subclasses)
  ec3f6e8  Phase 3: 10 parser tests incl. Rule #46 META-CANARIES
  dd8af4e  Phase 4: cross-stack alignment for 10 kernel_config_* sources
  310e4b3  Phase 5a: walker state-machine columns
  fb54d27  Phase 5b: KSPP walker (Rule digitalandrew#39 triplet + 11 tests)
  <this>   Phase 6: MCP audit + lookup tools (Rule #44 mandatory)

End-to-end: parser extracts IKCFG → walker emits Findings → MCP
exposes per-firmware audit + corpus-wide lookup. DS1 Tegra L4T R32.3.1
reference case will fire 7 high-severity Findings each on the 2
firmware after the next backend+worker rebuild (Rule digitalandrew#8).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant