observability: implement redaction filter (default strict)

[AbdelStark]

Problem

RFC-0013 §3.5 and docs/spec/06-security.md require a single redaction chokepoint between callers and the log sink: DNA strings ≥ 20 bp, deny-listed field names, and out-of-allowlist payloads must never leak. INV-OBS-4 ("DNA strings ≥ 20 bp never appear in any log record") is non-negotiable.

Solution

• geno_lewm/_redaction.py — four-rule filter:

  1. Per-event allowlist (EventSpec.allowed_keys): keys not listed are soft-dropped (registry drift, not a bypass — does NOT raise even in strict mode).
  2. Type allowlist: None | bool | int | float | str and shallow containers thereof. bytes / sets / deep nesting / tensors all drop; raise in strict mode.
  3. DNA pattern: ^[ACGTNacgtn]{20,}$ matched at any depth → drop; raise in strict mode.
  4. Deny list (vcf_content, genotype, sample_id, user_email, email, phone, address, dob, birthdate) — at any depth → drop; raise in strict mode.

  Strict mode is on by default (GENO_LEWM_REDACTION_STRICT=0 disables). All raises go through geno_lewm.errors.InvariantViolation — the linter from errors: AST linter raise_geno_lewm_error and registered_error_code #22 confirms.

• geno_lewm/observability.py — extends EventSpec with allowed_keys: frozenset[str]. Populated for every v0.1 event with realistic key sets. Tightening is MAJOR; adding is MINOR.

• The filter is wired into GenoLeWMLogger._log so every record passes through the chokepoint before JSON serialization. Unknown events fall back to an empty allowlist — every payload key is soft-dropped, no leak possible.

• RedactionStats (counter object): dropped_keys / dropped_denied / dropped_dna / dropped_type. The metric geno_lewm.observability.redacted_keys will be exported by observability: implement metrics registry + Prometheus textfile exporter #25 — today the counter is observable via redaction_stats().

Validation

$ python -m pytest tests/ -q
205 passed in 0.62s

$ python -m tools.lint.check_error_codes
$ echo $?
0

Tests:

• tests/unit/test_redaction.py (16 cases): strict-mode raises on every rule including nested DNA / deny-listed keys; permissive drops + counts; type allowlist (scalars / lists / shallow dicts); defensive copy; per-event allowlist round-trip for every registered event.
• tests/property/test_redaction.py (2 tests, 12k payloads total): permissive run over 10k random payloads — zero leaks (the acceptance criterion); strict-mode run over 2k — either raises cleanly or zero leaks.
• Existing observability tests updated where they used keys outside the new allowlists (only one record-shape test had to change: cfg → config_path).

Caveats / out of scope

• geno_lewm.observability.redacted_keys as a Prometheus-exported metric depends on the metrics registry — that's observability: implement metrics registry + Prometheus textfile exporter #25. Counter is live today, exporter wiring lands with observability: implement metrics registry + Prometheus textfile exporter #25.
• OQ-OBS-2 (clinvar_id re-personalization) intentionally not in the deny list — the open question stays open.
• The property test uses random.Random not Hypothesis to keep the bootstrap suite zero-dependency. Acceptable per the issue ("10k random payloads — zero leaks").

Closes #24