Merge pull request #380 from Cisco-Talos/dev

chore: release EvidenceForge 1.15.0

Author: DavidJBianco

.github/workflows/ci.yml

@@ -24,7 +24,7 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
 
       - name: Install uv
-        uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
+        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
 
       - name: Set up Python
         uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7
@@ -43,7 +43,7 @@ jobs:
   tests:
     name: Tests (Python ${{ matrix.python-version }})
     runs-on: ubuntu-latest
-    timeout-minutes: 15
+    timeout-minutes: 25
     strategy:
       fail-fast: false
       matrix:
@@ -54,9 +54,10 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
 
       - name: Install uv
-        uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
+        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
         with:
           enable-cache: true
+          prune-cache: true
 
       - name: Set up Python
         uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7

.github/workflows/release-slow.yml

@@ -22,9 +22,10 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
 
       - name: Install uv
-        uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
+        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
         with:
           enable-cache: true
+          prune-cache: true
 
       - name: Set up Python
         uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7

CHANGELOG.md

@@ -6,6 +6,100 @@ Detailed development history for the EvidenceForge project. Transferred from TOD
 
 ## Unreleased
 
+## v1.15.0 (2026-08-10)
+
+This minor release completes the non-breaking canonical event and action-bundle
+architecture, substantially strengthens cross-source lifecycle realism, and
+incorporates repeated blind-assessment fixes across endpoint, network, email,
+and evaluation behavior.
+
+**Canonical architecture and safety**
+
+- Added the canonical contract foundation and completed occurrence, dispatch,
+  session/authentication, interactive-session, network-projection,
+  world-capability, evaluation-validity, and workload-safety boundaries
+  (`0b0ad32c`, `b89855c9`, `739dbea2`, `b2f1dd68`, `91247365`, `c139c77a`,
+  `03401ad0`, `868eb35d`, `53934a16`).
+- Closed the post-migration realism blockers without restoring compatibility
+  layers or changing the public scenario schema (`2ab8e5e9`).
+
+**Windows endpoint, identity, and session realism**
+
+- Enforced module lifecycles, channel-time-derived record IDs, and more varied
+  startup module profiles (`49148992`, `f4979524`, `df33ad10`).
+- Modeled authentication attempts and privilege-event ownership, and kept
+  desktop bootstrap and session lifecycles attached to their canonical logons
+  (`cfd6e10a`, `5c7786d6`, `f5604488`, `2942320d`).
+- Made enterprise software cohorts and endpoint singleton planning coherent
+  and atomic across process and network ownership (`c0a01395`, `b96a96be`,
+  `4a816bcc`, `b7275437`, `d5c6900a`).
+- Reused resident desktop resources and named service processes, and bound
+  endpoint effects and ambient Windows file activity to native process owners
+  (`b3cd49a0`, `d7162d57`, `4f062448`, `90a997db`).
+- Preserved host timezone state and canonical shell bootstrap timing, modeled
+  complete Windows session-process lifecycles, and prevented duplicate RDP
+  shell bootstrap (`bfbe3ed3`, `a1227098`, `ea914c19`, `81fc795c`, `4cc75358`).
+
+**Remote access and Linux lifecycle ownership**
+
+- Enforced Linux process-observation chronology and bounded SSH client
+  ownership, then attached Windows SSH clients, credentials, receiver
+  lifecycles, routine scheduling, and responder processes to their owning
+  sessions (`8616105d`, `a8ddb632`, `2312ec24`, `eeb97e11`, `00a36029`,
+  `316f0da7`, `84294984`).
+- Preserved explicit-credential source semantics and modeled one-shot caller
+  behavior, while assigning Linux SMB activity to native client processes
+  (`f0c6117d`, `17f73d05`, `ebab5c8f`).
+
+**Network, protocol, proxy, and source-native projection**
+
+- Ordered HTTP activity and made persistent transactions share coherent,
+  durable observation identities (`04eb354c`, `8f1593f3`, `9dc294ad`,
+  `481f0ed8`).
+- Ordered and retained ASA translations, modeled coherent sensor clocks and
+  passive datagram accounting, and clipped perimeter teardown observations to
+  visible intervals (`20f08d56`, `2ed74a5d`, `dc4d519c`, `0e75af19`,
+  `e5f5aac8`).
+- Defined and annotated proxy tunnel accounting, omitted unavailable totals,
+  reconciled byte scopes, and attached proxy flows to native client processes
+  (`76421d75`, `1f9e4f5d`, `1f854be8`, `998d9942`, `1ee7ad95`, `ea53038a`).
+- Enforced half-open collection windows, gated analyzers on captured content,
+  normalized Zeek histories, preserved TLS certificate observation, and
+  excluded implausible external client networks (`4565b2e0`, `9e3e7c28`,
+  `e3e9ed9a`, `fc51a85d`, `1f9dd5d1`).
+- Rendered source-native output against explicit versioned schemas
+  (`c3f17ead`).
+
+**Long-lived services, maintenance, packages, and applications**
+
+- Honored bounded process deadlines and completed ownership/closure for APT
+  frontends, package managers, and their transport helpers (`318dfe59`,
+  `490e363b`, `eba43cd9`, `ed2e4d80`).
+- Modeled durable ServiceHealth agents and stateful fleet maintenance
+  (`2a205e0b`, `294d2c80`).
+- Reused resident mail clients, preserved durable but varied rsyslog health
+  state, and aligned generated email artifacts with their owning applications
+  (`07751e47`, `481a48d1`, `772ebabc`, `60ae2587`).
+
+**Evaluation correctness and regression coverage**
+
+- Reconciled storyline pivot identity, explicit-proxy paths, duration windows,
+  account deletion, and truthful already-satisfied action no-ops, restoring
+  full pivot linkability on the expanded IDS assessment (`dcb89ca6`).
+- Corrected the private-IPv6 network regression expectation (`f17096f0`).
+
+**Assessment, documentation, and CI**
+
+- Published the complete realism review, reconciled its remediation roadmap,
+  and recorded the post-Batch-2 and post-Batch-7b effectiveness gates
+  (`7238ca61`, `2b473a3d`, `f2a15f6a`, `0934e808`, `ca03d731`).
+- Recorded assessment loops 15-34 and their individual handoffs, then completed
+  the expanded iteration-test loop documentation (`ff7928e0`, `2eb98588`,
+  `53c454aa`, `6472d02a`, `45467a6b`, `161f30ff`, `6339e586`).
+- Extended CI time allowance for the expanded validation suite (`40ac5f71`).
+- Updated Ruff to 0.16.1 and setup-uv to 9.0.0, retaining explicit cache
+  pruning for cache-enabled GitHub Actions jobs (`8c4685c7`, `93a4510e`).
+
 ## v1.14.1 (2026-08-04)
 
 This patch release corrects source-native network and endpoint behavior found

README.md

@@ -13,7 +13,11 @@ For background on the project and why we built it, read our announcement:
 
 Most synthetic log generators produce isolated, single-format data that experienced analysts identify as fake within seconds. EvidenceForge takes a fundamentally different approach:
 
-- **Consistency by construction.** A canonical `SecurityEvent` model feeds all log formats from a single source of truth. Two emitters cannot disagree about a port number, timestamp, or LogonID because there is only one value — on the event object. This eliminates the cross-source inconsistencies that are the #1 tell of synthetic data.
+- **Consistency by construction.** Canonical events and immutable plans own shared connection,
+  identity, lifecycle, and protocol facts for migrated action families; emitters perform only
+  source-native projection. Dispatch rejects unknown or incomplete canonical occurrences. Some
+  mutable compatibility views remain during the approved migration, and the explicit `raw` path
+  is outside cross-source consistency guarantees.
 
 - **Causal event ordering.** Events respect real-world dependencies — DNS queries precede connections, Kerberos TGT/TGS precede domain logons, audit events follow administrative commands. A composable rule engine auto-generates prerequisites with realistic timing offsets, so the data tells a coherent causal story across log sources.
 
@@ -25,7 +29,9 @@ Most synthetic log generators produce isolated, single-format data that experien
 
 - **Deterministic engine, LLM-assisted authoring.** Scenario creation uses Claude Code Skills for interactive, research-backed attack planning. Log generation is fully deterministic — no LLM calls, no API costs, reproducible output every time.
 
-- **Built-in quality evaluation.** A 4-pillar scoring framework (21 sub-scores) measures parseability, plausibility, causality, and timing. Know exactly how good your data is before using it.
+- **Built-in quality evaluation.** A 4-pillar scoring framework (22 sub-scores) measures
+  parseability, plausibility, causality, and timing, with additional concern-oriented views for
+  source schema, canonical invariants, scenario completeness, and distribution realism.
 
 ## Quick Start
 
@@ -80,9 +86,9 @@ For scripted or non-interactive use:
 
 | Command | Description |
 |---------|-------------|
-| `eforge generate <scenario.yaml> -o <dir>` | Generate logs from a scenario file |
-| `eforge validate <scenario.yaml>` | Validate scenario schema and cross-references |
-| `eforge eval <output_dir> -s <scenario.yaml>` | Evaluate data quality (4 pillars, 21 sub-scores) |
+| `eforge generate <scenario.yaml> -o <dir> [--seed N] [--allow-large-workload]` | Generate logs; `--seed` overrides the scenario seed and the trusted workload override bypasses resource limits, not path safety |
+| `eforge validate <scenario.yaml> [--allow-large-workload]` | Validate schema, cross-references, and the default workload envelope |
+| `eforge eval <output_dir> -s <scenario.yaml> [--allow-large-evaluation]` | Evaluate quality (4 pillars, 22 sub-scores); the trusted override bypasses evaluator corpus limits |
 | `eforge info [field]` | Show installation info, config paths, and data inventories. Pass a dot-path field for a specific value (e.g., `eforge info personas`). Use `--fields` to list available fields, `--json` for machine output. |
 | `eforge validate-config` | Validate config files for cross-reference integrity. Use `--json` for machine output. |
 | `eforge install-skills [--agent all\|claude\|chatgpt\|codex] [--global]` | Install project-local or user-wide agent skills; defaults to all agents (`codex` aliases `chatgpt`) |
@@ -128,7 +134,8 @@ Every generated scenario includes a `GROUND_TRUTH.md` file. Attack scenarios doc
 - **Ground truth documentation** — Every run generates a GROUND_TRUTH.md; attack scenarios include narrative, timeline, and IOCs
 - **Parallel generation** — Threaded emitters write all formats simultaneously with temporal consistency
 - **Scenario validation** — Cross-reference checking, uniqueness constraints, and network topology validation
-- **Data quality evaluation** — 4-pillar scoring framework (21 sub-scores) with acceptance criteria
+- **Data quality evaluation** — 4-pillar scoring framework (22 sub-scores), concern-oriented
+  diagnostic categories, non-vacuous applicability, and acceptance criteria
 - **Multi-timezone support** — Pattern-based timezone overrides per system hostname
 
 ## Supported Log Formats
@@ -214,7 +221,19 @@ EvidenceForge includes a built-in evaluation framework that scores generated dat
 | Causality | 25% | Causal ordering, event presence, indicator accuracy, pivot linkability |
 | Timing | 20% | Attack-chain timing, burstiness, diurnal patterns, volume adequacy |
 
-**Two-tier acceptance**: hard gates (minimum, must pass) + aspirational targets (stretch goals, informational). Hard gates: Spec Conformance ≥ 95%, Value Plausibility ≥ 95%, Causal Ordering ≥ 90%, Event Presence ≥ 85%. Thresholds are configurable in `src/evidenceforge/config/evaluation/thresholds.yaml`.
+**Two-tier acceptance**: applicable hard gates must pass, while aspirational targets remain
+informational. Gates cover source conformance, value and field consistency, exact IDS integrity,
+causal/scenario reconciliation, and linkability. The authoritative thresholds are configurable in
+`src/evidenceforge/config/evaluation/thresholds.yaml`.
+
+The compatibility pillars remain the weighted public score. The report also groups the same
+applicable measures by review concern: source/schema fidelity, canonical cross-source invariants,
+declared-scenario completeness, and distribution realism. Required measures with no applicable
+denominator are reported as unavailable instead of receiving a vacuous perfect score.
+
+By default, evaluation accepts at most 512 MiB of input, 10,000 files, and 500,000 parsed records.
+Use `--allow-large-evaluation` only for a trusted corpus after reviewing available memory. The
+override does not relax file or parser safety checks.
 
 ```bash
 uv run eforge eval ./output -s scenario.yaml
@@ -235,7 +254,7 @@ GenerationEngine (hour-by-hour orchestration)
 WorldModel / WorldPlanner (compile host roles, user placement, session bootstrap)
     |
     v
-ActivityGenerator (builds SecurityEvents with composable contexts)
+ActivityGenerator (builds canonical occurrences with composable contexts)
     |
     v
 EventDispatcher (routes to StateManager + matching emitters)
@@ -257,7 +276,7 @@ emitters apply it only where file shape differs.
 
 `WorldModel` compiles authoritative host and user capabilities from scenario fields like `primary_system`, `roles`, `services`, and workstation assignments. `WorldPlanner` then chooses realistic interactive, network, SSH, and RDP session paths before `ActivityGenerator` emits the correlated evidence.
 
-See [Architecture Documentation](docs/ARCHITECTURE.md) for the full deep dive including the world-model layer, SecurityEvent model, state management, and emitter system.
+See [Architecture Documentation](docs/ARCHITECTURE.md) for the full deep dive including the world-model layer, CanonicalOccurrence model, state management, and emitter system.
 
 ## Development
 
@@ -312,7 +331,7 @@ full-dataset runner command, and failure report details.
 ### Design Documents
 
 - [PRD](docs/design/PRD.md) — Product requirements and specifications
-- [Event Model Design](docs/design/event-model-prd.md) — Canonical SecurityEvent architecture
+- [Event Model Design](docs/design/event-model-prd.md) — Canonical occurrence architecture
 - [Data Quality Design](docs/design/data-quality-prd.md) — Evaluation framework design
 - [Research Report](docs/design/synthetic-log-generation-research.md) — Analysis of existing tools