Fix eval issues from Phase 5 generator changes

- Fix eCAR REGISTRY NDJSON bug: backslash-escape registry_key and
  registry_value in Jinja2 template (unescaped Windows paths broke
  json.loads, causing ~2,571 multi-line records)
- Add missing fields to Windows 4625 variant (TransmittedServices,
  LmPackageName, KeyLength, ProcessId, ProcessName)
- Fix flaky baseline logon test to accept 4624 or 4625
- Update TODO.md Phase 5.1/5.2/5.5 checkboxes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: DavidJBianco

TODO.md

@@ -527,65 +527,65 @@
 
 **Goal:** Fix the hard acceptance failure (parsability) and the most obvious per-record tells. Smallest changes, biggest eval score impact.
 
-- [ ] **Fix empty SID fields in Windows events**
+- [x] **Fix empty SID fields in Windows events**
   - Generate realistic SIDs: `S-1-5-21-{domain_sub_authorities}-{user_rid}`
   - Assign per-domain base SID at engine init (e.g., `S-1-5-21-3623811015-3361044348-30300820`)
   - Map each user to a unique RID (starting at 1001, incrementing)
   - Well-known SIDs for system accounts: `S-1-5-18` (SYSTEM), `S-1-5-19` (LOCAL SERVICE), `S-1-5-20` (NETWORK SERVICE)
   - Populate `SubjectUserSid` and `TargetUserSid` in all Windows event data dicts in `activity.py`
   - Files: `activity.py`, `engine.py` (SID registry initialization)
-- [ ] **Add logoff generation to baseline activity**
+- [x] **Add logoff generation to baseline activity**
   - Track active sessions per user; probabilistically end sessions (e.g., 30% chance per hour after first hour)
   - Emit Windows 4634 (logoff) and eCAR USER_SESSION/LOGOUT
   - Ensure logon→activity→logoff ordering within each session
   - Files: `activity.py` (`execute_baseline_activity`), `engine.py` (`_generate_baseline`)
-- [ ] **Vary Zeek conn_state and history strings**
+- [x] **Vary Zeek conn_state and history strings**
   - Replace hardcoded `SF`/`ShADadfF` with probabilistic selection
   - Connection states: SF (85%), S0 (5%), S1 (3%), REJ (2%), RSTO (3%), RSTR (1%), OTH (1%)
   - Generate history strings that match conn_state (e.g., S0→`S`, REJ→`Sr`, RSTO→`ShADaR`)
   - Adjust orig_bytes/resp_bytes to be consistent (e.g., S0 = 0 resp_bytes)
   - Files: `activity.py` (`generate_connection`)
-- [ ] **Expand process template pools**
+- [x] **Expand process template pools**
   - Windows: add system backbone (svchost.exe, lsass.exe, explorer.exe, services.exe, csrss.exe, RuntimeBroker.exe, SearchIndexer.exe) + user apps (chrome.exe, firefox.exe, outlook.exe, teams.exe, OneDrive.exe)
   - Linux: add system processes (systemd, cron, sshd, rsyslogd, NetworkManager) + user apps (firefox, thunderbird, git, docker, python3)
   - Per-persona weighting: developers see more compilers/editors, executives see more Office/browser
   - Files: `activity.py` (PROCESS_TEMPLATES, PROCESS_TEMPLATES_LINUX)
-- [ ] Test: Parsability score ≥ 98% (SIDs valid format)
-- [ ] Test: Logoff events present in output, paired with logons
-- [ ] Test: conn_state distribution is varied (not 100% SF)
-- [ ] Test: Process path count > 30 unique paths
+- [x] Test: Parsability score ≥ 98% (SIDs valid format)
+- [x] Test: Logoff events present in output, paired with logons
+- [x] Test: conn_state distribution is varied (not 100% SF)
+- [x] Test: Process path count > 30 unique paths
 
 ### 5.2 Event Type Diversity
 
 **Goal:** Expand the vocabulary of events generated. Address "only 2 Event IDs" and "only 2 eCAR object types."
 
-- [ ] **Add Windows Event IDs to format definition and emitters**
+- [x] **Add Windows Event IDs to format definition and emitters**
   - 4625: Failed logon (account does not exist, bad password, account locked)
   - 4672: Special privileges assigned to new logon (admin logons)
   - 4689: Process termination (pair with 4688)
   - 4648: Explicit credential logon (RunAs, scheduled tasks)
   - 5156: Windows Filtering Platform connection allowed (host firewall)
   - Update `windows_event_security.yaml` schema, Jinja2 templates, and validation rules
   - Files: `formats/definitions/windows_event_security.yaml`, `generation/emitters/windows.py`
-- [ ] **Generate failed logons in baseline**
+- [x] **Generate failed logons in baseline**
   - 5-15% of logon attempts fail (configurable via persona risk_profile)
   - Failure reasons: bad password (most common), account locked, expired password
   - Emit Windows 4625 + eCAR USER_SESSION/LOGON_FAILURE
   - Files: `activity.py` (`generate_logon` or new `generate_failed_logon`)
-- [ ] **Add eCAR object type diversity**
+- [x] **Add eCAR object type diversity**
   - FILE/CREATE, FILE/MODIFY, FILE/DELETE — generated alongside process activity
   - REGISTRY/MODIFY — Windows system processes and app installs
   - FLOW/CONNECT — parallel to Zeek connections for eCAR-equipped hosts
   - MODULE/LOAD — DLL loads for Windows processes
   - Files: `activity.py` (new methods), `generation/emitters/ecar.py`
-- [ ] **Add process termination events**
+- [x] **Add process termination events**
   - Pair 4689 with 4688: terminate processes after realistic duration (seconds to hours)
   - Track running processes in StateManager, probabilistically terminate
   - Files: `activity.py`, `state_manager.py`
-- [ ] Test: ≥ 6 unique Windows Event IDs in output
-- [ ] Test: ≥ 5 unique eCAR object types in output
-- [ ] Test: Failed logon rate between 5-15% of total logon events
-- [ ] Test: Process termination events present, paired with creation
+- [x] Test: ≥ 6 unique Windows Event IDs in output
+- [x] Test: ≥ 5 unique eCAR object types in output
+- [x] Test: Failed logon rate between 5-15% of total logon events
+- [x] Test: Process termination events present, paired with creation
 
 ### 5.3 Protocol & Network Diversity
 
@@ -654,34 +654,34 @@
 
 **Goal:** Replace uniform event distribution with realistic human timing patterns. Address "metronomic spacing" and "hard rectangular work hours."
 
-- [ ] **Soft ramp-up/ramp-down for work hours**
+- [x] **Soft ramp-up/ramp-down for work hours**
   - Replace binary on/off with sigmoid curve: 10% activity at work_start-1h, ramp to 100% by work_start+1h
   - Soft lunch dip (50% reduction, not 0%)
   - Evening tail: 20% activity for 1-2 hours after work_end
   - Occasional late-night activity (1-3% probability per user per night)
   - Files: `engine.py` (`_calculate_events_for_hour`)
-- [ ] **Activity cluster model**
+- [x] **Activity cluster model**
   - Replace `_distribute_events_in_hour()` uniform distribution with cluster generation
   - Each "activity" becomes a burst of 3-15 correlated events over 5-30 seconds
   - Cluster types per persona: developer (editor→compile→test→git), executive (email→calendar→browser), analyst (query→export→review)
   - Inter-cluster gaps: 2-15 minutes (exponential distribution)
   - Files: `engine.py` (`_distribute_events_in_hour` → `_generate_activity_clusters`), `activity.py` (cluster templates)
-- [ ] **Per-user work hour jitter**
+- [x] **Per-user work hour jitter**
   - Randomize each user's actual start/end/lunch times ±30min from persona defaults
   - Applied once at engine init, consistent throughout scenario
   - Early arrivals, late starters, short/long lunches
   - Files: `engine.py` (init), `engine.py` (`_calculate_events_for_hour`)
-- [ ] **Per-persona behavioral differentiation**
+- [x] **Per-persona behavioral differentiation**
   - Developers: longer clusters (sustained coding sessions), more process events, fewer web connections
   - Executives: short frequent clusters (meetings → quick email checks), more web/email, fewer processes
   - Analysts: medium clusters with heavy DB/query activity
   - Each persona type gets distinct cluster templates and inter-cluster timing
   - Files: `activity.py` (persona-specific cluster definitions)
-- [ ] Test: Events cluster with sub-second intra-cluster timing
-- [ ] Test: Inter-cluster gaps follow non-uniform distribution
-- [ ] Test: Work hour profile shows gradual ramp (not step function)
-- [ ] Test: Per-user timing varies (different arrival times)
-- [ ] Test: Human burstiness CV > 1.0 (eval dimension)
+- [x] Test: Events cluster with sub-second intra-cluster timing
+- [x] Test: Inter-cluster gaps follow non-uniform distribution
+- [x] Test: Work hour profile shows gradual ramp (not step function)
+- [x] Test: Per-user timing varies (different arrival times)
+- [x] Test: Human burstiness CV > 1.0 (eval dimension)
 
 **Phase 5 Milestone:** Generated data passes qualitative review — no instant tells. Eval score ≥ 85, all hard acceptance criteria pass. Background noise has protocol diversity (TCP+UDP+ICMP), event type depth (≥ 6 Windows Event IDs, ≥ 5 eCAR objects), realistic timing patterns, and hundreds of unique destination IPs.
 

src/evidenceforge/formats/definitions/ecar.yaml

@@ -196,11 +196,11 @@ output:
         {%- endif %}
         {%- if registry_key %}
         {%- if command_line or image_path or file_path or src_ip or src_port or dst_ip or dst_port or protocol or md5 or sha256 %},{% endif %}
-        "registry_key": "{{ registry_key }}"
+        "registry_key": "{{ registry_key | replace('\\', '\\\\') | replace('"', '\\"') }}"
         {%- endif %}
         {%- if registry_value %}
         {%- if command_line or image_path or file_path or src_ip or src_port or dst_ip or dst_port or protocol or md5 or sha256 or registry_key %},{% endif %}
-        "registry_value": "{{ registry_value }}"
+        "registry_value": "{{ registry_value | replace('\\', '\\\\') | replace('"', '\\"') }}"
         {%- endif %}
       }
     }

src/evidenceforge/formats/definitions/windows_event_security.yaml

@@ -401,6 +401,31 @@ variants:
           min_value: 0
           max_value: 65535
 
+      - name: TransmittedServices
+        type: string
+        required: false
+        description: "List of transmitted services"
+
+      - name: LmPackageName
+        type: string
+        required: false
+        description: "LAN Manager package name"
+
+      - name: KeyLength
+        type: integer
+        required: false
+        description: "Length of NTLM session security key"
+
+      - name: ProcessId
+        type: hex_string
+        required: false
+        description: "Process ID of the logon process"
+
+      - name: ProcessName
+        type: string
+        required: false
+        description: "Full path of the logon process executable"
+
   # EventID 4672: Special privileges assigned to new logon
   - name: special_privileges
     event_id: "4672"

tests/unit/test_activity.py

@@ -350,10 +350,10 @@ def test_execute_baseline_activity_logon(self, activity_gen, test_user, test_sys
 
         activity_gen.execute_baseline_activity(test_user, test_system, timestamp, 'logon')
 
-        # Verify Windows emitter received 4624 event
+        # Verify Windows emitter received logon event (4624 success or 4625 failed)
         assert mock_emitters['windows_event_security'].emit_event.called
         event_data = mock_emitters['windows_event_security'].emit_event.call_args[0][0]
-        assert event_data['EventID'] == 4624
+        assert event_data['EventID'] in (4624, 4625)
 
     def test_execute_baseline_activity_process_creates_session(self, activity_gen, test_user, test_system, state_manager, mock_emitters):
         """execute_baseline_activity should create session before process if needed."""

tests/unit/test_phase5_ecar_diversity.py

@@ -1,11 +1,15 @@
 """Unit tests for Phase 5.2.4: eCAR object type diversity."""
 
+import json
 import pytest
 from datetime import datetime, timezone
+from pathlib import Path
 from unittest.mock import Mock
 
 from evidenceforge.generation.activity import ActivityGenerator
+from evidenceforge.generation.emitters.ecar import EcarEmitter
 from evidenceforge.generation.state_manager import StateManager
+from evidenceforge.formats.loader import load_format
 from evidenceforge.models import User, System
 
 
@@ -131,6 +135,62 @@ def test_multiple_object_types_from_processes(self, activity_gen, test_user, win
         assert len(object_types) >= 3, f"Only {len(object_types)} object types: {object_types}"
 
 
+class TestEcarRegistryBackslashEscaping:
+    """Test that REGISTRY events with Windows paths produce valid NDJSON."""
+
+    def test_registry_key_with_backslashes_produces_valid_json(self, tmp_path):
+        """Registry keys with backslashes must be properly escaped in JSON output."""
+        fmt_def = load_format("ecar")
+        output_file = tmp_path / "ecar.json"
+        emitter = EcarEmitter(fmt_def, output_file, threaded=False)
+
+        event_data = {
+            'timestamp': datetime(2024, 3, 15, 10, 0, 0, tzinfo=timezone.utc),
+            'hostname': 'WKS-01',
+            'object': 'REGISTRY',
+            'action': 'MODIFY',
+            'pid': 1234,
+            'principal': 'alice.smith',
+            'registry_key': r'HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings',
+            'registry_value': 'ProxyEnable',
+        }
+        emitter.emit_event(event_data)
+        emitter.flush()
+
+        content = output_file.read_text()
+        lines = [l for l in content.strip().splitlines() if l.strip()]
+        assert len(lines) == 1, f"Expected single-line NDJSON, got {len(lines)} lines"
+
+        parsed = json.loads(lines[0])
+        assert parsed['object'] == 'REGISTRY'
+        assert parsed['action'] == 'MODIFY'
+        assert parsed['properties']['registry_key'] == r'HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings'
+        assert parsed['properties']['registry_value'] == 'ProxyEnable'
+
+    def test_registry_value_with_quotes_escaped(self, tmp_path):
+        """Registry values containing quotes must be JSON-escaped."""
+        fmt_def = load_format("ecar")
+        output_file = tmp_path / "ecar.json"
+        emitter = EcarEmitter(fmt_def, output_file, threaded=False)
+
+        event_data = {
+            'timestamp': datetime(2024, 3, 15, 10, 0, 0, tzinfo=timezone.utc),
+            'hostname': 'WKS-01',
+            'object': 'REGISTRY',
+            'action': 'MODIFY',
+            'pid': 1234,
+            'principal': 'alice.smith',
+            'registry_key': r'HKCU\Software\Test',
+            'registry_value': 'Value with "quotes"',
+        }
+        emitter.emit_event(event_data)
+        emitter.flush()
+
+        content = output_file.read_text()
+        parsed = json.loads(content.strip())
+        assert parsed['properties']['registry_value'] == 'Value with "quotes"'
+
+
 class TestEcarFlowFromConnection:
     """Test that connections emit FLOW/CONNECT eCAR events."""
 

tests/unit/test_phase5_failed_logon.py

@@ -4,6 +4,8 @@
 from datetime import datetime, timezone
 from unittest.mock import Mock
 
+from evidenceforge.formats.loader import load_format
+from evidenceforge.formats.validator import validate_event
 from evidenceforge.generation.activity import ActivityGenerator
 from evidenceforge.generation.state_manager import StateManager
 from evidenceforge.models import User, System
@@ -112,6 +114,47 @@ def test_emits_ecar_with_failure_reason(self, activity_gen, test_user, win_syste
         assert event_data['failure_reason'] == 'bad_password'
 
 
+class TestFailedLogonFormatValidation:
+    """Test that 4625 events pass format validation with all fields."""
+
+    def test_4625_with_all_fields_validates(self):
+        """A 4625 event with TransmittedServices, LmPackageName, KeyLength, ProcessId, ProcessName should validate."""
+        fmt_def = load_format("windows_event_security")
+        event = {
+            'EventID': 4625,
+            'TimeCreated': '2024-03-15T10:00:00Z',
+            'Computer': 'WKS-01.corp.local',
+            'Channel': 'Security',
+            'Level': 0,
+            'EventRecordID': 1001,
+            'ExecutionProcessID': 4,
+            'ExecutionThreadID': 64,
+            'SubjectUserSid': 'S-1-5-18',
+            'SubjectUserName': '-',
+            'SubjectDomainName': '-',
+            'SubjectLogonId': '0x0',
+            'TargetUserSid': 'S-1-0-0',
+            'TargetUserName': 'alice.smith',
+            'TargetDomainName': 'CORP',
+            'Status': '0xc000006d',
+            'SubStatus': '0xc0000064',
+            'FailureReason': '%%2313',
+            'LogonType': 3,
+            'LogonProcessName': 'NtLmSsp',
+            'AuthenticationPackageName': 'NTLM',
+            'WorkstationName': 'WKS-02',
+            'IpAddress': '10.0.10.2',
+            'IpPort': 49152,
+            'TransmittedServices': '-',
+            'LmPackageName': '-',
+            'KeyLength': 0,
+            'ProcessId': '0x0',
+            'ProcessName': '-',
+        }
+        result = validate_event(fmt_def, event, variant_name='failed_logon')
+        assert result.valid, f"Validation errors: {result.errors}"
+
+
 class TestFailedLogonRate:
     """Test that baseline activity includes ~10% failed logons."""