Restructure output into unified scenario directories

Each scenario now lives in one directory with all related files:

  scenarios/<name>/
    scenario.yaml       # scenario definition
    ENVIRONMENT.md      # student context (from /eforge scenario)
    GROUND_TRUTH.md     # answer key (from eforge generate)
    data/               # generated log files
      windows_event_security.xml
      zeek_conn.json
      ...

Key changes:
- CLI generate derives data/ path from scenario file location
- Re-generation clears and overwrites data/ (no timestamped dirs)
- GROUND_TRUTH.md written to scenario root, not inside data/
- Engine accepts separate ground_truth_dir parameter
- --output flag still works as full override for backward compat
- All 4 skills updated for new path conventions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: DavidJBianco

commands/eforge/evaluate.md

@@ -27,24 +27,34 @@ If they don't have generated output yet, suggest using `/eforge generate` first.
 
 ### Step 1: Locate the Output
 
-The user needs to provide:
-1. **Output directory** — the directory containing generated log files (e.g., `output/retail-store-ftp-attack-20260316-140908/`)
-2. **Scenario file** — the YAML scenario used for generation
+The user needs to provide (or you can infer) the scenario directory. The standard layout is:
 
-If the user doesn't specify, look for the most recent output directory under `output/` or wherever they typically generate. Ask if you can't find it.
+```
+scenarios/<scenario-name>/
+  scenario.yaml
+  ENVIRONMENT.md
+  GROUND_TRUTH.md
+  data/              ← this is the output_dir for eforge eval
+```
+
+If the user provides the scenario directory (e.g., `scenarios/retail-store-ftp-attack/`), derive:
+- Data directory: `scenarios/<name>/data/`
+- Scenario file: `scenarios/<name>/scenario.yaml`
+
+If they don't specify, look for scenario directories under `scenarios/`. Ask if you can't find it.
 
 ### Step 2: Run the Evaluation
 
 Run both text and JSON output:
 
 ```bash
-uv run eforge eval <output_dir> --scenario <scenario.yaml> --verbose
+uv run eforge eval scenarios/<name>/data/ --scenario scenarios/<name>/scenario.yaml --verbose
 ```
 
 Also capture the JSON for programmatic analysis:
 
 ```bash
-uv run eforge eval <output_dir> --scenario <scenario.yaml> --format json 2>/dev/null
+uv run eforge eval scenarios/<name>/data/ --scenario scenarios/<name>/scenario.yaml --format json 2>/dev/null
 ```
 
 ### Step 3: Interpret Results

commands/eforge/generate.md

@@ -76,15 +76,28 @@ cd /Users/dabianco/projects/SURGe/data-gen-test
 uv run eforge generate <scenario-file> --verbose
 ```
 
-Always use `--verbose` so you can see progress and diagnose issues. Generation creates a timestamped output directory like `output/scenario-name-20240115-100000/`.
+Always use `--verbose` so you can see progress and diagnose issues. Generation writes log files to a `data/` subdirectory alongside the scenario file:
+
+```
+scenarios/<scenario-name>/
+  scenario.yaml          ← input
+  ENVIRONMENT.md         ← created by /eforge scenario
+  GROUND_TRUTH.md        ← generated (answer key)
+  data/                  ← generated log files
+    windows_event_security.xml
+    zeek_conn.json
+    ...
+```
+
+Re-running generation overwrites the previous `data/` directory.
 
 ### 3. Post-Generation
 
 After successful generation:
 - List the generated files and their sizes
 - Check that expected formats were produced
-- If the scenario had a storyline, note that `GROUND_TRUTH.md` was generated in the output directory — this is the answer key containing the full attack timeline and IOCs
-- If an `ENVIRONMENT.md` exists alongside the scenario file (created by `/eforge scenario`), copy it into the output directory so it sits alongside the generated logs and GROUND_TRUTH.md
+- If the scenario had a storyline, note that `GROUND_TRUTH.md` was generated alongside the scenario file — this is the answer key containing the full attack timeline and IOCs
+- `ENVIRONMENT.md` (created by `/eforge scenario`) is already in the same directory — no copying needed
 - Summarize the output for the user
 
 ### 4. Diagnose Errors

commands/eforge/scenario.md

@@ -411,8 +411,8 @@ After generating the scenario YAML, also create an `ENVIRONMENT.md` file in the
 
 After the interview, generate both files:
 
-1. **Scenario YAML** — Write to the user's chosen path (default: `scenarios/<scenario-name>.yaml`)
-2. **ENVIRONMENT.md** — Write alongside the scenario YAML
+1. **Scenario YAML** — Write to the user's chosen path (default: `scenarios/<scenario-name>/scenario.yaml`)
+2. **ENVIRONMENT.md** — Write alongside the scenario YAML (default: `scenarios/<scenario-name>/ENVIRONMENT.md`)
 3. **Realism Review** — Before validating, review the entire scenario as a tough-but-fair devil's advocate. Check:
    - **Attack realism**: Does the attack chain make sense? Would a real attacker do this in this order? Are there missing steps (e.g., no reconnaissance before lateral movement, no persistence after initial access)?
    - **Technical accuracy**: Are command lines correct for the target OS? Are process paths right? Do the MITRE ATT&CK technique IDs match what's actually happening?

src/evidenceforge/cli/commands.py

@@ -259,18 +259,24 @@ def generate(
 
     # Determine output directory
     if output:
-        output_dir = output
-    elif scenario.output.destination:
-        output_dir = Path(scenario.output.destination)
+        # Explicit --output flag: use as data directory directly
+        data_dir = output
+        ground_truth_dir = output
     else:
-        output_dir = Path("./output")
+        # Default: derive from scenario file location
+        # scenarios/<name>/scenario.yaml → data goes to scenarios/<name>/data/
+        scenario_dir = scenario_file.parent
+        data_dir = scenario_dir / "data"
+        ground_truth_dir = scenario_dir
 
-    # Create timestamped subdirectory
-    from datetime import datetime
-    timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
-    output_dir = output_dir / f"{scenario.name}-{timestamp}"
+    console.print(f"\n[bold]Data directory:[/bold] {data_dir}")
+    console.print(f"[bold]Ground truth:[/bold] {ground_truth_dir / 'GROUND_TRUTH.md'}")
 
-    console.print(f"\n[bold]Output directory:[/bold] {output_dir}")
+    # Clear previous data on re-generation
+    if data_dir.exists():
+        import shutil
+        shutil.rmtree(data_dir)
+        console.print("[dim]Cleared previous data[/dim]")
 
     # Generate logs
     try:
@@ -333,23 +339,33 @@ def progress_callback(event_type: str, data: dict) -> None:
             # Generate logs with progress reporting
             engine = GenerationEngine(
                 scenario=scenario,
-                output_dir=output_dir,
-                progress_callback=progress_callback
+                output_dir=data_dir,
+                progress_callback=progress_callback,
+                ground_truth_dir=ground_truth_dir,
             )
             engine.generate()
 
         console.print("\n[bold green]✓ Generation complete![/bold green]")
-        console.print(f"\nGenerated logs:")
-        console.print(f"  Directory: {output_dir}")
+        console.print(f"\nGenerated files:")
+        console.print(f"  Scenario directory: {ground_truth_dir}")
 
-        # List generated files
-        if output_dir.exists():
-            for file in sorted(output_dir.iterdir()):
-                if file.is_file():
+        # List files in scenario root (GROUND_TRUTH.md)
+        if ground_truth_dir.exists():
+            for file in sorted(ground_truth_dir.iterdir()):
+                if file.is_file() and file.name == "GROUND_TRUTH.md":
                     size = file.stat().st_size
                     size_str = f"{size:,} bytes" if size < 1024 else f"{size / 1024:.1f} KB"
                     console.print(f"  • {file.name} ({size_str})")
 
+        # List generated log files in data/
+        if data_dir.exists():
+            console.print(f"  Data: {data_dir}")
+            for file in sorted(data_dir.iterdir()):
+                if file.is_file():
+                    size = file.stat().st_size
+                    size_str = f"{size:,} bytes" if size < 1024 else f"{size / 1024:.1f} KB"
+                    console.print(f"    • {file.name} ({size_str})")
+
         # Success - exit normally
         return
 

src/evidenceforge/generation/engine.py

@@ -59,18 +59,21 @@ def __init__(
         self,
         scenario: Scenario,
         output_dir: Path,
-        progress_callback: Optional[Callable[[str, dict], None]] = None
+        progress_callback: Optional[Callable[[str, dict], None]] = None,
+        ground_truth_dir: Optional[Path] = None,
     ):
         """Initialize generation engine.
 
         Args:
             scenario: Validated scenario object
-            output_dir: Output directory path
+            output_dir: Output directory path for generated log files
             progress_callback: Optional callback for progress reporting.
                 Called with (event_type: str, data: dict) at key milestones.
+            ground_truth_dir: Directory for GROUND_TRUTH.md. Defaults to output_dir.
         """
         self.scenario = scenario
         self.output_dir = output_dir
+        self.ground_truth_dir = ground_truth_dir or output_dir
         self.progress_callback = progress_callback
         self.state_manager = StateManager()
         self.emitters: dict[str, WindowsEventEmitter | ZeekEmitter | EcarEmitter | SyslogEmitter | BashHistoryEmitter | SnortEmitter | WebEmitter] = {}
@@ -724,7 +727,8 @@ def _generate_ground_truth(self) -> None:
         Creates comprehensive attack documentation including narrative,
         timeline, and IOCs for threat hunting training.
         """
-        output_path = self.output_dir / "GROUND_TRUTH.md"
+        self.ground_truth_dir.mkdir(parents=True, exist_ok=True)
+        output_path = self.ground_truth_dir / "GROUND_TRUTH.md"
 
         generator = GroundTruthGenerator(
             scenario=self.scenario,