Merge pull request #7 from trouze/feat/demo-fixture

demo: add demo_project fixture + scripts/demo.sh walkthrough

Author: trouze

CHANGELOG.md

@@ -8,6 +8,7 @@ All notable changes to this project will be documented in this file. The format
 
 - `replay` cost overlay: pass `--warehouse-size` (Snowflake XS…6XL) or `--credits-per-hour` (non-Snowflake adapters) to translate wall-clock into dollars. Renders **Run cost**, **Critical-path floor**, **Headroom** (= run − floor; the prize for better parallelization), and **Idle cost** (the $ equivalent of thread-idle warehouse-seconds). Defaults to $2.00/credit (Snowflake Standard On-Demand); override with `--rate-per-credit`. Snowflake's 60-second minimum-billing floor is applied automatically; pass `--no-minimum-billing` to see raw wall-clock × rate.
 - New module `dbt_dag_opt.cost` with `CostInputs`, `CostReport`, `compute_cost()`, `credits_per_hour_for()`, and `cost_inputs_from_replay()`. Designed primitive-first so a future `whatif` simulator can call `compute_cost` against simulated schedules and diff the resulting `CostReport`s.
+- `scripts/demo.sh` + `tests/fixtures/demo_project/` — narrated end-to-end demo script driving every subcommand against a synthetic 24-model DAG with a shared bottleneck, 4 threads, and ~7.5-min wall-clock. Fixture is regenerable via `tests/fixtures/generate_demo_fixture.py`.
 
 ## [0.1.0] - 2026-04-24
 

README.md

@@ -139,6 +139,16 @@ It **is** a CLI tool that points at the slowest chains in your DAG, reconstructs
 It **isn't** (yet):
 - A predictive scheduler simulator. `replay` reconstructs what already happened; it doesn't yet project what would happen under a different `--threads N` or if you sped up a specific model. That "what-if" loop is planned next, and will diff two cost reports to show projected $ savings.
 
+## Demo
+
+An end-to-end walkthrough you can record or run locally:
+
+```bash
+./scripts/demo.sh
+```
+
+Drives every subcommand (`analyze`, `analyze --show-path`, `replay`, `replay --warehouse-size L/XL`, `--credits-per-hour` for non-Snowflake, JSON + `jq`) against a synthetic 24-model baseball-analytics DAG in `tests/fixtures/demo_project/`. Set `PAUSE=0` to dry-run without narration beats.
+
 ## Development
 
 ```bash
@@ -148,6 +158,12 @@ uv run mypy src
 uv run pytest
 ```
 
+Regenerate the demo fixture after editing its topology:
+
+```bash
+uv run python tests/fixtures/generate_demo_fixture.py
+```
+
 ## License
 
 Apache 2.0 — see [LICENSE](LICENSE).

scripts/demo.sh

@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+# Demo script for dbt-dag-opt. Designed to be recorded (asciinema / QuickTime).
+#
+# Runs against a synthetic 24-model dbt project under tests/fixtures/demo_project/
+# — a baseball analytics warehouse with 4 threads, ~7.5 min wall-clock, and one
+# shared bottleneck (int_game_events) sitting on three of the top longest paths.
+#
+# Usage:
+#   ./scripts/demo.sh
+#
+# Each command is echoed in bold before it runs, with a narration hint above.
+# Pause between commands by setting PAUSE=2 (default) or call with PAUSE=0 to
+# rush through for a dry-run.
+
+set -euo pipefail
+
+PAUSE="${PAUSE:-2}"
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+MANIFEST="$ROOT/tests/fixtures/demo_project/manifest.json"
+RUN_RESULTS="$ROOT/tests/fixtures/demo_project/run_results.json"
+
+bold() { printf "\033[1m%s\033[0m\n" "$*"; }
+dim() { printf "\033[2m%s\033[0m\n" "$*"; }
+section() {
+    echo
+    printf "\033[1;36m▌ %s\033[0m\n" "$*"
+    echo
+}
+run() {
+    bold "\$ $*"
+    sleep "$PAUSE"
+    eval "$@"
+    echo
+    sleep "$PAUSE"
+}
+
+section "1 · Which paths through the DAG are actually slow?"
+dim "analyze uses manifest + run_results to compute the critical path — the"
+dim "longest cumulative chain of model execution times. That's the bound on"
+dim "how fast your pipeline could possibly run."
+run "uv run dbt-dag-opt analyze --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --top 5"
+
+section "2 · The Bottleneck column names the slowest model on each path"
+dim "Watch for a model that appears as the bottleneck on MULTIPLE rows — that's"
+dim "shared-node leverage. Optimizing it speeds up several paths at once."
+
+section "3 · Drill into the full chain with --show-path"
+run "uv run dbt-dag-opt analyze --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --top 3 --show-path"
+
+section "4 · What actually happened? (replay reconstructs the observed schedule)"
+dim "replay reads thread_id + timing from run_results to reconstruct the"
+dim "per-thread Gantt, identify the observed critical path, and attribute"
+dim "every idle gap to the upstream model a thread was waiting on."
+run "uv run dbt-dag-opt replay --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --top-idle-gaps 5"
+
+section "5 · Put a price on it: --warehouse-size translates wall-clock to dollars"
+dim "Four framed numbers:"
+dim "  • Run cost         — what this run billed"
+dim "  • Critical-path floor — the irreducible cost of your slowest chain"
+dim "  • Headroom         — run − floor; prize for better parallelization"
+dim "  • Idle cost        — \$ equivalent of thread-idle warehouse-seconds"
+run "uv run dbt-dag-opt replay --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --warehouse-size L --top-idle-gaps 3"
+
+section "6 · Change the warehouse, change the bill (same run, XL)"
+dim "Doubling warehouse size doubles the rate. Same wall-clock, 2x cost."
+run "uv run dbt-dag-opt replay --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --warehouse-size XL --top-idle-gaps 0"
+
+section "7 · Non-Snowflake adapters: pass --credits-per-hour directly"
+dim "Databricks, BigQuery, Redshift — pass the cost/hour your adapter charges."
+run "uv run dbt-dag-opt replay --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --credits-per-hour 12 --rate-per-credit 1.5 --top-idle-gaps 0"
+
+section "8 · Machine-readable: --format json"
+dim "Everything in the text output is also in JSON — pipe to jq for dashboards,"
+dim "Slack alerts, or CI annotations."
+run "uv run dbt-dag-opt replay --manifest \"$MANIFEST\" --run-results \"$RUN_RESULTS\" --warehouse-size L --format json | jq '.cost'"
+
+section "Wrap"
+dim "Three takeaways from this run:"
+dim "  1. int_game_events is the shared bottleneck on 3 of the top 5 paths."
+dim "  2. 5% of the bill is pure parallelism headroom (small — DAG is well-shaped)."
+dim "  3. 30% of warehouse-seconds are idle threads — you're overprovisioned"
+dim "     on thread count for this DAG shape. Consider --threads 2 next run."
+echo
+bold "pip install dbt-dag-opt"
+echo

tests/conftest.py

@@ -10,6 +10,7 @@
 
 FIXTURES_DIR = Path(__file__).parent / "fixtures"
 DBT_DUGOUT_DIR = FIXTURES_DIR / "dbt_dugout"
+DEMO_PROJECT_DIR = FIXTURES_DIR / "demo_project"
 
 
 @pytest.fixture
@@ -52,6 +53,27 @@ def dbt_dugout_artifacts(
     return DagArtifacts(manifest=manifest, run_results=run_results)
 
 
+@pytest.fixture
+def demo_project_manifest_path() -> Path:
+    return DEMO_PROJECT_DIR / "manifest.json"
+
+
+@pytest.fixture
+def demo_project_run_results_path() -> Path:
+    return DEMO_PROJECT_DIR / "run_results.json"
+
+
+@pytest.fixture
+def demo_project_artifacts(
+    demo_project_manifest_path: Path, demo_project_run_results_path: Path
+) -> DagArtifacts:
+    with demo_project_manifest_path.open() as fh:
+        manifest = json.load(fh)
+    with demo_project_run_results_path.open() as fh:
+        run_results = json.load(fh)
+    return DagArtifacts(manifest=manifest, run_results=run_results)
+
+
 def _phase(started: str, completed: str, name: str = "execute") -> dict[str, str]:
     return {"name": name, "started_at": started, "completed_at": completed}