replay: Snowflake cost overlay (--warehouse-size)#6
Merged
Conversation
Translates wall-clock + thread-idleness + critical-path-seconds into dollars so users can answer "what did this run cost, how much was waste, and what's the floor I'm working against?" - New dbt_dag_opt.cost module: CostInputs / CostReport dataclasses, compute_cost(), credits_per_hour_for() with Snowflake size aliases (XS…6XL, long-form and punctuation tolerant), and cost_inputs_from_replay() adapter. Primitive-driven so a future whatif simulator can call compute_cost() against simulated schedules and diff two CostReports. - replay CLI flags: --warehouse-size, --credits-per-hour (non- Snowflake escape hatch, mutually exclusive with size), --rate-per-credit (default $2.00, Standard On-Demand), and --no-minimum-billing (opt out of Snowflake's 60s floor, applied by default). - render_replay() accepts optional cost and adds a "Cost estimate" table (text) or top-level "cost" key (JSON). Four framed numbers: Run cost, Critical-path floor, Headroom, Idle cost. - 7 new tests (53 total). ruff + mypy --strict clean. Does not bump __version__; ships alongside whatif in the eventual v0.2.0 release. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI (Linux, color-on) rendered the BadParameter error with inline ANSI escapes that split "--warehouse-size" across style boundaries, so the plain substring check failed. Local macOS runs didn't reproduce because rich skipped color (no TTY). Fix: strip ANSI before searching. Also moved _ANSI_RE below imports (ruff E402) and added .actrc so contributors can reproduce CI locally via `act -j test`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a cost overlay to
replaythat translates the observed schedule into dollars. Lands onmainunder[Unreleased]; eventual v0.2.0 will bundle this with thewhatifsimulator (no intermediate tag).Four numbers frame the new output:
run − floor; what better parallelization could save.$equivalent of thread-idle warehouse-seconds (distinct from headroom — idleness includes long-tail critical path that can't be parallelized away).Design
src/dbt_dag_opt/cost.py: pure functions + frozen dataclasses (CostInputs,CostReport).compute_cost()takes primitives, not aReplayReport, so the futurewhatifsimulator can call it against simulated schedules and diff twoCostReports. A thincost_inputs_from_replay()adapter keeps the CLI ergonomic.ReplayReportstays cost-free — the CLI composes replay + cost and passes both intorender_replay(report, fmt, cost=...). Keepsreplay.pyandcost.pyindependently testable.--no-minimum-billing.--warehouse-sizeand--credits-per-hour(non-Snowflake escape hatch) are mutually exclusive.--rate-per-creditand--no-minimum-billingrequire one of them.New flags
Sample output (dbt_dugout fixture,
--warehouse-size L)The sub-second fixture run ends up floored at 60s — a real Snowflake-billed signal ("at this runtime you're paying the billing minimum; no amount of parallelization matters unless you run longer"). Idle cost stays visible because
waste_fractionis computed on raw wall-clock × thread_count.Test plan
tests/test_cost.py(53 total; all pass).ruff check .clean.mypy --strict srcclean.--warehouse-size Lrenders Cost estimate; JSONcostkey has all 12 fields.--rate-per-creditwithout size → BadParameter, exit 2.--credits-per-hour 12 --no-minimum-billingrenderscustom (12 credits/hr)row with raw wall-clock.🤖 Generated with Claude Code