Skip to content

feat(cost): expose priced-hours coverage and estimated queue rates - #129

Open
khluu wants to merge 1 commit into
mainfrom
task/cost-priced-coverage
Open

feat(cost): expose priced-hours coverage and estimated queue rates#129
khluu wants to merge 1 commit into
mainfrom
task/cost-priced-coverage

Conversation

@khluu

@khluu khluu commented Sep 3, 2026

Copy link
Copy Markdown
Member

What

On /cost (2026-08-20 → 2026-09-03), Total Cost showed $31,176 "Known queues only" while only ~20% of compute hours were pricedh200_35gb (23.3k h) and amd_mi300_1 (20.5k h) had no rate, making the headline total misleading.

  • API (src/app/api/cost/route.ts): adds pricedHoursShare / estimatedHoursShare (priced hours / total hours) to the response, plus an estimated flag per row in byQueue.
  • Pricing (src/lib/queue-costs.ts): adds clearly-labeled estimated: true rates with source comments for the large partner/self-hosted queues — h200_35gb (scaled from the known $0.30/hr h200_18gb slice), full H200, AMD mi250/mi300/mi355 (market rental ~$1.80/$2.50/$3.00 per GPU/hr × queue GPU count), mithril-h100-pool, b200-k8s, l4-k8s, a100_queue, gh200_queue, medium_cpu_queue_premerge (assumed r6in.4xlarge), arm64_cpu_queue_premerge (assumed same as postmerge). Queues with no defensible rate (intel-cpu, amd-cpu, arm-cpu, intel-gpu, intel-hpu, ascend, dgx-spark) stay unpriced.
  • Cost page (src/app/cost/page.tsx):
    • Total Cost card detail now reads e.g. Covers 21% of compute hours (+75% est.) (StatCard API unchanged).
    • Estimated rows in the Cost by Queue table get an est. suffix on $/hr and Est. Cost (no new colors).
    • The Unpriced Queues card is now expandable (<details>) and lists the remaining unpriced queue names.
  • New computePricingCoverage() helper in queue-costs.ts with unit tests (src/lib/queue-costs.test.ts).

Coverage on the example range goes from 20.5% priced → 20.5% priced + 75.3% estimated (95.8% covered), 4.2% still unpriced (7 small queues, now listed in the card).

Verification

  • npm test (tsx --test): 65/65 pass, including new queue-costs.test.ts (5 tests: estimated rates have sources, coverage split math, empty input, previously-unpriced big queues now priced).
  • npm run lint: clean. tsc --noEmit: clean.
  • Coverage numbers above verified by running computePricingCoverage against the live /api/cost byQueue payload.

Daily chart investigation (unverified)

The "Daily cost by queue" chart reportedly rendered empty in two headless captures despite 105 dailyCostByQueue rows with total_cost > 0. I could not reproduce in a real browser: npm run dev needs DATABRICKS_*/DATABASE_URL credentials that I don't have access to, so no confirmed bug fix here. Static review of the chart path found no defect: series keys (cost__<queue>/hours__<queue>) contain only -/_ (no Recharts path-separator characters), data flows correctly through the stacking useMemo, and ResponsiveContainer has an explicit height. Most likely the captures were taken before SWR resolved (the page early-returns "Loading cost data...") or before ResponsiveContainer/bar animation settled in the headless environment. Worth a follow-up capture with network-idle wait if it still reproduces.

The Cost page total only covered ~20% of compute hours (7 of 33 queues
priced) while presenting a single Total Cost figure. Add estimated,
clearly-sourced rates for the large partner/self-hosted queues (H200
slices, AMD MI250/MI300X/MI355X, H100/B200/L4/A100/GH200 pools), expose
pricedHoursShare/estimatedHoursShare from /api/cost, mark estimated
rows with an 'est.' suffix in the Cost by Queue table, and make the
Unpriced Queues card expandable with the remaining unpriced queue
names.
@vercel

vercel Bot commented Sep 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vllm-ci-dashboard Ready Ready Preview Sep 3, 2026 11:33am UTC

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant