[OPIK-7859] [BE] feat: add cipx_savings_facts table for realized savings measurement - #7913
Open
andriidudar wants to merge 2 commits into
Open
[OPIK-7859] [BE] feat: add cipx_savings_facts table for realized savings measurement#7913andriidudar wants to merge 2 commits into
andriidudar wants to merge 2 commits into
Conversation
…ngs measurement The storage for measured savings: one row per (workspace, day, user, recommendation, harness, quantity, tier, causal link), written nightly by the cost API's measurement job from cipx_spend_blocks and the recommendation apply log. Characters are frozen, not dollars — but the dollar figure is computed on the day the row is written, per (model, billing tier) inside the job, so the money not spent on a day is priced at that day's rates and stays stable through later rate-table changes. A rate correction is a partition drop and a recompute from the source blocks. The three attribution axes are columns rather than something the read derives: quantity (was the spend real, or never paid), tier (are the tokens observed or modelled), and causal link (did we configure it, perform it, or merely detect it). All three are derived by the engine from the evidence that fed each row, then frozen as provenance, so the presentation reads from this table alone with no cross-store join and no recomputation. Harness is in the key because the apply log is keyed by it: the same recommendation applied for two harnesses is two applied things, and rows that shared a key would collapse to one, silently dropping one harness's savings. Versioned on last_updated_at with a now64(6) default, like every other cipx ReplacingMergeTree — a run interrupted mid-insert, or a deliberate recompute after a measurement fix, is idempotent. Keyed by workspace_id like the rest of the family, so the whole cipx set filters by one scope key. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
⏱️ pre-commit per-hook timing
⏭️ 38 skipped (no matching files changed)
|
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
Contributor
Python SDK E2E Tests Results (Python 3.10)296 tests 289 ✅ 4m 56s ⏱️ Results for commit 2461209. ♻️ This comment has been updated with latest results. |
andriidudar
marked this pull request as ready for review
August 19, 2026 16:32
Collaborator
|
No test needed here. This is a schema-only migration: it creates cipx_savings_facts and nothing else. Advisory, from the QA test radar. Nothing here blocks this PR, and anything it proposes is a draft for review. |
BorisTkachenko
requested changes
Aug 20, 2026
| last_updated_at | ||
| ) | ||
| PARTITION BY toYYYYMM(day) | ||
| ORDER BY (workspace_id, day, user_uuid, recommendation_id, harness, quantity, tier, causal_link); |
Contributor
There was a problem hiding this comment.
We should have project_id here to match opik project level concept same as in other tables. Also I don't think you need to put so many columns here, as those end up being the primary key and sorting will be done for all of them, looks like an overhead. Probably we can remove some of these: recommendation_id, harness, quantity, tier, causal_link.
Not really in the context, but the key should be unique, but shouldn't have values like quantity
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
Adds the ClickHouse table that stores measured savings: one row per (workspace, day, user, recommendation, harness, quantity, tier, causal link), written nightly by the cost API's measurement job from
cipx_spend_blocksand the recommendation apply log. Table only — no Java reads or writes it; the writer lives inai-cost-backend.harnessis in the sort key because the apply log is keyed by it: the same recommendation applied for two harnesses is two applied things, and rows sharing a key would collapse on the ReplacingMergeTree, silently dropping one harness's savings.last_updated_atwith anow64(6)default, matching every other cipx ReplacingMergeTree, so an interrupted run or a deliberate recompute is idempotent. Keyed byworkspace_idlike the rest of the family.Deploy order: this migration must be applied before the
ai-cost-backendchange that writes to the table (comet-ml/ai-cost-backend#TBD) — that writer expectsharnessin the key and does not send aversioncolumn.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
FINALto the later row, and the freshness read returns that row'slast_updated_at.ADD COLUMNwithout a default in the same statement asMODIFY ORDER BY— which is whyharnessis included now rather than later.Documentation
Column-by-column rationale is in the migration's own header comment. Measurement methodology and the calibration procedure live in
ai-cost-backend/docs/auto-memory-calibration.md.