Store money as authoritative microdollars - #1224
Conversation
roborev: Combined Review (
|
roborev: Combined Review (
|
dcc0736 to
6e08b51
Compare
roborev: Combined Review (
|
roborev: Combined Review (
|
Floating-point dollar values currently cross storage, aggregation, synchronization, and public contracts, so the implementation needs one exact authority rather than a database-only conversion. Document the approved microdollar representation, destructive-free forward migrations, integer pricing arithmetic, machine JSON contract, and mandatory dollar-formatted human presentation before implementation begins.
The money conversion crosses parsers, three storage backends, exports, APIs, CLI rendering, and the frontend, and cannot safely use temporary float compatibility adapters. Record the test-first execution order, backend migration gates, exact public contract, and verification boundaries before changing production code.
Money needs one exact value and arithmetic boundary before storage and public contracts can stop relying on binary floating-point dollars. Add checked signed microdollar sums, exact scaled-decimal parsing, wide token-rate multiplication with row-level rounding, and integer-only dollar presentation that safely handles the full int64 range.
Floating-point dollars allowed binary rounding to leak through ingestion, storage, aggregation, exports, and APIs. Use one signed int64 microdollar authority so every machine-facing monetary value has an exact representation and checked arithmetic. SQLite and PostgreSQL migrate legacy values transactionally, while the disposable DuckDB mirror rebuilds on its schema bump. Human-facing CLI tables and frontend labels continue to render ordinary dollar amounts from the exact value.
Clean CI and release checkouts cannot reuse the locally converted pricing snapshot. Pin the artifact commit that carries the exact-money schema so restoration validates against the existing digest and builds no longer fetch the obsolete floating-point payload.
Current main expanded the OpenAPI contract while its checked-in client still reflected the earlier schema. Regenerate it alongside the money contract and adapt optional embedding-store calls so frontend type checking remains authoritative after the rebase.
The microdollar migration added provenance to session-backed daily rows but left Cursor's PostgreSQL branch one column short, making every daily usage UNION fail at runtime. Keep the branch shape and pgtest fixtures on the exact-money contract so integration CI exercises the same schema as production.
Persisted Cursor dedup keys and PostgreSQL push fingerprints are identity boundaries: changing their canonical representation can duplicate usage or continually republish otherwise unchanged sessions. Keep those boundaries stable while retaining integer-exact costs beyond floating-point precision. Legacy configuration must also fail loudly instead of silently installing zero-cost overrides, and backend migrations must agree on midpoint rounding. Advance the disposable DuckDB schema past every released shape so stale mirrors rebuild rather than masquerading as current.
e9071b0 to
93e685b
Compare
roborev: Combined Review (
|
Microdollars are the smallest representable monetary unit, so each deduplicated usage row must cross the pricing boundary independently. Aggregating tokens first could combine several sub-microdollar rows into a fabricated whole-microdollar cost that SQLite and PostgreSQL correctly omit. Keep DuckDB rows separate until exact integer pricing has produced whole microdollars, then aggregate only those representable values. Ratio-based allocation of an authoritative integer total remains separate and continues to apportion only whole microdollars.
roborev: Combined Review (
|
Read-only PostgreSQL startup must reject legacy schemas before exact-money queries reach missing columns; otherwise a database can be declared compatible and fail only under usage traffic. The pairwise response legitimately omits a per-session delta when either side has no sessions, so publish that nullability in OpenAPI and keep the generated client aligned with runtime JSON.
roborev: Combined Review (
|
Exact microdollars must fail closed when archive schemas are ambiguous or arithmetic exceeds the representable range. Reject mixed SQLite money columns before mutation, and return pricing failures through every backend instead of panicking in CLI or HTTP paths. Keep DuckDB row-level quantization without materializing the archive-sized usage stream, and derive daily session counts during that same pass. PostgreSQL compatibility now distinguishes an optional absent pricing table from inaccessible or incomplete exact-money columns.
roborev: Combined Review (
|
Independent source rows must cross the whole-microdollar boundary before any aggregation; otherwise multiple unrepresentable fractions can fabricate stored cost. Keep ratio allocation limited to apportioning an authoritative integer total. Return arithmetic overflow through backend, activity, export, and HTTP paths instead of panicking. Preserve migrated Cursor identity after legacy values are quantized, and retain token events when only their reported cost is invalid.
roborev: Combined Review (
|
Quantizing legacy Cursor costs changes the only representable value that can participate in a stable fingerprint. Rekey both SQLite and PostgreSQL from that integer value so a full-history push cannot duplicate the same event across backends. Process migration keys in bounded batches, collapse quantization collisions deterministically, and keep valid Kilo token usage when only its reported cost is malformed.
Cursor fingerprints cross SQLite and PostgreSQL, so they cannot retain nanoseconds that PostgreSQL TIMESTAMPTZ cannot represent. Canonicalize to shared microsecond precision before hashing to prevent migration-and-push duplicates.
roborev: Combined Review (
|
Artifact manifests are a persistent machine-readable boundary and must not reintroduce floating-point dollars after the exact-money conversion. Encode Money directly while the format remains unreleased. Shared PostgreSQL schemas may be upgraded by multiple clients. Serialize the PR-local money migration per database and schema, then re-inspect columns under the lock so followers observe committed names instead of replaying stale DDL.
roborev: Combined Review (
|
Manifest usage costs changed from floating-point dollars to exact integer microdollars, so keeping the same wire version would let mixed-version peers silently ignore the authoritative representation. Version artifact kinds independently so this manifest-only change does not invalidate checkpoints, segments, or metadata.\n\nThe released artifact package has no production manifest producer or parser, so no v1 cost-bearing manifests can exist to migrate. Avoid adding an unreachable float compatibility path that would restore rounding at the money boundary.
roborev: Combined Review (
|
Main migrated ParsedUsageEvent.CostUSD to Cost *money.Money (authoritative microdollars, #1224) under this branch. Keep the wire-format float math for per-model apportionment and convert at the event boundary via omnigentCost, which drops unconvertible values (negative, non-finite) so catalog pricing applies, matching the parser's fail-soft posture.
Waypoint D of the staged upstream catch-up (specs/merge-upstream-waypoints.md). Merges u/main up to 8dc6adc (173 commits) — the money-as-authoritative- microdollars rewrite (new internal/money package; all cost/rate fields become money.Money int64 microdollars) plus a kit-ui bump, embeddings, and machine breakdowns. Pricing (convert the fork's 1h cache-write work to money, contract-preserving): - export.ModelRates.CacheWrite1hPerMTok is now money.Money (still internal: not in canonical_json / EffectiveModelRate, so the export wire stays byte-identical). - catalog.ModelPricing / db.ModelPricing gain a money.Money 1h field; the model_pricing column is cache_creation_1h_microdollars_per_mtok INTEGER alongside upstream's renamed microdollar columns; threaded through the batched upsert/insert/copy SQL, GetModelPricing, loadPricingMap, modelPricingRates, fallbackRateMap, customPricing, and pricingrefresh. - Fill1hCacheRate derives 1h from 5m in money terms as exactly 8/5 (multiply-by-8 then money.Divide by 5), replacing the float 2.0/1.25 ratio. - dailyUsageAmounts and sessionRowCost compute the 5m/1h split cost via money.CostPerMillion (5 RatedTokens) and savings via SignedCostPerMillion (read + 5m + 1h deltas), propagating money errors. - Bundled snapshot re-fetched in money format (SHA-pinned artifact); 1h derived at load. usage projects CLI cost is money.Money. Subagent rollup: rollupProjectExpr re-injected into all four daily-usage templates (upstream added an s.machine column). Artifact ledger: manifestSession gains a private field to stay byte-identical with db.Session (fork's private column); canonical-manifest golden updated. Frontend: adopted upstream's kit-ui CSS-var palette + hashColor + seriesColorMap (its collision-relief supersedes the fork's 24-color golden-angle palette and disambiguateColors), keeping the fork's org-theming (setOrgTheme) on top; AttributionPanel/costs read money via .microdollars; forced a kit-ui refetch to the pinned SHA (Card/Toggle/Checkbox). Deferred (unchanged): 1h-cache parity for the PostgreSQL and DuckDB cost paths. Green: go build/vet, all money/pricing/db/export/service/snapshot/artifact tests, svelte-check, make build. (Environmental: the snapshot test needing a git `origin` remote; one flaky sync concurrency-bound test that passes on re-run.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AgentsView currently carries currency as binary floating-point dollars across ingestion, storage, aggregation, and public contracts. That makes rounding behavior depend on where conversion happens and allows storage backends or API clients to disagree about the same charge.
This change makes signed int64 microdollars the single machine representation. Public money values use semantic fields containing {"microdollars": ...}; CLI tables and UI labels continue to render ordinary dollars. SQLite and PostgreSQL convert legacy columns transactionally after validating them, while the disposable DuckDB mirror bumps its schema and rebuilds.
The deliberate tradeoff is a broad contract change: export schemas are bumped and old floating-point fields are removed instead of retained through dual reads, writes, or aliases. Reviewers should focus on integer arithmetic and rounding boundaries, migration failure behavior, and parity across the supported storage backends.
generated by a clanker