status-page: target finelog's DataFusion SQL dialect, not DuckDB#6569
Merged
Conversation
Live verification against the prod finelog showed its read engine is
Apache DataFusion (53), not DuckDB: `json_extract_string` and `epoch_ms`
don't exist there (it registers only prefix/regexp_matches/contains
UDFs, no JSON functions).
- drop json_extract_string from the checks query: select the raw labels
blob and decode the `probe` name in JS (same as the provisioning path),
partitioning the window by labels (one `{"probe":...}` value per check)
- read collected_at as epoch millis via arrow_cast(...,'Int64') instead
of epoch_ms(), and drop the redundant value::DOUBLE cast (already f64)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08096a27de
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| AND collected_at >= TIMESTAMP '${cutoff}' | ||
| ) | ||
| SELECT probe, metric, value::DOUBLE AS value, epoch_ms(collected_at)::BIGINT AS collected_ms | ||
| SELECT labels, metric, value, arrow_cast(collected_at, 'Int64') AS collected_ms |
There was a problem hiding this comment.
When this query runs against finelog, collected_at is stored as Arrow Timestamp(Microsecond, None) (lib/finelog/rust/src/store/schema.rs documents the microsecond storage unit), so arrow_cast(collected_at, 'Int64') yields microseconds, not epoch milliseconds. The parsed rows are later passed directly to new Date(...), so every probe/provisioning freshness timestamp will be about 1000x too large and render as a far-future/negative-relative time; the same cast in provisioningSql has the same effect.
Useful? React with 👍 / 👎.
Replace the in-process worker-count ring buffer (lost on every Cloud Run restart) with durable 24h history read from the infra/probes canary's infra.canary.metrics finelog namespace, and add a provisioning success-ratio history chart beside it. - New server/sources/clusterHistory.ts: two bounded DataFusion queries over the trailing 24h — per-region worker_healthy, and fleet + per-pool provision_ready/outcomes rolled up into a fleet-average plus per-region create-success ratio (zones collapse to regions). - /api/workers/history now reads finelog; add /api/provisioning/history. Drop the WorkerHistory ring buffer and its 30s sampler; the iris-ping and control-plane latency buffers stay in process. - Workers panel shows the availability and provisioning charts side by side, sharing one region color map (built off the union of both charts' regions) so a region reads the same color in both. - Fix the collected_at decode: arrow_cast(...,'Int64') yields epoch microseconds, not millis — probe timestamps were rendering ~58000 years in the future. Shared microsToMillis + decodeLabels helpers in finelogQuery.ts.
Address the lint-catalog review of the prior commit: probes.ts and clusterHistory.ts had each re-derived the canary-namespace query scaffold. - Lift the shared bits into finelogQuery.ts: CANARY_METRICS_NAMESPACE, FLEET_SCOPE, the CanaryMetricRow row type, and the asCanaryRows micros-normalizer. Both sources import them now. - Rename the probes.ts SQL timestamp alias collected_ms -> collected_us; the cast yields epoch micros and is only normalized to millis by asCanaryRows, so the old name was misleading and inconsistent with clusterHistory.ts. - Collapse workersHistory/provisioningHistory's identical fetchedAt+cutoff+try/catch bodies into one historyResponse(buildSql, parse) helper. No behavior change; /api/workers/history, /api/provisioning/history, and /api/probes verified to return identical shapes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QueryRPC runs Apache DataFusion 531; verified live against prod finelog, where the first query failed withInvalid function 'json_extract_string'json_extract_stringfrom the checks query: select the rawlabelsblob and decode theprobename app-side (same as the provisioning path), partitioning the window bylabels(each health check emits exactly one{"probe": "<name>"}value)collected_atas epoch millis viaarrow_cast(collected_at, 'Int64')instead ofepoch_ms(), and drop the redundantvalue::DOUBLEcast (already f64)🤖 Generated with Claude Code
Footnotes
finelog registers only
prefix/regexp_matches/containsUDFs (lib/finelog/rust/src/query/udf.rs) — no JSON functions and noepoch_ms. typecheck/lint/build green; mock parse re-verified. the SQL dialect is now confirmed against the live server; remaining check is the parse against realinfra.canary.metricsdata, done locally. ↩