feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (#7987)#8392
feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (#7987)#8392onepizzateam wants to merge 10 commits into
Conversation
…US (fixes GreptimeTeam#7987) Signed-off-by: onepizzateam <palakjha916@gmail.com>
…US (fixes GreptimeTeam#7987) Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: Palak Jha <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new system schema table, information_schema.flow_statistics, along with the SHOW FLOW STATUS SQL command to expose runtime statistics for flows, such as flow ID, name, start time, last execution time, uptime, and state size. Both streaming and batching flow engines have been updated to track and report these start times. The review feedback highlights two key areas for improvement: first, optimizing the uptime calculation in flow_statistics.rs by fetching the current time once outside the loop and handling potential clock skew defensively; second, reducing inter-thread communication overhead in stat.rs by combining multiple sequential worker queries into a single request and executing them concurrently.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…mp uptime Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
000a50d to
13bf956
Compare
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
Closes #7987
What's changed and what's your intention?
Implements the flow runtime observability design approved in #7987: a system table
information_schema.flow_statisticsplus aSHOW FLOW STATUSSQL command, so userscan inspect per-flow runtime status directly through SQL.
Summary of the change
information_schema.flow_statisticssystem table (table id 44) exposingper-flow runtime stats with columns:
flow_id,flow_name,start_time,last_execution_time,uptime_seconds,state_size.SHOW FLOW STATUS [LIKE ...]statement (parser + AST + Display, operatordispatch, frontend permission handling) as a user-friendly view over the same table.
start_timein both the streaming and batchingengines and surface it through the existing
FlowStatpipeline (extendsFlowStat/FlowStateValuewith astart_time_map).flow/flow_status) and regenerate the affected goldenresults (
information_schema,show_databases_tables,view/create).How it works
SHOW FLOW STATUSis resolved to a projection overinformation_schema.flow_statistics.The table builder joins flow metadata (
flow_name_manager) with runtime stats obtainedvia
information_extension.flow_stats().uptime_secondsis derived as(now - start_time) / 1000.Limitations
flow_id,flow_name,last_execution_time, andstate_sizereturn real data inboth standalone and distributed modes.
start_time/uptime_secondsare populated in standalone mode only. In distributedmode they return NULL because the cross-node heartbeat wire type
(
api::v1::meta::FlowStat) does not yet carry start-time information — consistent withthe proposal's "handle missing values as NULL". Propagating it through the heartbeat is
a documented follow-up (
TODO(#7987-followup)).processed_rowsandlast_errorsfrom the proposal are intentionally deferred to afollow-up PR (incremental delivery per the issue).
API / data compatibility
SHOW FLOW STATUSand the new table are additive.FlowStateValue.start_time_mapis annotated#[serde(default)], so existing persistedflow-state metadata deserializes unchanged.
Docs
SHOW FLOW STATUSand theinformation_schema.flow_statisticstable) that need documentation on the GreptimeDBdocs site. I'll open a follow-up docs PR against the docs repo to cover the new command
and table schema.
PR Checklist
Please convert it to a draft if some of the following conditions are not met.