[PYF-260] Add flink statement result list command and display results on create --wait#3261
[PYF-260] Add flink statement result list command and display results on create --wait#3261Zander Matheson (awmatheson) wants to merge 1 commit intoconfluentinc:mainfrom
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
Zander Matheson (awmatheson)
force-pushed
the
PYF-260
branch
from
963aca0 to
129c2f4
Compare
… on create --wait
Zander Matheson (awmatheson)
force-pushed
the
PYF-260
branch
from
129c2f4 to
4a3a6bc
Compare
| traits := statement.Status.GetTraits() | ||
| schema := traits.GetSchema() | ||
| if columns := schema.GetColumns(); len(columns) > 0 && traits.GetIsBounded() { | ||
| statementResults, err := fetchAllResults(client, environmentId, name, c.Context.LastOrgId, schema, defaultMaxResultRows) |
There was a problem hiding this comment.
Looks like there's no --max-rows support on create --wait
The logic hardcodes defaultMaxResultRows (100) and doesn't print a truncation warning. If a query returns >100 rows, the output is silently truncated. Compare with result list which both allows --max-rows and prints a truncation warning to stderr. Could we either:
- Add --max-rows flag, or
- Print truncation warning?
| return fmt.Errorf("statement %q is still pending, use --wait to wait for it to complete", args[0]) | ||
| } | ||
|
|
||
| err = retry.Retry(time.Second, 2*time.Minute, func() error { |
There was a problem hiding this comment.
Is this intentional that the timeout is 2x longer for result list compared to create? Otherwise it looks like an inconsistency.
| schema := traits.GetSchema() | ||
| columns := schema.GetColumns() | ||
| if len(columns) == 0 { | ||
| fmt.Fprintln(cmd.OutOrStdout(), "Statement has no results to display.") |
There was a problem hiding this comment.
When a DDL statement has no schema (empty columns), the code prints a plain text message regardless of --output flag.
So user would get plain text instead of a structured JSON response if running flink statement result list no-results -o json. Contrast this with the printStatementResults function which correctly handles the serialized case for empty rows, we should be able to handle serialized output here as well.
| maxRows, err := cmd.Flags().GetInt("max-rows") | ||
| if err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
Could we add a check to make sure max-rows is positive value?
if maxRows < 1 {
return fmt.Errorf("--max-rows must be a positive integer")
}
2 participants
Summary
confluent flink statement result list <statement-name>command that fetches and displays SQL statement result rows from the Flink Gateway API--output human/json/yaml,--wait, and--max-rowsflagsconfluent flink statement create --waitto automatically fetch and display results for bounded queries (e.g.SHOW DATABASES), falling through to existing metadata output for DDL statements and unbounded queriesTest plan
go build ./...compiles successfullyTestFlinkStatementResultListintegration tests pass (human, JSON, YAML, failed, pending, no-results, unbounded)TestFlinkStatementCreatetests pass (including--wait)TestFlinkStatementtests pass (2 pre-existing failures unrelated to this change)SHOW DATABASES,SHOW TABLES, DDL, JSON/YAML output,--max-rowstruncationTest & Review
Integration tests cover 7 scenarios:
--waitManual testing performed against live Confluent Cloud with
SHOW DATABASES,SHOW TABLES,CREATE TABLE(DDL), and streamingSELECTqueries.Checklist
Whatsection below whether this PR applies to Confluent Cloud, Confluent Platform, or both.Test & Reviewsection below.Blast Radiussection below.What
Adds non-interactive result fetching for bounded Flink SQL statements. This enables scripting and automation use cases like listing catalogs, databases, and tables via Flink SQL from the command line, which were previously only possible through the interactive Flink shell (REPL).
Key design decisions:
traits.GetIsBounded()from the Flink Gateway API to distinguish bounded (snapshot) queries from unbounded (streaming) queries--max-rows(default 100)result listreturns an error;create --waitfalls through to metadata outputresultsub-command registered only for cloud login (on-prem can be added later with a separate child command)Blast Radius
flink statement result list) with no changes to existing command behaviorflink statement create --wait, which now additionally displays results for bounded queries instead of only showing metadata. Unbounded queries and DDL statements behave exactly as beforeReferences