Skip to content

Commit b9fcd80

Browse files
authored
Deprecate ZetaSQL (#34563)
* Deprecate ZetaSQL * Announce deprecation in Beam website, changes, Javadoc * Issue warning on SqlTransform expansion to ZetaSQL * Fix spotless * Add alternatives in deprecation statements
1 parent 2a51d36 commit b9fcd80

File tree

5 files changed

+18
-19
lines changed

5 files changed

+18
-19
lines changed

Diff for: CHANGES.md

+2
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,8 @@
8181
## Deprecations
8282

8383
* X behavior is deprecated and will be removed in X versions ([#X](https://github.com/apache/beam/issues/X)).
84+
* Beam ZetaSQL is deprecated and will be removed no earlier than Beam 2.68.0 ([#34423](https://github.com/apache/beam/issues/34423)).
85+
Users are recommended to switch to [Calcite SQL](https://beam.apache.org/documentation/dsls/sql/calcite/overview/) dialect.
8486

8587
## Bugfixes
8688

Diff for: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java

+2
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,8 @@ public ZetaSQLQueryPlanner(FrameworkConfig config) {
8989
* reflectively.
9090
*/
9191
public ZetaSQLQueryPlanner(JdbcConnection jdbcConnection, Collection<RuleSet> ruleSets) {
92+
LOG.warn(
93+
"Beam ZetaSQL has been deprecated. See https://github.com/apache/beam/issues/34423 for details.");
9294
plannerImpl =
9395
new ZetaSQLPlannerImpl(
9496
defaultConfig(jdbcConnection, modifyRuleSetsForZetaSql(ruleSets, DEFAULT_CALC)));

Diff for: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java

+7-1
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,11 @@
1616
* limitations under the License.
1717
*/
1818

19-
/** ZetaSQL Dialect package. */
19+
/**
20+
* ZetaSQL Dialect package.
21+
*
22+
* <p>
23+
*
24+
* @deprecated Use Calcite SQL dialect. Beam ZetaSQL has been deprecated.
25+
*/
2026
package org.apache.beam.sdk.extensions.sql.zetasql;

Diff for: website/www/site/content/en/documentation/dsls/sql/overview.md

+4-18
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,11 @@ bounded and unbounded `PCollections` with SQL statements. Your SQL query
2323
is translated to a `PTransform`, an encapsulated segment of a Beam pipeline.
2424
You can freely mix SQL `PTransforms` and other `PTransforms` in your pipeline.
2525

26-
Beam SQL includes the following dialects:
26+
Beam SQL uses Calcite SQL based on [Apache Calcite](https://calcite.apache.org),
27+
a dialect widespread in big data processing.
2728

28-
- [Beam Calcite SQL](https://calcite.apache.org)
29-
- [Beam ZetaSQL](https://github.com/google/zetasql)
30-
31-
Beam Calcite SQL is a variant of Apache Calcite, a dialect widespread in
32-
big data processing. Beam Calcite SQL is the default Beam SQL dialect. Beam ZetaSQL is more compatible with BigQuery, so it's especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html).
29+
<!-- TODO: Remove deprecation statement when ZetaSQL dialect is removed. -->
30+
**Note:** Beam SQL supports for [ZetaSQL dialect](/documentation/dsls/sql/zetasql/overview) has been deprecated.
3331

3432
To change dialects, pass [the dialect's full package name](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/package-summary.html) to the [`setPlannerName`](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/impl/BeamSqlPipelineOptions.html#setPlannerName-java.lang.String-) method in the [`PipelineOptions`](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) interface.
3533

@@ -50,17 +48,5 @@ The [Shell page](/documentation/dsls/sql/shell) describes how to work with the i
5048
The [Beam Calcite SQL overview](/documentation/dsls/sql/calcite/overview) summarizes Apache Calcite operators,
5149
functions, syntax, and data types supported by Beam Calcite SQL.
5250

53-
## ZetaSQL dialect
54-
For more information on the ZetaSQL features in Beam SQL, see the [Beam ZetaSQL dialect reference](/documentation/dsls/sql/zetasql/overview).
55-
56-
To switch to Beam ZetaSQL, configure the [pipeline options](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) as follows:
57-
```
58-
PipelineOptions options = ...;
59-
options
60-
.as(BeamSqlPipelineOptions.class)
61-
.setPlannerName("org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner");
62-
```
63-
Note, Use of the `ZetaSQLQueryPlanner` requires an additional dependency on `beam-sdks-java-extensions-sql-zetasql` in addition to the `beam-sdks-java-extensions-sql` package required for `CalciteQueryPlanner`.
64-
6551
## Beam SQL extensions
6652
Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects.

Diff for: website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md

+3
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ See the License for the specific language governing permissions and
1616
limitations under the License.
1717
-->
1818
# Beam ZetaSQL overview
19+
20+
**Note:** Beam ZetaSQL has been deprecated ([details](https://github.com/apache/beam/issues/34423)). Please switch to use the default [Calcite SQL](/documentation/dsls/sql/calcite/overview) dialect.
21+
1922
Beam SQL supports a variant of the [ZetaSQL](https://github.com/google/zetasql) language. ZetaSQL is similar to the language in BigQuery's SQL framework. This Beam SQL dialect is especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html).
2023

2124
Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects, including Beam ZetaSQL.

0 commit comments

Comments
 (0)