Skip to content

Deprecate ZetaSQL #34563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 10, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@
## Deprecations

* X behavior is deprecated and will be removed in X versions ([#X](https://github.com/apache/beam/issues/X)).
* Beam ZetaSQL is deprecated and will be removed no earlier than Beam 2.68.0 ([#34423](https://github.com/apache/beam/issues/34423)).

## Bugfixes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ public ZetaSQLQueryPlanner(FrameworkConfig config) {
* reflectively.
*/
public ZetaSQLQueryPlanner(JdbcConnection jdbcConnection, Collection<RuleSet> ruleSets) {
LOG.warn(
"Beam ZetaSQL has been deprecated. See https://github.com/apache/beam/issues/34423 for details.");
plannerImpl =
new ZetaSQLPlannerImpl(
defaultConfig(jdbcConnection, modifyRuleSetsForZetaSql(ruleSets, DEFAULT_CALC)));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,11 @@
* limitations under the License.
*/

/** ZetaSQL Dialect package. */
/**
* ZetaSQL Dialect package.
*
* <p>
*
* @deprecated Use Calcite SQL dialect. Beam ZetaSQL has been deprecated.
*/
package org.apache.beam.sdk.extensions.sql.zetasql;
22 changes: 4 additions & 18 deletions website/www/site/content/en/documentation/dsls/sql/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,11 @@ bounded and unbounded `PCollections` with SQL statements. Your SQL query
is translated to a `PTransform`, an encapsulated segment of a Beam pipeline.
You can freely mix SQL `PTransforms` and other `PTransforms` in your pipeline.

Beam SQL includes the following dialects:
Beam SQL uses Calcite SQL based on [Apache Calcite](https://calcite.apache.org),
a dialect widespread in big data processing.

- [Beam Calcite SQL](https://calcite.apache.org)
- [Beam ZetaSQL](https://github.com/google/zetasql)

Beam Calcite SQL is a variant of Apache Calcite, a dialect widespread in
big data processing. Beam Calcite SQL is the default Beam SQL dialect. Beam ZetaSQL is more compatible with BigQuery, so it's especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html).
<!-- TODO: Remove deprecation statement when ZetaSQL dialect is removed. -->
**Note:** Beam SQL supports for [ZetaSQL dialect]((/documentation/dsls/sql/zetasql/overview) has been deprecated.

To change dialects, pass [the dialect's full package name](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/package-summary.html) to the [`setPlannerName`](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/impl/BeamSqlPipelineOptions.html#setPlannerName-java.lang.String-) method in the [`PipelineOptions`](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) interface.

Expand All @@ -50,17 +48,5 @@ The [Shell page](/documentation/dsls/sql/shell) describes how to work with the i
The [Beam Calcite SQL overview](/documentation/dsls/sql/calcite/overview) summarizes Apache Calcite operators,
functions, syntax, and data types supported by Beam Calcite SQL.

## ZetaSQL dialect
For more information on the ZetaSQL features in Beam SQL, see the [Beam ZetaSQL dialect reference](/documentation/dsls/sql/zetasql/overview).

To switch to Beam ZetaSQL, configure the [pipeline options](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) as follows:
```
PipelineOptions options = ...;
options
.as(BeamSqlPipelineOptions.class)
.setPlannerName("org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner");
```
Note, Use of the `ZetaSQLQueryPlanner` requires an additional dependency on `beam-sdks-java-extensions-sql-zetasql` in addition to the `beam-sdks-java-extensions-sql` package required for `CalciteQueryPlanner`.

## Beam SQL extensions
Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects.
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
# Beam ZetaSQL overview

**Note:** Beam ZetaSQL has been deprecated ([details](https://github.com/apache/beam/issues/34423)). Please switch to use the default Calcite SQL dialect.

Beam SQL supports a variant of the [ZetaSQL](https://github.com/google/zetasql) language. ZetaSQL is similar to the language in BigQuery's SQL framework. This Beam SQL dialect is especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html).

Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects, including Beam ZetaSQL.
Expand Down
Loading