diff --git a/CHANGES.md b/CHANGES.md index 6f1a602ad1e0..168ed296de30 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -77,6 +77,8 @@ ## Deprecations * X behavior is deprecated and will be removed in X versions ([#X](https://github.com/apache/beam/issues/X)). +* Beam ZetaSQL is deprecated and will be removed no earlier than Beam 2.68.0 ([#34423](https://github.com/apache/beam/issues/34423)). + Users are recommended to switch to [Calcite SQL](https://beam.apache.org/documentation/dsls/sql/calcite/overview/) dialect. ## Bugfixes diff --git a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java index df14cfa73fb4..8210afed8119 100644 --- a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java +++ b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSQLQueryPlanner.java @@ -89,6 +89,8 @@ public ZetaSQLQueryPlanner(FrameworkConfig config) { * reflectively. */ public ZetaSQLQueryPlanner(JdbcConnection jdbcConnection, Collection ruleSets) { + LOG.warn( + "Beam ZetaSQL has been deprecated. See https://github.com/apache/beam/issues/34423 for details."); plannerImpl = new ZetaSQLPlannerImpl( defaultConfig(jdbcConnection, modifyRuleSetsForZetaSql(ruleSets, DEFAULT_CALC))); diff --git a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java index 51f434853f25..0400e76ca028 100644 --- a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java +++ b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/package-info.java @@ -16,5 +16,11 @@ * limitations under the License. */ -/** ZetaSQL Dialect package. */ +/** + * ZetaSQL Dialect package. + * + *

+ * + * @deprecated Use Calcite SQL dialect. Beam ZetaSQL has been deprecated. + */ package org.apache.beam.sdk.extensions.sql.zetasql; diff --git a/website/www/site/content/en/documentation/dsls/sql/overview.md b/website/www/site/content/en/documentation/dsls/sql/overview.md index 0b0c03ad6c51..7aa9d7bab298 100644 --- a/website/www/site/content/en/documentation/dsls/sql/overview.md +++ b/website/www/site/content/en/documentation/dsls/sql/overview.md @@ -23,13 +23,11 @@ bounded and unbounded `PCollections` with SQL statements. Your SQL query is translated to a `PTransform`, an encapsulated segment of a Beam pipeline. You can freely mix SQL `PTransforms` and other `PTransforms` in your pipeline. -Beam SQL includes the following dialects: +Beam SQL uses Calcite SQL based on [Apache Calcite](https://calcite.apache.org), +a dialect widespread in big data processing. -- [Beam Calcite SQL](https://calcite.apache.org) -- [Beam ZetaSQL](https://github.com/google/zetasql) - -Beam Calcite SQL is a variant of Apache Calcite, a dialect widespread in -big data processing. Beam Calcite SQL is the default Beam SQL dialect. Beam ZetaSQL is more compatible with BigQuery, so it's especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html). + +**Note:** Beam SQL supports for [ZetaSQL dialect](/documentation/dsls/sql/zetasql/overview) has been deprecated. To change dialects, pass [the dialect's full package name](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/package-summary.html) to the [`setPlannerName`](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/sql/impl/BeamSqlPipelineOptions.html#setPlannerName-java.lang.String-) method in the [`PipelineOptions`](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) interface. @@ -50,17 +48,5 @@ The [Shell page](/documentation/dsls/sql/shell) describes how to work with the i The [Beam Calcite SQL overview](/documentation/dsls/sql/calcite/overview) summarizes Apache Calcite operators, functions, syntax, and data types supported by Beam Calcite SQL. -## ZetaSQL dialect -For more information on the ZetaSQL features in Beam SQL, see the [Beam ZetaSQL dialect reference](/documentation/dsls/sql/zetasql/overview). - -To switch to Beam ZetaSQL, configure the [pipeline options](https://beam.apache.org/releases/javadoc/2.15.0/org/apache/beam/sdk/options/PipelineOptions.html) as follows: -``` -PipelineOptions options = ...; -options - .as(BeamSqlPipelineOptions.class) - .setPlannerName("org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner"); -``` -Note, Use of the `ZetaSQLQueryPlanner` requires an additional dependency on `beam-sdks-java-extensions-sql-zetasql` in addition to the `beam-sdks-java-extensions-sql` package required for `CalciteQueryPlanner`. - ## Beam SQL extensions Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects. diff --git a/website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md b/website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md index b68324a53677..5db8898dec9a 100644 --- a/website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md +++ b/website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md @@ -16,6 +16,9 @@ See the License for the specific language governing permissions and limitations under the License. --> # Beam ZetaSQL overview + +**Note:** Beam ZetaSQL has been deprecated ([details](https://github.com/apache/beam/issues/34423)). Please switch to use the default [Calcite SQL](/documentation/dsls/sql/calcite/overview) dialect. + Beam SQL supports a variant of the [ZetaSQL](https://github.com/google/zetasql) language. ZetaSQL is similar to the language in BigQuery's SQL framework. This Beam SQL dialect is especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html). Beam SQL has additional extensions leveraging Beam’s unified batch/streaming model and processing complex data types. You can use these extensions with all Beam SQL dialects, including Beam ZetaSQL.