v2.7.3

release v2.7.3

v2.8.0

What's Changed

• test(cdc): migrate mongodb-cdc tests to inline style with all data types by @xiangjinwu in #23753
• fix: change placeholder of username in ldap filter config by @yezizp2012 in #23782
• feat(vector): support stream vector index lookup by @wenym1 in #23288
• refactor: improve cluster kill-and-recovery tests with precise recovery tracking per database by @shanicky in #23728
• feat(iceberg): support s3 and glue iam role arn by @chenzl25 in #23775
• chore(deps): Bump mach2 from 0.5.0 to 0.6.0 by @dependabot[bot] in #23784
• feat(streaming): add functionality to override config when building actors by @BugenZhao in #23735
• chore(ci): add maven cache for build-other.sh step by @BugenZhao in #23781
• feat(metrics): add relation busy rate to dashboard by @kwannoel in #23730
• chore: rename iceberg common fields to better names. by @chenzl25 in #23790
• feat(iceberg): support assume role for s3 and glue catalog together with enable_config_load by @chenzl25 in #23788
• feat(iceberg): change iceberg list interval sec to 10 sec by @chenzl25 in #23791
• fix(iceberg): rebuild stream when errors occur for iceberg list executor by @chenzl25 in #23793
• chore: replace number_prefix with unit-prefix by @Standing-Man in #23801
• feat(mysql-cdc): monitor the progress of binlog offsets in the upstream and downstream processes by @wcy-fdu in #23692
• fix(iceberg): fix iceberg engine with compaction extra config by @chenzl25 in #23809
• refactor(meta): refine inflight info ownership and avoid unnecessary clone by @wenym1 in #23754
• fix(iceberg): fix batch iceberg list executor by @chenzl25 in #23811
• fix(dashboard): fix some foyer metrics name followed by OpenMetrics by @MrCroxx in #23818
• feat: add periodic refresh table jobs & refactor ProgressTracker by @tabVersion in #23737
• fix(iceberg): fix batch iceberg list executor by @chenzl25 in #23824
• fix: support AdaptiveParallelismStrategy as string format in config by @shanicky in #23817
• chore: add back grafana json by @tabVersion in #23831
• chore(iceberg): better logging if mis-config enable_config_load on Cloud by @tabVersion in #23822
• feat: migrate JoinEncodingType from session config to per-job config override by @BugenZhao in #23768
• fix(ci): use larger runner with more disk space for workflows that build RW by @BugenZhao in #23842
• fix(streaming): fix locality backfill state -- write upstream chunk to the state by @chenzl25 in #23838
• fix: update Minio client initialization to use keyword arguments by @tabVersion in #23836
• fix(ci):  bump and pin versions of mdbook and related tools by @BugenZhao in #23840
• fix(ci): update connector node integration test pipeline to java 17 & 21 by @tabVersion in #23854
• feat(meta): include view definition in diagnostic report by @zwang28 in #23853
• fix: Supports config checks within compaction group scheduling. by @Li0k in #23841
• feat: implement watermark derivation for AsOf joins by @yuhao-su in #23803
• fix(sink): use downstream pk to compact when stream key is its subset by @BugenZhao in #23859
• perf(expr): support writer-style #[function] for array type by @silver-ymz in #23767
• feat: add last_success_time tracking to refresh jobs by @tabVersion in #23834
• fix(streaming): deduplicate locality backfill ordering edge casued by share sub-plan by @chenzl25 in #23869
• fix(optimizer): fix gap fill column pruning by @chenzl25 in #23867
• fix(test): delay mongodb-cdc test insertion by @xiangjinwu in #23866
• fix(iceberg): add snapshot id to iceberg scan by @chenzl25 in #23882
• feat(frontend): add handler for ALTER .. [SET | RESET] CONFIG by @BugenZhao in #23748
• refactor(meta): introduce strong typed subscriber id by @wenym1 in #23883
• fix(optimizer): use shard_by_exact_key instead of shard_by_key for locality backfill by @chenzl25 in #23888
• feat(optimizer): improve index selection for topn by @chenzl25 in #23894
• refactor(foyer): upgrade foyer to v0.21.0-dev by @MrCroxx in #23750
• refactor(config): remove unnecessary prefix in configs by @BugenZhao in #23889
• chore: upgrade parking_lot to 0.12.5 by @Li0k in #23902
• chore: replace sleep + query pattern to retry X backoff 1s by @tabVersion in #23855
• feat(cdc): allow user alter debezium.max.queue.size to avoid JVM OOM by @wcy-fdu in #23886
• fix: invalid schema type for LogicalVectorSearchLookupJoin by @silver-ymz in #23893
• fix(test): Use tokio time pausing in ut to avoid unstable clocks for periodic barriers by @Chiro11 in #23905
• fix(ci): fix several failures in main-cron by @BugenZhao in #23900
• test(backfill): add more tests for locality backfill by @chenzl25 in #23906
• fix: use on_conflict_do_nothing for mysql backend by @tabVersion in #23915
• fix(optimizer): fix agg to dedup optimization by @chenzl25 in #23922
• refactor: improve stream parallelism resolution and error handling by @shanicky in #23921
• feat(meta): add service impl for ALTER .. SET CONFIG by @BugenZhao in #23864
• fix(mongo-cdc): add connection check for mongo cdc by @wcy-fdu in #23850
• feat(sink): exactly once sink on coordination process by @Chiro11 in #23657
• feat(frontend): add system table rw_streaming_job_config by @BugenZhao in #23914
• feat: use datafusion engine for Iceberg batch query by @silver-ymz in #23860
• feat: migrate configs for sync log store and over-window executor from session config to per-job config override by @BugenZhao in #23792
• fix(sink): fix redshift timestamptz error by @xxhZs in #23812
• feat: persist compactor iceberg flag across meta and diagnostics by @Li0k in #23884
• chore: allow altering owner of iceberg table by @yezizp2012 in #23944
• chore(deps): Bump uuid from 1.18.1 to 1.19.0 by @dependabot[bot] in #23951
• feat: Expose license status, limits, and features in diagnose output by @Li0k in #23885
• refactor(hummock): refine fetch_block impl, restore old style by @MrCroxx in #23957
• feat(stream,dashboard): aggregate tokio actor metrics by fragment by @kwannoel in #23747
• refactor(streaming): move MaterializeCache to a separate module by @BugenZhao in #23961
• feat: support setting pause_on_next_bootstrap_offline by @EdwinaZhu in #23910
• chore(streaming): add explicit target for expression error log (stream_expr_error) by @kwannoel in #23964
• fix(cdc): fix snapshot schema by @zwang28 in #23970
• feat(meta): add ...

v2.8.0-rc.1

release v2.8.0-rc.1

v2.7.2

release v2.7.2

v2.7.1

release v2.7.1

v2.7.0

release v2.7.0

Upgrade-time behavior changes

In v2.7.0, following the refactoring of sink-into-table, the system will automatically migrate fragments of all legacy tables to a unified pattern.

• Migration process: Upon upgrading and starting the cluster, a background task will automatically initiate the migration for all tables. You can see meta logs like:
  • "migrated table fragments table_id={table_id}" indicate a successful migration.
  • "failed to migrate legacy table fragments" indicate an error on migration.
• Impact: Migration errors triggering these logs will not affect existing jobs or system availability. The system will continuously retry the migration.
  • If the migration fails but does not impact your usage, please report the issue via Slack or GitHub issue and wait for bug fix in the next version.
  • To disable automatic migration, set enable_legacy_table_migration = false in the [meta] section of your toml file (requires meta node restart).
• Limitations & Workaround: You cannot create a new sink-into-table or drop an existing one on a table that has not completed migration. If the background migration is stuck or taking too long and you urgently need to perform these DDL operations, you can force a migration by running:

  ALTER TABLE {table_name} ADD COLUMN {col_name} int;
  ALTER TABLE {table_name} DROP COLUMN {col_name};
  

  After this, you can use it as a normal table.

SQL features

• SQL commands:
  • Supports vector index Top-N lookups in streaming queries, enabling MVs and sinks to enrich each row with nearest-neighbor results. #23288
  • Supports ALTER FRAGMENT ... SET PARALLELISM command to alter the parallelism of streaming fragments. #23523
  • Reduces reserved keywords in EXPLAIN options (e.g., BACKFILL, TRACE) by parsing them as identifiers.#23208
  • Adds schema name to name column in the output of SHOW [TABLE|VIEW|SOURCE|SINK|...] commands. #23119
  • When DROP/CANCEL takes longer than slow_ddl_notification_secs (30s), you can run RECOVER for immediate effect. #23613
  • Supports ALTER TABLE ... ADD COLUMN command on tables with incoming sinks.#23766
  • Supports querying vector index in append-only streaming. #23288
• SQL functions & operators:
  • Supports parsing STRUCT< > as empty struct type. #23099
  • Supports rw_cluster_id()  function to return the cluster's unique identifier as a VARCHAR. #23260
• System catalog:
  • Adds system catalog rw_recovery_info to display the database-level recovery state. #23728
  • Adds column parallelism_policy in system catalog rw_fragments. #23524

Connectors

• Breaking change: When decoding Protobuf messages, RisingWave returns NULL for missing optional fields and default values for all other fields. #23120
• Supports using CREATE SOURCE directly for UPSERT-format sources requiring a primary key, without creating a table first. #22856
• File sink now uses UUIDs instead of executor_id in output file names. #23981
• Supports on-demand manual refresh for Iceberg batch tables with position deletes and equity deletes. #23527
• Technical preview: Supports periodic automatic refresh for batch tables with persistent state tracking and improved observability. #23737
• Technical preview: Supports files-with-delete Iceberg compaction and introduces additional compaction parameters. #23660, #23809
• Supports Azure Data Lake Storage Gen2 for Iceberg sink when using REST catalog. #23350
• Supports configuring S3 and Glue with independent credentials for Iceberg connector when using Glue catalog. #23775
• Support assuming IAM roles for S3 and Glue when enable_config_load is enabled for Iceberg connector when using Glue catalog. #23788
• Supports key-pair authentication using RSA private keys for Snowflake sink. #23525
• Supports JDBC sink connector for SQL Server. #23416
• Changes the default value of commit_checkpoint_interval from 10 to 60 for Iceberg sink. #23552
• Supports is_exactly_once for Iceberg sink by default. #23553
• Supports vended credentials for Iceberg connector when using REST catalog. #23567
• Supports configuring postgres.is.aws.rds to specify if the upstream PostgreSQL is hosted on AWS RDS. #23420
• Supports using Base64-encoded JSON strings for BigQuery credentials. #23100
• Supports TLS connections to Redis sink using rediss:// protocol. #23223
• Supports Redis Streams for Redis sink. #23412
• Supports COPY (<query>) TO STDOUT syntax to load data from PostgreSQL tables. #23198
• Supports ack for Pulsar source, enabling control over broker resending of unacknowledged messages. #22158
• Supports force_compaction on sinks to enable buffering and compaction of updates when the stream key is the same as the primary key. #23681

Installation and deployment

• Counts the CPU cores of all nodes (instead of only compute nodes) in the cluster against the cpu_core_limit specified in the license key. #23390
• Changes cpu_core_limit in the license key to rwu_limit to limit both CPU cores and total cluster memory. #23403
• Breaking change: Requires the correct aud claim for authentication. #23155
• Supports LDAP authentication to verify users against an external LDAP directory server. #23150

Cluster configuration changes

• Adds session variable enable_index_selection to control whether to enable index selection for both streaming and batch queries. #23092
• Technical preview: Adds session variable enable_locality_backfill to control locality backfilling, which extends index-based optimizations to the entire backfill pipeline. #23275
• Adds configuration option RW_SSL_ENFORCE to control SSL enforcement for security. #23306

Assets

• Run this version from Docker:
  docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.7.0-standalone single_node
• Prebuilt all-in-one library for Linux
• Source code (zip)
• Source code (tar.gz)
• risectl - a CLI tool for managing and accessing RisingWave clusters

v2.6.3

release v2.6.3

v2.7.0-rc.1

release v2.7.0-rc.1

v2.6.2

release v2.6.2

v2.5.3

release v2.5.3