TypeScript SDK v1.0.2

Bug Fixes

• Split platform-specific native binaries into separate npm packages (@databricks/zerobus-ingest-sdk-linux-x64-gnu, -linux-arm64-gnu, -win32-x64-msvc). npm now auto-installs only the binary matching the user's OS/arch via optionalDependencies, reducing download size from ~15MB to ~5MB.

Java SDK v1.1.1

Bug Fixes

• Fixed isClosed() returning false after the stream internally closed due to non-retryable errors (e.g. server rejection, auth failure). The JNI layer now checks both the wrapper state and the Rust stream's internal closed flag. Affects both proto/JSON streams and Arrow streams.

Java SDK v1.1.0

Major Changes

• License: Migrated from the Databricks License to the Apache License 2.0

New Features and Improvements

• Arrow Flight Support (Experimental): Added support for ingesting Apache Arrow VectorSchemaRoot batches via Arrow Flight protocol
  • Note: Arrow Flight is not yet supported by default from the Zerobus server side.
  • New ZerobusArrowStream class with ingestBatch(), waitForOffset(), flush(), close(), getUnackedBatches() methods
  • New ArrowStreamConfigurationOptions for configuring Arrow streams (max inflight batches, recovery, timeouts, IPC compression)
  • Configurable IPC compression via ArrowStreamConfigurationOptions.setIpcCompression() (supports LZ4_FRAME and ZSTD)
  • New createArrowStream() and recreateArrowStream() methods on ZerobusSdk
  • Accepts VectorSchemaRoot directly via ingestBatch() (IPC serialization handled internally)
  • Arrow is opt-in: add arrow-vector and arrow-memory-netty as dependencies (provided scope, >= 15.0.0)

Bug Fixes

• Classloader Isolation Compatibility: Fixed NoClassDefFoundError when using the SDK inside Spring Boot. JNI class references are now cached as GlobalRefs during JNI_OnLoad, so native daemon threads no longer rely on FindClass through the system classloader.
• Fixed proto generation tool to skip reserved field numbers 19000-19999 for tables with more than 19000 columns

Internal Changes

• Added arrow-vector 17.0.0 as provided dependency for Arrow Flight support
• Added arrow-memory-netty 17.0.0 as test dependency for integration tests
• Uses existing JNI Arrow Flight bindings from Rust SDK (nativeCreateArrowStream, nativeIngestBatch, etc.)

API Changes

• Added createArrowStream(String tableName, Schema schema, String clientId, String clientSecret) to ZerobusSdk
• Added createArrowStream(String tableName, Schema schema, String clientId, String clientSecret, ArrowStreamConfigurationOptions options) to ZerobusSdk
• Added recreateArrowStream(ZerobusArrowStream closedStream) to ZerobusSdk
• Added ZerobusArrowStream class with methods: ingestBatch(), waitForOffset(), flush(), close(), getUnackedBatches(), isClosed(), getTableName(), getOptions()
• Added ArrowStreamConfigurationOptions class with fields: maxInflightBatches, recovery, recoveryTimeoutMs, recoveryBackoffMs, recoveryRetries, serverLackOfAckTimeoutMs, flushTimeoutMs, connectionTimeoutMs, ipcCompression
• Added IPCCompressionType enum with values: NONE, LZ4_FRAME, ZSTD
• Added optional dependency: org.apache.arrow:arrow-vector >= 15.0.0 (provided scope)

Go SDK v1.1.1

Re-release of v1.1.0 with pre-built FFI libraries included. v1.1.0 is retracted due to missing native libraries.

Major Changes

• License: Migrated from the Databricks License to the Apache License 2.0

New Features and Improvements

[Experimental] Arrow Flight Ingestion: Added experimental Arrow Flight support for high-throughput Apache Arrow RecordBatch ingestion

• New CreateArrowStream and CreateArrowStreamWithHeadersProvider methods on ZerobusSdk
• New ZerobusArrowStream type with IngestBatch, WaitForOffset, Flush, Close, and GetUnackedBatches methods
• Configurable IPC compression via ArrowStreamConfigurationOptions.IpcCompression (supports LZ4Frame and Zstd)

TypeScript SDK v1.0.1

Bug Fixes

• Fixed npm packaging: v1.0.0 was published without the napi-rs generated index.js loader and index.d.ts type declarations, causing MODULE_NOT_FOUND on require('@databricks/zerobus-ingest-sdk'). The platform-specific native binary packages (e.g. @databricks/zerobus-ingest-sdk-linux-x64-gnu) were also missing from npm. This release includes all generated files and platform packages.

FFI Libraries v1.1.0

Major Changes

• License: Migrated from the Databricks License to the Apache License 2.0
• Removed macOS x86_64 and macOS aarch64 support.

New Features and Improvements

• Added dynamic library (.so / .dylib / .dll) output alongside static library

Rust SDK v1.2.0

Major Changes

• License: Migrated from the Databricks License to the Apache License 2.0

New Features and Improvements

• Added the schema module with descriptor_from_uc_columns /
  descriptor_from_uc_schema, which convert a Unity Catalog table schema
  (including nested STRUCT, ARRAY, and MAP columns via type_json) into
  a prost_types::DescriptorProto that can be passed to
  TableProperties::descriptor_proto. Enables building descriptors at runtime
  without pre-generating .proto files.

Internal Changes

• The generate_files CLI tool now delegates schema → descriptor conversion
  to the SDK's new schema module instead of its own hand-rolled DDL-string
  parser, and renders the resulting DescriptorProto back to proto2 text.

Breaking Changes

• generate_files: the emitted .proto files have changed shape for
  non-trivial schemas. Consumers regenerating existing files should expect:
  • Field numbers now follow Unity Catalog's position + 1 (so gaps from
    DROP COLUMN under Delta column-mapping are preserved) instead of the
    previous 1,2,3… sequential numbering with a 19000-range skip.
  • Nested struct messages use path-based names (e.g. OuterInner instead of
    Inner) and are emitted hierarchically inside their parent message.
  • Struct field nullability now honors Unity Catalog's nullable flag
    instead of being forced to optional.

Rust SDK v1.1.0

[Rust] Release 1.1.0 (#191)

Signed-off-by: elenagaljak-db <elena.galjak@databricks.com>

Rust SDK v1.0.1

Bug Fixes

• Fixed TLS certificate validation failure when behind corporate VPN/proxy with MITM certificates (e.g., GlobalProtect). Changed reqwest TLS configuration from rustls-tls to rustls-tls-native-roots + rustls-tls-webpki-roots, so the SDK now loads CA certificates from the OS native trust store (respecting SSL_CERT_FILE and system certificate stores) while keeping bundled Mozilla roots as a fallback for minimal environments.

New Features and Improvements

• Exported OAuthHeadersProvider in the public API, allowing clients to directly construct and use the built-in OAuth 2.0 headers provider.

JNI Libraries v1.0.1

Native JNI libraries for zerobus-sdk-java v1.0.1

Platforms included:

• linux-x86-64
• linux-aarch64
• windows-x86-64