Skip to content

CHANGELOG.md

Latest commit

 

History

History
182 lines (157 loc) · 11.6 KB

CHANGELOG.md

File metadata and controls

182 lines (157 loc) · 11.6 KB
Raw

CHANGELOG

Inspired from Keep a Changelog

Unreleased

Dependencies

  • Bumps commons-logging:commons-logging from 1.3.5 to 1.3.6
  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.21.1 to 2.21.3
  • Bumps jakarta.xml.bind:jakarta.xml.bind-api from 4.0.2 to 4.0.5
  • Bumps com.google.protobuf:protobuf-java from 4.34.0 to 4.34.1 in /spark/core
  • Bumps software.amazon.awssdk:http-client-spi from 2.42.4 to 2.42.17
  • Bumps commons-codec:commons-codec from 1.21.0 to 1.22.0
  • Bumps software.amazon.awssdk:identity-spi from 2.42.18 to 2.44.0
  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.20 to 2.21

2.0.0

Added

  • Add support for Amazon OpenSearch Serverless (#586)
  • Add Apache Spark 4.0 support (#684)
  • Add dedicated opensearch.search_after.size setting for serverless mode page size (#695)
  • Add Apache Spark 3.5 support with dedicated opensearch-spark-35 module (#717)

Changed

  • Switched to more reliable OpenSearch Lucene snapshot location (#597)
  • Make $JAVA8_HOME optional by using Gradle auto-detection and remove unnecessary JDK 8/11 setup from Spark 4.0 CI workflow (#703)
  • Upgrade to OpenSearch 3.x build-tools and test against OpenSearch 3.5.0 (#730)
  • Drop JDK 8 support: raise minimum runtime to Java 11, require JDK 21 to build (#730)
  • Upgrade Hadoop from 3.1.2 to 3.4.3 for JDK 11+ compatibility (#730)

Deprecated

Removed

  • Remove Apache Spark 2.x (sql-20) module support (#700)
  • Remove unused MultiReaderIterator and multiReader method (#667)

Fixed

  • Fixed serverless mode writes failing with 404 on _refresh endpoint (#692)
  • Fixed opensearch-spark-40 dependency license SHAs and added dependabot config for sql-40 module (#688)
  • Fixed global refresh when using dynamic index patterns (#686)
  • Fixed build failures when downloading Apache project dependencies (Hadoop, Hive, Spark) (#595)
  • Fixed serverless mode SaveMode.Overwrite failing when document count exceeds scroll size (#693)
  • Fixed RowSerializationEventConverterTest for Spark 3.4+ StructType.toString() format change (#702)
  • Fixed object fields with enabled: false returning empty structs or throwing exceptions when read via the connector (#715)
  • Fixed SigV4 signing failure on bulk retry after partial success causing x-amz-content-sha256 invalid (#759)

Security

Dependencies

  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.17.2 to 2.21.1
  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.17.2 to 2.21
  • Bumps io.netty:netty-all from 4.1.112.Final to 4.2.10.Final
  • Bumps commons-codec:commons-codec from 1.17.1 to 1.21.0
  • Bumps com.google.protobuf:protobuf-java from 4.28.2 to 4.34.0
  • Bumps commons-logging:commons-logging from 1.3.4 to 1.3.6
  • Bumps Spark3 version from 3.2.4 to 3.4.4 (#551)
  • Bumps Scala2.12 version from 2.12.8 to 2.12.20 (#551)
  • Bumps Scala2.13 version from 2.13.6 to 2.13.16 (#551)
  • Bumps org.opensearch.gradle:build-tools from 2.11.1 to 2.19.0 (#557)
  • Bumps org.slf4j:slf4j-api from 2.0.16 to 2.0.17
  • Bumps com.fasterxml.jackson.module:jackson-module-scala_2.12 from 2.16.1 to 2.19.1
  • Bumps AWS SDK from AWS SDK v1 com.amazonaws:aws-java-sdk-bundle 1.12.783 to AWS SDK v2 2.31.59 (#594)
  • Bumps com.gradleup.shadow from 8.3.6 to 9.3.2
  • Bumps software.amazon.awssdk:sdk-core from 2.31.59 to 2.41.25
  • Bumps software.amazon.awssdk:regions from 2.31.78 to 2.39.6
  • Bumps software.amazon.awssdk:auth from 2.34.5 to 2.42.0
  • Bumps software.amazon.awssdk:utils from 2.38.2 to 2.42.4
  • Bumps software.amazon.awssdk:http-client-spi from 2.39.6 to 2.40.3

1.3.0

Added

  • Added basic support for HTTP compression when writing to OpenSearch (#451)
  • Added support for k-nn vectors (#424)

Removed

  • Removed obsolete backported classes for Jackson 1.5 support (#471)

Fixed

  • Remove unnecessary outputCommitter setting (#465)

Dependencies

  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.17.0 to 2.17.2
  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.17.0 to 2.17.2
  • Bumps commons-logging:commons-logging from 1.3.1 to 1.3.4
  • Bumps com.google.protobuf:protobuf-java from 4.26.1 to 4.28.2
  • Bumps io.netty:netty-all from 4.1.109.Final to 4.1.112.Final
  • Bumps jakarta.servlet:jakarta.servlet-api from 6.0.0 to 6.1.0
  • Bumps commons-codec:commons-codec from 1.17.0 to 1.17.1
  • Bumps org.slf4j:slf4j-api from 2.0.13 to 2.0.16

1.2.0

Changed

  • Changes invalid request exception messages to always be detailed (#438)

Fixed

  • Fixes AWS SigV4 signing of POST requests with empty bodies such as scroll (#443)

Dependencies

  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.16.1 to 2.17.0
  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.16.1 to 2.17.0
  • Bumps commons-logging:commons-logging from 1.3.0 to 1.3.1
  • Bumps com.google.protobuf:protobuf-java from 3.25.3 to 4.26.1
  • Bumps io.netty:netty-all from 4.1.107.Final to 4.1.109.Final
  • Bumps org.slf4j:slf4j-api from 2.0.12 to 2.0.13
  • Bumps commons-codec:commons-codec from 1.16.1 to 1.17.0

1.1.0

Changed

Fixed

  • Corrected the delete by query endpoint to match the OpenSearch API (#350)

Dependencies

  • Bumps com.google.protobuf:protobuf-java from 3.22.3 to 3.25.3
  • Bumps com.esotericsoftware.kryo:kryo from 2.21 to 2.24.0
  • Bumps io.netty:netty-all from 4.1.92.Final to 4.1.107.Final
  • Bumps org.codehaus.woodstox:stax2-api from 3.1.4 to 4.2.2
  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.15.0 to 2.16.1
  • Bumps org.slf4j:slf4j-api from 1.7.6 to 2.0.12
  • Bumps com.google.code.findbugs:jsr305 from 2.0.1 to 3.0.2
  • Bumps org.apache.htrace:htrace-core4 from 4.1.0-incubating to 4.2.0-incubating
  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.15.0 to 2.16.1
  • Bumps org.json4s:json4s-ast_2.10 from 3.2.10 to 3.6.12
  • Bumps org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7 from 1.0.0 to 1.1.1
  • Bumps org.apache.avro:avro from 1.7.7 to 1.8.2
  • Bumps com.amazonaws:aws-java-sdk-bundle from 1.12.451 to 1.12.651
  • Bumps commons-codec:commons-codec from 1.15 to 1.16.1
  • Bumps com.amazonaws:aws-java-sdk-bundle from 1.12.451 to 1.12.524
  • Bumps commons-logging:commons-logging from 1.2 to 1.3.0
  • Bumps org.json4s:json4s-jackson_2.12 from 4.0.6 to 4.0.7
  • Bumps com.fasterxml.jackson.module:jackson-module-scala_2.12 from 2.15.2 to 2.16.1

1.0.1 - 05/30/2023

Dependencies

  • Bumps commons-logging:commons-logging from 1.1.1 to 1.2

Fixed

  • Fixes imports from being thirdparty to local (#245)

1.0.0 - 05/18/2023

Added

  • Added CHANGELOG and verifier workflow (65)
  • Added snapshot publication workflow (218)
  • Added release workflow (227)

Changed

  • [Spark Distribution] Default Assemble artifact to Spark 3 (107)
  • Changed the default deserialization/serialization logic from Object based to JSON based (154)

Fixed

  • Restored skipped push down tests (125)
  • Fixed spark failures due to deserialization failed logic (219)

Dependencies

  • Bumps com.google.guava:guava from 16.0.1 to 23.0
  • Bumps com.google.guava:guava from 11.0 to 23.0
  • Bumps commons-codec:commons-codec from 1.4 to 1.15
  • Bumps com.google.code.findbugs:jsr305 from 2.0.1 to 3.0.2
  • Bumps jakarta.servlet:jakarta.servlet-api from 4.0.3 to 6.0.0
  • Bumps com.fasterxml.jackson.core:jackson-databind from 2.7.8 to 2.15.0
  • Bumps commons-httpclient:commons-httpclient from 3.0.1 to 3.1
  • Bumps org.apache.rat:apache-rat from 0.13 to 0.15
  • Bumps com.esotericsoftware.kryo:kryo from 2.21 to 2.24.0
  • Bumps org.codehaus.woodstox:stax2-api from 3.1.4 to 4.2.1
  • Bumps org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7 from 1.0.0 to 1.1.1
  • Bumps com.fasterxml.jackson.core:jackson-annotations from 2.6.7 to 2.15.0
  • Bumps org.json4s:json4s-ast_2.10 from 3.2.10 to 3.6.12
  • Bumps commons-logging:commons-logging from 1.1.1 to 1.2
  • Bumps com.amazonaws:aws-java-sdk-bundle from 1.12.397 to 1.12.451
  • Bumps org.slf4j:slf4j-api from 1.7.6 to 1.7.36
  • Bumps com.google.protobuf:protobuf-java from 2.5.0 to 3.22.3
  • Bumps io.netty:netty-all from 4.0.29.Final to 4.1.92.Final
  • Bumps jline:jline from 0.9.94 to 1.0
  • Bumps org.json4s:json4s-jackson_2.12 from 3.2.11 to 4.0.6
  • Bumps org.apache.avro:avro from 1.7.7 to 1.11.1