[Spark] Make deltaRestApi.enabled default to true#6951
Conversation
yili-db
force-pushed
the
stack/delta_api_default
branch
3 times, most recently
from
0fabe0b to
9823770
Compare
| // case-insensitive so defaults don't create duplicate keys. | ||
| val merged = new java.util.HashMap[String, String](options.asCaseSensitiveMap()) | ||
| Seq( | ||
| UCTokenBasedRestClientFactory.DELTA_REST_API_ENABLED_KEY -> "true", |
There was a problem hiding this comment.
No longer needed as the down stream will treat absent as true anyway.
yili-db
force-pushed
the
stack/delta_api_default
branch
6 times, most recently
from
ebc71b3 to
cb37c11
Compare
yili-db
force-pushed
the
stack/delta_api_default
branch
from
cb37c11 to
6f2ba62
Compare
TimothyW553
left a comment
TimothyW553
left a comment
There was a problem hiding this comment.
LGTM. mechanical change -- simple.
yili-db
force-pushed
the
stack/delta_api_default
branch
2 times, most recently
from
f667dd6 to
035d3c1
Compare
|
I have updated the way |
yili-db
force-pushed
the
stack/delta_api_default
branch
2 times, most recently
from
f0fa6f6 to
ac5ef5e
Compare
| getSupportIceberg.value), | ||
| libraryDependencies ++= Seq( | ||
| "io.unitycatalog" %% "unitycatalog-spark" % unityCatalogVersion excludeAll( | ||
| "io.unitycatalog" %% s"unitycatalog-spark${getSparkPackageSuffix.value}" % unityCatalogVersion excludeAll( |
There was a problem hiding this comment.
Locally SPARK_PACKAGE_SUFFIX is empty so this requests unitycatalog-spark_2.13 while the script publishes unitycatalog-spark_4.1_2.13, so a standalone examples/scala build cannot resolve it. Could we default the suffix to _4.1 to match the script?
I ran sbt update on this PR head with the env unset and it failed on unitycatalog-spark_2.13:0.5.0-SNAPSHOT-9196fa4 (Not found). The script's artifact name resolves to unitycatalog-spark_4.1_2.13, and CI is unaffected since it sets the env.
yili-db
force-pushed
the
stack/delta_api_default
branch
from
3a8b99f to
7b770c7
Compare
Signed-off-by: Yi Li <yi.li@databricks.com>
Signed-off-by: Yi Li <yi.li@databricks.com>
Signed-off-by: Yi Li <yi.li@databricks.com>
Signed-off-by: Yi Li <yi.li@databricks.com>
yili-db
force-pushed
the
stack/delta_api_default
branch
from
7b770c7 to
8cc4ad8
Compare
Signed-off-by: Yi Li <yi.li@databricks.com>
2 participants
🥞 Stacked PR
Use this link to review incremental changes.
Which Delta project/connector is this regarding?
Description
[Spark] Make deltaRestApi.enabled default to true
How was this patch tested?
CI
Does this PR introduce any user-facing changes?
No