Skip to content

Commit fa8a691

Browse files
NvTimLiuNvTimLiu
authored
Merge pull request #11218 from NVIDIA/merge-branch-24.06-to-main
Merge branch-24.06 into main [skip ci]
2 parents f932a78 + 54c9d96 commit fa8a691

File tree

73 files changed

+240
-148
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+240
-148
lines changed

‎CHANGELOG.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Change log
2-
Generated on 2024-06-13
2+
Generated on 2024-07-18
33

44
## Release 24.06
55

@@ -48,6 +48,12 @@ Generated on 2024-06-13
4848
### PRs
4949
|||
5050
|:---|:---|
51+
|[#11221](https://github.com/NVIDIA/spark-rapids/pull/11221)|Change cudf version back to 24.06.0-SNAPSHOT [skip ci]|
52+
|[#11217](https://github.com/NVIDIA/spark-rapids/pull/11217)|Update latest changelog [skip ci]|
53+
|[#11211](https://github.com/NVIDIA/spark-rapids/pull/11211)|Use fixed seed for test_from_json_struct_decimal|
54+
|[#11203](https://github.com/NVIDIA/spark-rapids/pull/11203)|Update version to 24.06.1-SNAPSHOT|
55+
|[#11205](https://github.com/NVIDIA/spark-rapids/pull/11205)|Update docs for 24.06.1 release [skip ci]|
56+
|[#11056](https://github.com/NVIDIA/spark-rapids/pull/11056)|Update latest changelog [skip ci]|
5157
|[#11052](https://github.com/NVIDIA/spark-rapids/pull/11052)|Add spark343 shim for scala2.13 dist jar|
5258
|[#10981](https://github.com/NVIDIA/spark-rapids/pull/10981)|Update latest changelog [skip ci]|
5359
|[#10984](https://github.com/NVIDIA/spark-rapids/pull/10984)|[DOC] Update docs for 24.06.0 release [skip ci]|

‎CONTRIBUTING.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -130,15 +130,15 @@ mvn -pl dist -PnoSnapshots package -DskipTests
130130
Verify that shim-specific classes are hidden from a conventional classloader.
131131

132132
```bash
133-
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.0-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl
133+
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.1-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl
134134
Error: class not found: com.nvidia.spark.rapids.shims.SparkShimImpl
135135
```
136136

137137
However, its bytecode can be loaded if prefixed with `spark3XY` not contained in the package name
138138

139139
```bash
140-
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.0-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
141-
Warning: File dist/target/rapids-4-spark_2.12-24.06.0-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
140+
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.1-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
141+
Warning: File dist/target/rapids-4-spark_2.12-24.06.1-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
142142
Compiled from "SparkShims.scala"
143143
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
144144
```
@@ -181,7 +181,7 @@ mvn package -pl dist -am -Dbuildver=340 -DallowConventionalDistJar=true
181181
Verify `com.nvidia.spark.rapids.shims.SparkShimImpl` is conventionally loadable:
182182
183183
```bash
184-
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.0-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
184+
$ javap -cp dist/target/rapids-4-spark_2.12-24.06.1-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
185185
Compiled from "SparkShims.scala"
186186
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
187187
```

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ as a `provided` dependency.
7373
<dependency>
7474
<groupId>com.nvidia</groupId>
7575
<artifactId>rapids-4-spark_2.12</artifactId>
76-
<version>24.06.0</version>
76+
<version>24.06.1</version>
7777
<scope>provided</scope>
7878
</dependency>
7979
```

‎aggregator/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828
<artifactId>rapids-4-spark-aggregator_2.12</artifactId>
2929
<name>RAPIDS Accelerator for Apache Spark Aggregator</name>
3030
<description>Creates an aggregated shaded package of the RAPIDS plugin for Apache Spark</description>
31-
<version>24.06.0</version>
31+
<version>24.06.1</version>
3232

3333
<properties>
3434
<rapids.module>aggregator</rapids.module>

‎api_validation/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../shim-deps/pom.xml</relativePath>
2727
</parent>
2828
<artifactId>rapids-4-spark-api-validation_2.12</artifactId>
29-
<version>24.06.0</version>
29+
<version>24.06.1</version>
3030

3131
<properties>
3232
<rapids.module>api_validation</rapids.module>

‎datagen/README.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,12 @@ Where `$SPARK_VERSION` is a compressed version number, like 330 for Spark 3.3.0.
2424

2525
After this the jar should be at
2626
`target/datagen_2.12-$PLUGIN_VERSION-spark$SPARK_VERSION.jar`
27-
for example a Spark 3.3.0 jar for the 24.06.0 release would be
28-
`target/datagen_2.12-24.06.0-spark330.jar`
27+
for example a Spark 3.3.0 jar for the 24.06.1 release would be
28+
`target/datagen_2.12-24.06.1-spark330.jar`
2929

3030
To get a spark shell with this you can run
3131
```shell
32-
spark-shell --jars target/datagen_2.12-24.06.0-spark330.jar
32+
spark-shell --jars target/datagen_2.12-24.06.1-spark330.jar
3333
```
3434

3535
After that you should be good to go.

‎datagen/ScaleTest.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ $SPARK_HOME/bin/spark-submit \
4444
--conf spark.sql.parquet.datetimeRebaseModeInWrite=CORRECTED \
4545
--class com.nvidia.rapids.tests.scaletest.ScaleTestDataGen \ # the main class
4646
--jars $SPARK_HOME/examples/jars/scopt_2.12-3.7.1.jar \ # one dependency jar just shipped with Spark under $SPARK_HOME
47-
./target/datagen_2.12-24.06.0-spark332.jar \
47+
./target/datagen_2.12-24.06.1-spark332.jar \
4848
1 \
4949
10 \
5050
parquet \

‎datagen/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,13 @@
2121
<parent>
2222
<groupId>com.nvidia</groupId>
2323
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
24-
<version>24.06.0</version>
24+
<version>24.06.1</version>
2525
<relativePath>../shim-deps/pom.xml</relativePath>
2626
</parent>
2727
<artifactId>datagen_2.12</artifactId>
2828
<name>Data Generator</name>
2929
<description>Tools for generating large amounts of data</description>
30-
<version>24.06.0</version>
30+
<version>24.06.1</version>
3131
<properties>
3232
<rapids.module>datagen</rapids.module>
3333
<target.classifier/>

‎delta-lake/delta-20x/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-20x_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.0.x Support</name>
3131
<description>Delta Lake 2.0.x support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-20x</rapids.module>

‎delta-lake/delta-21x/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-21x_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.1.x Support</name>
3131
<description>Delta Lake 2.1.x support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-21x</rapids.module>

‎delta-lake/delta-22x/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-22x_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.2.x Support</name>
3131
<description>Delta Lake 2.2.x support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-22x</rapids.module>

‎delta-lake/delta-23x/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-parent_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-23x_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.3.x Support</name>
3131
<description>Delta Lake 2.3.x support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-23x</rapids.module>

‎delta-lake/delta-24x/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-24x_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.4.x Support</name>
3131
<description>Delta Lake 2.4.x support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-24x</rapids.module>

‎delta-lake/delta-spark330db/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-spark330db_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Databricks 11.3 Delta Lake Support</name>
3131
<description>Databricks 11.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-spark330db</rapids.module>

‎delta-lake/delta-spark332db/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-spark332db_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Databricks 12.2 Delta Lake Support</name>
3131
<description>Databricks 12.2 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-spark332db</rapids.module>

‎delta-lake/delta-spark341db/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-spark341db_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Databricks 13.3 Delta Lake Support</name>
3131
<description>Databricks 13.3 Delta Lake support for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.compressed.artifact>false</rapids.compressed.artifact>

‎delta-lake/delta-stub/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,14 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828

2929
<artifactId>rapids-4-spark-delta-stub_2.12</artifactId>
3030
<name>RAPIDS Accelerator for Apache Spark Delta Lake Stub</name>
3131
<description>Delta Lake stub for the RAPIDS Accelerator for Apache Spark</description>
32-
<version>24.06.0</version>
32+
<version>24.06.1</version>
3333

3434
<properties>
3535
<rapids.module>../delta-lake/delta-stub</rapids.module>

‎dist/pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@
2222
<parent>
2323
<groupId>com.nvidia</groupId>
2424
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
25-
<version>24.06.0</version>
25+
<version>24.06.1</version>
2626
<relativePath>../jdk-profiles/pom.xml</relativePath>
2727
</parent>
2828
<artifactId>rapids-4-spark_2.12</artifactId>
2929
<name>RAPIDS Accelerator for Apache Spark Distribution</name>
3030
<description>Creates the distribution package of the RAPIDS plugin for Apache Spark</description>
31-
<version>24.06.0</version>
31+
<version>24.06.1</version>
3232
<dependencies>
3333
<dependency>
3434
<groupId>com.nvidia</groupId>

‎docs/archive.md

+84
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,90 @@ nav_order: 15
55
---
66
Below are archived releases for RAPIDS Accelerator for Apache Spark.
77

8+
## Release v24.06.0
9+
### Hardware Requirements:
10+
11+
The plugin is tested on the following architectures:
12+
13+
GPU Models: NVIDIA V100, T4, A10/A100, L4 and H100 GPUs
14+
15+
### Software Requirements:
16+
17+
OS: Ubuntu 20.04, Ubuntu 22.04, CentOS 7, or Rocky Linux 8
18+
19+
NVIDIA Driver*: R470+
20+
21+
Runtime:
22+
Scala 2.12, 2.13
23+
Python, Java Virtual Machine (JVM) compatible with your spark-version.
24+
25+
* Check the Spark documentation for Python and Java version compatibility with your specific
26+
Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1.
27+
28+
Supported Spark versions:
29+
Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4
30+
Apache Spark 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4
31+
Apache Spark 3.4.0, 3.4.1, 3.4.2, 3.4.3
32+
Apache Spark 3.5.0, 3.5.1
33+
34+
Supported Databricks runtime versions for Azure and AWS:
35+
Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0)
36+
Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2)
37+
Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1)
38+
39+
Supported Dataproc versions (Debian/Ubuntu):
40+
GCP Dataproc 2.0
41+
GCP Dataproc 2.1
42+
43+
Supported Dataproc Serverless versions:
44+
Spark runtime 1.1 LTS
45+
Spark runtime 2.0
46+
Spark runtime 2.1
47+
Spark runtime 2.2
48+
49+
*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet
50+
for your hardware's minimum driver version.
51+
52+
*For Cloudera and EMR support, please refer to the
53+
[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ.
54+
55+
### RAPIDS Accelerator's Support Policy for Apache Spark
56+
The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html)
57+
58+
### Download RAPIDS Accelerator for Apache Spark v24.06.0
59+
60+
| Processor | Scala Version | Download Jar | Download Signature |
61+
|-----------|---------------|--------------|--------------------|
62+
| x86_64 | Scala 2.12 | [RAPIDS Accelerator v24.06.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.06.0/rapids-4-spark_2.12-24.06.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.06.0/rapids-4-spark_2.12-24.06.0.jar.asc) |
63+
| x86_64 | Scala 2.13 | [RAPIDS Accelerator v24.06.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.06.0/rapids-4-spark_2.13-24.06.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.06.0/rapids-4-spark_2.13-24.06.0.jar.asc) |
64+
| arm64 | Scala 2.12 | [RAPIDS Accelerator v24.06.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.06.0/rapids-4-spark_2.12-24.06.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/24.06.0/rapids-4-spark_2.12-24.06.0-cuda11-arm64.jar.asc) |
65+
| arm64 | Scala 2.13 | [RAPIDS Accelerator v24.06.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.06.0/rapids-4-spark_2.13-24.06.0-cuda11-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/24.06.0/rapids-4-spark_2.13-24.06.0-cuda11-arm64.jar.asc) |
66+
67+
This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with
68+
CUDA 11.8 through CUDA 12.0.
69+
70+
### Verify signature
71+
* Download the [PUB_KEY](https://keys.openpgp.org/[email protected]).
72+
* Import the public key: `gpg --import PUB_KEY`
73+
* Verify the signature for Scala 2.12 jar:
74+
`gpg --verify rapids-4-spark_2.12-24.06.0.jar.asc rapids-4-spark_2.12-24.06.0.jar`
75+
* Verify the signature for Scala 2.13 jar:
76+
`gpg --verify rapids-4-spark_2.13-24.06.0.jar.asc rapids-4-spark_2.13-24.06.0.jar`
77+
78+
The output of signature verify:
79+
80+
gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) <[email protected]>"
81+
82+
### Release Notes
83+
* Improve support for Unity Catalog on Databricks
84+
* Added support for parse_url PATH
85+
* Added support for array_filter
86+
* Added support for Spark 3.4.3
87+
* For updates on RAPIDS Accelerator Tools, please visit [this link](https://github.com/NVIDIA/spark-rapids-tools/releases)
88+
89+
For a detailed list of changes, please refer to the
90+
[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).
91+
892
## Release v24.04.1
993
### Hardware Requirements:
1094

‎docs/configs.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ The following is the list of options that `rapids-plugin-4-spark` supports.
1010
On startup use: `--conf [conf key]=[conf value]`. For example:
1111

1212
```
13-
${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-24.06.0-cuda11.jar \
13+
${SPARK_HOME}/bin/spark-shell --jars rapids-4-spark_2.12-24.06.1-cuda11.jar \
1414
--conf spark.plugins=com.nvidia.spark.SQLPlugin \
1515
--conf spark.rapids.sql.concurrentGpuTasks=2
1616
```

0 commit comments

Comments
 (0)