Skip to content

Replace AutoTuner Maven lookup with offline plugin staleness detection - #2127

Merged
parthosa merged 4 commits into
NVIDIA:devfrom
parthosa:fix/remove-autotuner-maven-lookup
Aug 26, 2026
Merged

Replace AutoTuner Maven lookup with offline plugin staleness detection#2127
parthosa merged 4 commits into
NVIDIA:devfrom
parthosa:fix/remove-autotuner-maven-lookup

Conversation

@parthosa

@parthosa parthosa commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Contributes: #2098

Summary

Replace AutoTuner's live Maven latest-release lookup with an offline staleness check. This removes an external network dependency from tuning and cluster recommendation paths, preventing Maven Central HTTP 429 responses while retaining useful plugin update guidance.

Approach

  • Continue detecting the existing rapids-4-spark_<scala>-YY.MM.patch artifact name; the repository and product were renamed to NVIDIA cuDF plugin for Apache Spark, but the Maven artifact name is unchanged.
  • Treat a plugin release month as potentially outdated only after a two-month grace period.
  • Recommend checking the stable cuDF Spark download page without querying Maven or naming an exact latest version.
  • Keep the existing missing-JAR and multiple-JAR recommendations.
  • Remove the live Maven crawler tests and exact latest-version expectations while keeping the OOM and cluster recommendation tests enabled.

Testing

  • cd core && mvn test
  • 804 tests passed, 0 failed, 16 ignored.
  • Focused AutoTuner and cluster recommendation suites: 101 passed, 0 failed, 7 ignored.

Follow-up

@github-actions github-actions Bot added the core_tools Scope the core module (scala) label Aug 24, 2026
@parthosa
parthosa force-pushed the fix/remove-autotuner-maven-lookup branch from f207de5 to 0bf1632 Compare August 24, 2026 21:55
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
@parthosa
parthosa force-pushed the fix/remove-autotuner-maven-lookup branch from 0bf1632 to 9e760e3 Compare August 24, 2026 22:09
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
@parthosa parthosa changed the title Remove AutoTuner Maven release lookup Replace AutoTuner Maven lookup with offline plugin staleness detection Aug 24, 2026
@parthosa parthosa self-assigned this Aug 24, 2026
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Comment thread core/src/main/scala/com/nvidia/spark/rapids/tool/tuning/AutoTuner.scala Outdated
SYangster
SYangster previously approved these changes Aug 26, 2026
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
@parthosa
parthosa merged commit c2a9be2 into NVIDIA:dev Aug 26, 2026
17 checks passed
@parthosa
parthosa deleted the fix/remove-autotuner-maven-lookup branch August 26, 2026 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core_tools Scope the core module (scala)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants