Skip to content

Remove deprecated GpuTimeZoneDB cache overload usage#14500

Merged
res-life merged 1 commit intoNVIDIA:mainfrom
res-life:chongg/remove-deprecated-gputimezonedb-cache-api
Apr 1, 2026
Merged

Remove deprecated GpuTimeZoneDB cache overload usage#14500
res-life merged 1 commit intoNVIDIA:mainfrom
res-life:chongg/remove-deprecated-gputimezonedb-cache-api

Conversation

@res-life
Copy link
Copy Markdown
Collaborator

@res-life res-life commented Mar 31, 2026

Description

Previously, we use hybrid mode:

  • before 2200 year, generate a transition cache for DST timezone, and run on GPU for timezone rebasing.
    This mehod wastes GPU memories.
  • after 2200 year, use CPU to do timezone rebasing.
    This is slow.

Currently, we do not use hybrid mode any more, so remove the deprecated methods.

  • Replace deprecated GpuTimeZoneDB.cacheDatabaseAsync(int) and cacheDatabase(int) usage with the no-arg APIs.
  • Remove the now-unused spark.rapids.timezone.transitionCache.maxYear config and obsolete docs entry.
  • Update timezone perf and cast tests to call the replacement API.

Related change

Checklists

  • This PR has added documentation for new or modified features or behaviors.
  • This PR has added new tests or modified existing tests to cover new code paths.
    (Updated existing timezone perf and cast tests to exercise the replacement API entry points.)
  • Performance testing has been performed and its results are added in the PR description. Or, an issue has been filed with a link in the PR description.

Signed-off-by: Chong Gao chongg@nvidia.com

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 31, 2026

Greptile Summary

This PR completes the removal of the hybrid timestamp-processing mode by replacing all usages of the deprecated GpuTimeZoneDB.cacheDatabaseAsync(int) / cacheDatabase(int) overloads with their no-arg equivalents, and deletes the now-meaningless spark.rapids.timezone.transitionCache.maxYear configuration key alongside its documentation entry.

Key changes:

  • RapidsConf.scala: TIMESTAMP_RULES_END_YEAR config entry and timestampRulesEndYear lazy accessor are fully removed; no remaining references exist anywhere in the codebase.
  • Plugin.scala: Executor startup now calls GpuTimeZoneDB.cacheDatabaseAsync() unconditionally rather than passing a year ceiling.
  • cast_test.py: Six integration tests updated to call the no-arg cacheDatabase().
  • TimeZonePerfSuite.scala: Eight perf tests updated; stale timestampRulesEndYear local field removed; copyright year updated to 2026.
  • advanced_configs.md: Stale config table row removed.

Confidence Score: 5/5

Safe to merge — mechanical deprecation removal with complete and consistent updates across all call sites and documentation.

All usages of the deprecated int-overload APIs are replaced, no remaining references to the removed config exist anywhere in the codebase, and the documentation is kept in sync. No logic changes beyond removing the year ceiling, which is the intended behaviour.

No files require special attention.

Important Files Changed

Filename Overview
sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsConf.scala Removes TIMESTAMP_RULES_END_YEAR config entry and its lazy accessor timestampRulesEndYear — clean deletion with no remaining references.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/Plugin.scala Executor plugin init updated to call cacheDatabaseAsync() with no args instead of passing conf.timestampRulesEndYear.
integration_tests/src/main/python/cast_test.py Six cast-to-timestamp tests updated to call no-arg cacheDatabase() instead of cacheDatabase(2200).
tests/src/test/scala/com/nvidia/spark/rapids/timezone/TimeZonePerfSuite.scala Eight perf tests updated to call cacheDatabase() with no args; removes the now-unused timestampRulesEndYear private field; copyright year bumped to 2026.
docs/additional-functionality/advanced_configs.md Documentation entry for spark.rapids.timezone.transitionCache.maxYear removed to match the config deletion.

Sequence Diagram

sequenceDiagram
    participant EP as RapidsExecutorPlugin
    participant TZDB as GpuTimeZoneDB (JNI)
    participant GPU as GPU Memory

    Note over EP: Before: cacheDatabaseAsync(conf.timestampRulesEndYear)<br/>After: cacheDatabaseAsync()
    EP->>TZDB: cacheDatabaseAsync()
    TZDB-->>GPU: Load full timezone transition table asynchronously
    Note over TZDB,GPU: No year ceiling — full table always cached

    Note over EP: Tests (cast_test.py / TimeZonePerfSuite)
    EP->>TZDB: cacheDatabase()
    TZDB-->>GPU: Synchronous cache load
    EP->>EP: Thread.sleep(5ms)
    EP->>GPU: Execute timezone-dependent operations
Loading

Reviews (2): Last reviewed commit: "Remove deprecated GpuTimeZoneDB cache ov..." | Re-trigger Greptile

@res-life
Copy link
Copy Markdown
Collaborator Author

build

Switch the plugin and timezone tests to the no-arg cache APIs so they stop relying on deprecated max-year overloads. This also removes the unused transition cache config and docs entry that no longer affect behavior.

Made-with: Cursor
Signed-off-by: Chong Gao <res_life@163.com>
@res-life res-life force-pushed the chongg/remove-deprecated-gputimezonedb-cache-api branch from 89ada20 to bc69ad8 Compare March 31, 2026 08:21
@res-life res-life requested a review from firestarman March 31, 2026 08:38
Copy link
Copy Markdown
Collaborator

@firestarman firestarman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@res-life
Copy link
Copy Markdown
Collaborator Author

res-life commented Apr 1, 2026

build

@res-life res-life merged commit eb47571 into NVIDIA:main Apr 1, 2026
50 checks passed
res-life added a commit to NVIDIA/spark-rapids-jni that referenced this pull request Apr 2, 2026
## Summary
- remove the deprecated `GpuTimeZoneDB.cacheDatabaseAsync(int)`,
`cacheDatabase(int)`, and `getTransitions()` methods
- update the remaining timezone DB test to use the supported no-arg
cache API

## Related change
* NVIDIA/spark-rapids#14500

Signed-off-by: Chong Gao <res_life@163.com>
Co-authored-by: Chong Gao <res_life@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants