Skip to content

Enable precision 4 for HLLPP#14430

Merged
res-life merged 3 commits intoNVIDIA:mainfrom
res-life:fix-hllpp-precision-4
Mar 31, 2026
Merged

Enable precision 4 for HLLPP#14430
res-life merged 3 commits intoNVIDIA:mainfrom
res-life:fix-hllpp-precision-4

Conversation

@res-life
Copy link
Copy Markdown
Collaborator

@res-life res-life commented Mar 18, 2026

Fixes #12452.

Description

Enable precision 4 for HLLPP(Hyper Log Log Plus Plus) since cuDF fixed the bug NVIDIA/cuCollections#696

Depends on

Checklists

  • This PR has added documentation for new or modified features or behaviors.
  • This PR has added new tests or modified existing tests to cover new code paths.
    (Please explain in the PR description how the new code paths are tested, such as names of the new/existing tests that cover them.)
  • Performance testing has been performed and its results are added in the PR description. Or, an issue has been filed with a link in the PR description.

Signed-off-by: Chong Gao chongg@nvidia.com

Signed-off-by: Chong Gao <res_life@163.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 18, 2026

Greptile Summary

This PR enables GPU support for HLLPP (HyperLogLog++) at precision 4 by fixing the lower-bound guard in GpuOverrides.scala and removing the corresponding xfail markers from the integration tests. The change is directly tied to a cuCollections upstream bug fix (NVIDIA/cuCollections#696) and is a minimal, targeted fix.

Key changes:

  • GpuOverrides.scala: The precision guard is updated from precision <= 4 (blocking precision 4) to precision < 4 (allowing precision 4). The new supported range is [4, 14], matching the comment update. The error message is also improved to clearly state "out of range" and reference [4, 14].
  • hyper_log_log_plus_plus_test.py: xfail markers for relativeSD=0.3 (precision 4) are removed from both _relativeSD and the test_hllpp_precisions_groupby parametrize decorator, so precision 4 is now tested as a normal (expected-to-pass) case. The old conditional list-comprehension wrapping the mark is eliminated, simplifying the code.

No logic regressions are introduced; all previously supported precisions [5, 14] remain supported, and precision 4 is now added.

Confidence Score: 5/5

Safe to merge — the change is a one-line predicate fix with clean test coverage and no regressions to existing supported precisions.

Both changes are minimal and correct: the Scala guard predicate is fixed to include precision 4, the error message and comments are updated consistently, and the Python tests properly promote precision 4 from xfail to a first-class passing test. No new logic is added that could introduce regressions. All previously supported precisions [5, 14] are unaffected.

No files require special attention.

Important Files Changed

Filename Overview
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala Lower bound of precision check changed from <= 4 (exclusive of 4) to < 4 (inclusive of 4), expanding GPU-supported precision range from [5, 14] to [4, 14]; error message and comments updated accordingly.
integration_tests/src/main/python/hyper_log_log_plus_plus_test.py Removed xfail marks for precision 4 (relativeSD=0.3) from both _relativeSD list and test_hllpp_precisions_groupby parametrize decorator; simplifies test scaffolding and confirms the bug fix is testable end-to-end.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["HyperLogLogPlusPlus expr\n(tagExprForGpu)"] --> B["Compute precision\nfrom relativeSD"]
    B --> C{"precision < 4\nOR\nprecision > 14?"}
    C -- "Yes (out of range)" --> D["willNotWorkOnGpu\n(fallback to CPU)"]
    C -- "No (in range [4,14])" --> E["GPU supported\n(previously [5,14])"]
    E --> F["convertToGpu\nGpuHyperLogLogPlusPlus"]

    style E fill:#22c55e,color:#fff
    style D fill:#ef4444,color:#fff
Loading

Reviews (5): Last reviewed commit: "Format" | Re-trigger Greptile

….scala

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@res-life res-life self-assigned this Mar 18, 2026
@res-life
Copy link
Copy Markdown
Collaborator Author

build

@res-life res-life closed this Mar 30, 2026
@res-life res-life reopened this Mar 30, 2026
@res-life
Copy link
Copy Markdown
Collaborator Author

build

@nvauto
Copy link
Copy Markdown
Collaborator

nvauto commented Mar 30, 2026

NOTE: release/26.04 has been created from main. Please retarget your PR to release/26.04 if it should be included in the release.

Signed-off-by: Chong Gao <res_life@163.com>
@res-life res-life marked this pull request as draft March 31, 2026 02:31
@res-life
Copy link
Copy Markdown
Collaborator Author

build

@res-life res-life marked this pull request as ready for review March 31, 2026 08:09
@res-life res-life changed the title Fix HLLPP bug when precision is 4 Enable precision 4 for HLLPP Mar 31, 2026
@res-life res-life merged commit 0cd5dae into NVIDIA:main Mar 31, 2026
50 of 51 checks passed
@sameerz sameerz added the bug Something isn't working label Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] hyper_log_log_plus_plus_test.test_hllpp_precisions_groupby[0.3] failed in mismatch cpu and gpu result

4 participants