Fixes NVIDIA#14148. This commit adds support for the new BloomFilter v2 format that was added in Apache Spark 4.1.1 (via apache/spark@a08d8b0). The v1 format used INT32s for bit index calculation. When the number of items in the bloom-filter approaches INT_MAX, one sees a higher rate of collisions. The v2 format uses INT64 values for bit index calculations, allowing the full bit space to be addressed. Apparently, this reduces the false positive rates for large filters. Before the fix in this current PR was applied to spark-rapids, only certain bloom filter join tests would fail against Apache Spark 4.1.1; specifically: 1. `test_bloom_filter_join_cpu_build`, where the bloom filter is built on CPU and then probed on GPU. This failed because the CPU would produce a v2 filter that couldn't be treated as a v1 format on GPU. 2. `test_bloom_filter_join_split_cpu_build`, where the bloom filter is partially aggregated on CPU, then merged on GPU. Again, the GPU-side merging expected v1 format, while the CPU produced v2. Note that `test_bloom_filter_join_cpu_probe` and `test_bloom_filter_join` did not actually fail on 4.1.1. That is because: 1. `test_bloom_filter_join_cpu_probe` tests CPU probing, which supports v1 and v2 flexibly. 2. `test_bloom_filter_join` tests both the build and probe jointly being either on CPU, or GPU. The CPU ran v2 format, the GPU ran v1. Both produce the same query results, albeit with different formats. The fix in this commit allows for v1 and v2 formats to be jointly supported on GPU, depending on the Spark version. Signed-off-by: MithunR <mithunr@nvidia.com>

Commits

Commits on Mar 11, 2026

BloomFilter v2 support