[CI] Enable Java test in CI workflow

[rhdong]

No description provided.

[jameslamb]

@rhdong could you please put this PR into draft until you're ready for reviews? That'd reduce the notifications reviewers are getting, and help them understand when it's time to come review.

[rhdong]

@rhdong could you please put this PR into draft until you're ready for reviews? That'd reduce the notifications reviewers are getting, and help them understand when it's time to come review.

Thanks for the reminder! I’ve marked the PR as draft now.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[rhdong]

/ok to test

[rhdong]

/ok to test

[rhdong]

/ok to test

[raydouglass]

Does this actually run tests requiring a GPU? If yes, can be changed so that only the PR & test workflows run tests?

If it doesn't run tests, then would be good update this line and also to switch the node type for the build workflow from gpu-l4-latest-1 to a CPU runner.

[rhdong]

It will run tests.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[rhdong]

/ok to test

[copy-pr-bot]

/ok to test

@rhdong, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

[rhdong]

/ok to test 79a477e

[narangvivek10]

The reason for the Java build failure is that the script could not find jextract (needed for generating Panama bindings before Java build and tests). The CI servers need to have jextract preinstalled from https://jdk.java.net/jextract

@cjnolet @rhdong

[chatman]

@narangvivek10 @rhdong @cjnolet I've committed a fix [0] to download jextract automatically if not already installed. Reason for doing this is that jextract doesn't have a .deb or apt package for Ubuntu, and hence it the download of jextract needs to be scripted anyway.

[0] - 570fa2a in #831

[rhdong]

/ok to test 04f0bba

[chatman]

@narangvivek10 The jextract process failed due to:

c_api.h:19:10: error: 'cuda_runtime.h' file not found
fatal: Unexpected exception org.openjdk.jextract.clang.TypeLayoutError: Invalid. segment: org.openjdk.jextract.clang.Type@cc813a2e, fieldName: n_probes occurred
Jextract encountered issues (returned value 5)
Bindings generation did not complete normally (returned value 5)
Forcing this build process to abort

Any ideas where the cuda_runtime.h will be found?

[chatman]

@rhdong We have attempted to find the CUDA_HOME diras:
CUDA_HOME=$(which nvcc | cut -d/ -f-4)

And then tried to add the $CUDA_HOME/include dir to the include paths. Any ideas if this was the problem and is there a better way?

[chatman]

Also, I see the following:

2025-04-24T16:03:41.9414077Z Forcing this build process to abort
2025-04-24T16:03:41.9513171Z 
2025-04-24T16:03:41.9516434Z �[32mRAPIDS logger�[0m » [04/24/25 16:03:41]
2025-04-24T16:03:41.9517878Z �[32m┌─────────────────────────────────────────────────────────────────────────────┐�[0m
2025-04-24T16:03:41.9519658Z �[32m|    Initial Java build & test failed. Retrying with 'mvn clean verify -X'    |�[0m
2025-04-24T16:03:41.9521203Z �[32m└─────────────────────────────────────────────────────────────────────────────┘�[0m
2025-04-24T16:03:41.9522013Z 

I think this retrying is not necessary, and not correct either, since here the failure was in a step even before Maven is invoked (failure is in the generate-bindings.sh file). Due to this retry, the logs are polluted with a lot of symbol not found issues via Maven, and it masks the original problem that the Panama bindings were not properly generated.

[chatman]

@rhdong I've made the following changes:

• Debug printing of the CUDA_HOME variable, and the contents of the $CUDA_HOME/include
• If there's no include inside CUDA_HOME, try CUDA_HOME to be /usr/local/cuda

https://github.com/rapidsai/cuvs/pull/831/files/570fa2a7a792b39cb70c4ff1232661481ba8ecaa..306229d29b0123bc7f6e72adca6e7d155047f528

I'm hoping it will make things work. Can you please merge that and retest here?

[rhdong]

@rhdong We have attempted to find the CUDA_HOME diras: CUDA_HOME=$(which nvcc | cut -d/ -f-4)

And then tried to add the $CUDA_HOME/include dir to the include paths. Any ideas if this was the problem and is there a better way?

Hi @chatman @narangvivek10 , The docker image is rapidsai/ci-conda:latest, the cuda includes will be installed when creating the conda env test, as my local experiment, the cuda_runtime.h is in /opt/conda/envs/test/targets/x86_64-linux/include/cuda_runtime.h , the test env name is test. So I fixed it by the top commit, and the new error comes up:

jextract-22/bin/jextract.ps1
jextract downloaded to /cuvs/java/jextract-22
common.h:21:10: error: 'dlpack/dlpack.h' file not found
fatal: Unexpected exception org.openjdk.jextract.clang.TypeLayoutError: Invalid. segment: org.openjdk.jextract.clang.Type@1c99c732, fieldName: addr occurred
Jextract encountered issues (returned value 5)
Bindings generation did not complete normally (returned value 5)
Forcing this build process to abort

RAPIDS logger » [04/24/25 20:26:13]

[rhdong]

Also, I see the following:

2025-04-24T16:03:41.9414077Z Forcing this build process to abort
2025-04-24T16:03:41.9513171Z 
2025-04-24T16:03:41.9516434Z �[32mRAPIDS logger�[0m » [04/24/25 16:03:41]
2025-04-24T16:03:41.9517878Z �[32m┌─────────────────────────────────────────────────────────────────────────────┐�[0m
2025-04-24T16:03:41.9519658Z �[32m|    Initial Java build & test failed. Retrying with 'mvn clean verify -X'    |�[0m
2025-04-24T16:03:41.9521203Z �[32m└─────────────────────────────────────────────────────────────────────────────┘�[0m
2025-04-24T16:03:41.9522013Z 

I think this retrying is not necessary, and not correct either, since here the failure was in a step even before Maven is invoked (failure is in the generate-bindings.sh file). Due to this retry, the logs are polluted with a lot of symbol not found issues via Maven, and it masks the original problem that the Panama bindings were not properly generated.

Yeah, I agree, the main goal of the retry is to debug the issue of HNSW(has been resolved). The retry only happens when test fails. We can remove it at last.

[rhdong]

/ok to test cb5d1ba