Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable non-standard OpenBLAS extensions v2 #1572

Merged
merged 51 commits into from
Feb 23, 2025
Merged

Conversation

blueberry
Copy link
Contributor

The second try of the #1571
The macos build passed, so I'm opening the PR so you can see whether it now works with the changes you suggested (mapCommon).

I've tried to build everything on my machine, but the openblas build fails (independently of this change) due to some LLVM error that is probably related to some mismatch in I-don-t-know-which combination of native compilers, so I couldn't reach the Java gen stage and see how the final opencl_java class looks like. That's why I'm opening this PR. (btw I've tried the docker way too, as per JavaCPP guide, but then some internal docker stuff didn't work. I couldn't solve it right now).

@blueberry
Copy link
Contributor Author

FYI the following fail was due to a download (networking) fail, not related to the relevant code itself.

openblas / macosx-x86_64 (pull_request)

@saudet
Copy link
Member

saudet commented Jan 27, 2025

I think I got it working. Could you check that the functions you want are all there?

@blueberry
Copy link
Contributor Author

As soon as it appears in the snapshot repo (or is there a way to check right now?)

@saudet
Copy link
Member

saudet commented Jan 27, 2025

@blueberry
Copy link
Contributor Author

blueberry commented Feb 19, 2025

Now the logical part of the code seems to be working everywhere, but the build fails on Windows for the mysterious reason, after 6 hours. Here's the relevant part at the end of the log. I don't have a windows machine locally where I can experiment with this. Moreover, I didn't manage to build openblas itself on my linux machine. I managed to use the openblas installed by Arch linux. Any ideas what might be the stopper?

2025-02-19T04:03:58.4572042Z [INFO] --- javacpp:1.5.12-SNAPSHOT:build (javacpp-cppbuild-clean) @ openblas ---
2025-02-19T04:03:58.4592803Z [INFO] Detected platform "windows-x86_64"
2025-02-19T04:03:58.4606695Z [INFO] Building platform "windows-x86_64"
2025-02-19T04:03:58.4613408Z [INFO] bash cppbuild.sh clean openblas 
2025-02-19T04:03:58.6310128Z Detected platform "windows-x86_64"
2025-02-19T04:03:58.6328578Z Building platform "windows-x86_64"
2025-02-19T04:03:58.6418238Z Cleaning "openblas"
2025-02-19T04:03:58.7095979Z In file included from generic/ztrmm_uncopy_4.c:40:
2025-02-19T04:03:58.7102908Z ../common.h:423:10: fatal error: common_x86_64.h: No such file or directory
2025-02-19T04:03:58.7396978Z compilation terminated.
2025-02-19T04:03:58.7437143Z make[1]: *** [Makefile.L3:1826: ztrmm_ounncopy_SANDYBRIDGE.obj] Error 1
2025-02-19T04:03:58.7438934Z make[1]: *** Waiting for unfinished jobs....
2025-02-19T04:03:59.5684628Z 2025-02-19 04:03:59
2025-02-19T04:03:59.5689911Z Full thread dump OpenJDK 64-Bit Server VM (25.442-b06 mixed mode):

@blueberry
Copy link
Contributor Author

@saudet Maybe you'd prefer to disable automatic build runs on this PR until I figure it out.

@saudet
Copy link
Member

saudet commented Feb 20, 2025

I'm not sure what you're trying to do, but the build on Windows is failing because of these 2 functions, not cblas or lapack:

2025-02-19T19:21:22.2210191Z jniopenblas_full.obj : error LNK2001: unresolved external symbol openblas_set_num_threads_local
2025-02-19T19:21:22.2376894Z jniopenblas_full.obj : error LNK2001: unresolved external symbol openblas_set_threads_callback_function
2025-02-19T19:21:22.2377579Z jniopenblas_full.dll : fatal error LNK1120: 2 unresolved externals
2025-02-19T19:21:22.2746229Z [INFO] --------------------------------------------------------------------

@blueberry
Copy link
Contributor Author

I'm not sure what you're trying to do, bu the build on Windows is failing because of these 2 functions, not cblas or lapack:

2025-02-19T19:21:22.2210191Z jniopenblas_full.obj : error LNK2001: unresolved external symbol openblas_set_num_threads_local
2025-02-19T19:21:22.2376894Z jniopenblas_full.obj : error LNK2001: unresolved external symbol openblas_set_threads_callback_function
2025-02-19T19:21:22.2377579Z jniopenblas_full.dll : fatal error LNK1120: 2 unresolved externals
2025-02-19T19:21:22.2746229Z [INFO] --------------------------------------------------------------------

Thank for that pointer! I was not able to find the exact function that was the culprit, so I was trying to find that out by guesses. I'll remove these two and then I hope it will work!

@blueberry
Copy link
Contributor Author

@saudet Good news! The last commit builds on all platforms!

I can see that a few builds (windows is not among them) are still waiting in the queue due to previous actions still running (if you can, maybe you'd want to cancel them anyway), but on my repository, all build actions completed successfully.

Thanks for helping with this!

@blueberry blueberry requested a review from saudet February 22, 2025 18:12
@saudet
Copy link
Member

saudet commented Feb 23, 2025

Looks alright, but could you double-check that all the functions you're interested in are there?

@blueberry
Copy link
Contributor Author

Thanks! I've checked several and they are in the generated java file for openblas_full.

@saudet saudet merged commit e6f401d into bytedeco:master Feb 23, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants