PackageCompiler fails with Flux on embedded ARM/no GPU

[cirobr]

Hello,

Have added detailed description on Julia Discourse regarding execution failure of precompiled apps when Flux is invoked and the hardware has no GPU. Please refer to this link for details: click here

It appears the root cause is tied to the fact that CUDA package (and a number of CUDA artifacts) are part of Flux' dependency list.

The issue became a roadblock to implement Julia ML apps on embedded ARM. Perhaps the project may want to consider a GPU-agnostic approach? Simple ML apps may need only multi-core (no GPU) to run.

Thanks in advance.

[ToucheSir]

Looking through the Discourse thread, I think the problem is not specific to CUDA.jl but with something odd about your environment. I'd recommend reporting it with the usual JuliaGPU help channels.

To your comments on the same thread, the reason Flux depends on CUDA by default is the same reason PyTorch or TensorFlow do: it was the only game in town and there isn't (or wasn't) any other way to add GPU functionality given limitations of the package manager. We go to great lengths to make sure that the CUDA functionality is only activated when you ask for it, but we can't control what happens at install time. It may just be that your environment is sufficiently weird that it hits an edge case the CUDA.jl maintainers weren't thinking of, because they also go to great lengths to make sure CUDA.jl doesn't try to load GPU functionality unnecessarily on platforms which don't support it.

Perhaps the project may want to consider a GPU-agnostic approach?

See above. Note that Flux does support AMDGPU.jl through the very recent package extensions mechanism, but because its so new and we need to maintain backwards compat (with earlier versions of Flux and Julia), the CUDA.jl default path remains. It's not for lack of trying too: we want to do the same for CUDA.jl eventually, but unless someone figures out a way around the roadblocks in FluxML/NNlib.jl#492 (comment) we're presently stuck. Perhaps you know people who could help with that?

[cirobr]

Thanks for the insight. The environment has just Flux on it, and the code is as long as "using Flux; println()". Machine is plain Oracle instance of AArch64 + Ubuntu 22.04, no extras. Of course, the issue was found on a much more complex environment and much more complex code. The reporting of the issue reflects personal effort by removing non relevant code to the root cause analysis and to collaborate on the improvement of such a great package as Flux is.

I'll try other avenues, including those suggestions.

Thanks.

[ToucheSir]

I would try installing and importing CUDA.jl in a fresh environment. No calling functions, just import/using. If you can get that working, you should be able to use Flux without much issue.

[CarloLucibello]

CUDA is now supported through extensions, so this should be fixed

[cirobr]

CUDA is now supported through extensions, so this should be fixed

Thanks for the feedback. I've run my code against the latest updates. Problem is indeed gone.