[Torch] Updated recipe

[stemann]

Updated Torch recipe

• Changed to provide lazy artifacts.
• Replaced CUDA_full sources with CUDA module dependencies (and similar for CUDA 11.3).
• Updated cuda-part of script accordingly.
• Added xcrun executable for macOS which can handle --show-sdk-path.
• Using LLVM v13 (on macOS, v17 has issues).
• Adding cuda="none" tag to non-CUDA platforms with matching CUDA platforms.

Added comment to fancy_toys.jl should_build_platform

should_build_platform returns true, e.g. for "x86_64-linux-gnu-cxx11" when ARGS = ["x86_64-linux-gnu-cxx11-cuda+10.2"].

Added NVTX RuntimeDependency

[stemann]

@maleadt Does this look alright to you, wrt. CUDA aspects?

I.e., using lazy artifacts, and building for cuda="none", and cuda="10.2" and cuda="11.3", x86_64-linux-gnu platforms.

With the goal being to have Torch.jl be automatically loadable (also) on x86_64-linux-gnu (for General registration etc.) - while also selecting CUDA artifacts if appropriate (in accordance with the CUDA platform augmentation implementation).

[maleadt]

I think that looks right. Normally you'd use CUDA.is_supported to determine whether to use a cuda+none build, though here that happens not to matter because of not having a Windows build. But it might still be better to do it the "proper" way.

That said, I'm not entirely convinced anymore that unified CPU+GPU artifacts are the best way forward (as opposed to split packages). There's some fragility e.g. with the none tag, but also it removes the ability to use a non-GPU build as soon as you have the NVIDIA driver installed. I'll leave that up to you to decide, though.

[stemann]

Great, thanks!

I was a little unhappy with the handling around those lines anyway (the double platforms, cuda_platforms, loop), so I've adopted CUDA.is_supported (though I also had to add special handling for aarch64).

I agree that it might be more right to have separate packages... but keeping the current approach for now (to move things forward).

[stemann]

Hoping for the materialisation of BinaryBuilder2 to bring pkg extensions to Yggdrasil to handle CUDA-variants of JLLs...

[stemann]

@maleadt So, since Torch_jll depends on CUDNN_jll, which depends on CUDA_Runtime_jll, which in turn depends on CUDA_Driver_jll, is it impossible to get the cuda="none" artifact (CUDA_Driver_jll finds a "Forward-compatible driver") or is there a trick I've missed?

Tried setting version="none" for CUDA_Runtime_jll , but that does not seem to be anything other than "imagined API" :-)

Debugging in https://github.com/stemann/Torch.jl/actions/runs/12233357984/job/34120277156

[maleadt]

is it impossible to get the cuda="none" artifact (CUDA_Driver_jll finds a "Forward-compatible driver")

Correct; that's what I meant with #9785 (comment)

it removes the ability to use a non-GPU build as soon as you have the NVIDIA driver installed