Enable Ahead-of-Time Compilation by hiding the runtime functions in the GLOBAL_METHOD_TABLE

[apozharski]

As discussed in JuliaGPU/CUDA.jl#2998 and #611 currently GPUCompiler.jl leaks nonexistant gpu_* llvm functions into the cpu cache making ahead of time compilation impossible for any package that uses it.

I am currently fixing this by moving these runtime methods into the method table defined in the GPUCompiler module and having the CPU versions throw errors as is done in CUDA.jl. This feels like somewhat of a hack, however, it seems to work and without a better understanding of what this might break it seems to be the simplest solution.

[github-actions]

Your PR requires formatting changes to meet the project's style guidelines.
Please consider running Runic (git runic master) to apply these changes.

Click here to view the suggested changes.

diff --git a/src/utils.jl b/src/utils.jl
index 8242e5a..9f55e9c 100644
--- a/src/utils.jl
+++ b/src/utils.jl
@@ -196,12 +196,14 @@ macro device_function(ex)
         error("This function is not intended for use on the CPU")
     end
 
-    esc(quote
-        $(combinedef(def))
+    return esc(
+        quote
+            $(combinedef(def))
 
-        # NOTE: no use of `@consistent_overlay` here because the regular function errors
-        Base.Experimental.@overlay($(GPUCompiler).GLOBAL_METHOD_TABLE, $ex)
-    end)
+            # NOTE: no use of `@consistent_overlay` here because the regular function errors
+            Base.Experimental.@overlay($(GPUCompiler).GLOBAL_METHOD_TABLE, $ex)
+        end
+    )
 end
 
 

[KSepetanc]

Loaded both forked CUDA.jl and this PR and tried to compile my full code and got error:

Stacktrace is massive so I copied first several lines.

ERROR: LoadError: Invalid return type for runtime function 'box_bool': expected LLVM.PointerType(ptr addrspace(10)), got LLVM.VoidType(void)
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:44
  [2] emit_function!(mod::LLVM.Module, config::GPUCompiler.CompilerConfig{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}, f::Type, method::GPUCompiler.Runtime.RuntimeMethodInstance)
    @ GPUCompiler C:\Users\karlo\.julia\packages\GPUCompiler\vRm9U\src\rtlib.jl:81
  [3] build_runtime(job::GPUCompiler.CompilerJob)
    @ GPUCompiler C:\Users\karlo\.julia\packages\GPUCompiler\vRm9U\src\rtlib.jl:117
  [4] (::GPUCompiler.var"#load_runtime##0#load_runtime##1"{GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}})()
    @ GPUCompiler C:\Users\karlo\.julia\packages\GPUCompiler\vRm9U\src\rtlib.jl:159
  [5] lock(f::GPUCompiler.var"#load_runtime##0#load_runtime##1"{GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget, CUDA.CUDACompilerParams}}, l::ReentrantLock)
    @ Base .\lock.jl:335

Otherwise, I could compile GPUCompiler in image.

Do you have idea where could this come from?

[apozharski]

@KSepetanc yep, I caught that in the tests for the PR as well (somehow they were passing for me locally, but I suspect that was just poor environment management on my part).

Unsurprisingly, my hack seems to break things in GPUCompiler at runtime. I have some ideas, namely the fact that i am replacing the stub llvm call with an exception in the cpu-cache which simply returns void. It has taken me a bit to get something I can test with (since my machine at home has a quite broken CUDA installation 😅 ) but it seems I am able to test with OpenCL so hopefully I can get something a bit less hacky working soon depending on how much time I will have over the holidays.

I will turn this PR to a draft until then.

[KSepetanc]

@apozharski are you using CUDA 590 driver branch (it is CUDA 13.1)? I have seen maintainers are preparing support for it, but last I checked a few days ago it still was not released. Without knowing more about your system, I presume you just need to downgrade to 580 series driver that comes with CUDA 13.0. I had this issue too.

I will soon have more questions as it seems that more fixes are needed than just GPUCompiler.jl and CUDA.jl to AOT compile MadNLPGPU which I need, but it is still WIP so I will wait a bit.