Test embedded bitcode path

[ymardoukhi]

Test added for capturing Broken function exception and its corresponding stdout message regarding Called function is not the same type as the call! due to automatic inlining of ccall function.

See issue #2664

[github-actions]

Your PR requires formatting changes to meet the project's style guidelines.
Please consider running Runic (git runic main) to apply these changes.

Click here to view the suggested changes.

diff --git a/test/embedded_bitcode.jl b/test/embedded_bitcode.jl
index c1dbf0ab..8895ca1f 100644
--- a/test/embedded_bitcode.jl
+++ b/test/embedded_bitcode.jl
@@ -43,7 +43,7 @@ tmp_so_file = joinpath(tmp_dir, "func.so")
 run(
     pipeline(
         `$(clang()) -x ir - -Xclang -no-opaque-pointers -O3 -fPIC -fembed-bitcode -shared -o $(tmp_so_file)`;
-        stdin=IOBuffer(FUNC_LLVM_IR)
+        stdin = IOBuffer(FUNC_LLVM_IR)
     )
 )
 
@@ -54,15 +54,19 @@ const fptr = Libdl.dlsym(lib, :func_wrap)
 function func_ccall(t::Float64, arr::AbstractVector{Float64})
     nitems = length(arr)
     bitsize = Base.elsize(arr)
-    GC.@preserve arr begin
+    return GC.@preserve arr begin
         excinfo = Ptr{Ptr{Cvoid}}(C_NULL)
         base::Ptr{Cdouble} = pointer(arr)
 
-        ccall(fptr, Cdouble,
-            (Ptr{Ptr{Cvoid}}, Cdouble, Ptr{Cvoid}, Ptr{Cvoid},
-                Clong, Clong, Ptr{Cdouble}, Clong, Clong),
+        ccall(
+            fptr, Cdouble,
+            (
+                Ptr{Ptr{Cvoid}}, Cdouble, Ptr{Cvoid}, Ptr{Cvoid},
+                Clong, Clong, Ptr{Cdouble}, Clong, Clong,
+            ),
             excinfo, t, C_NULL, C_NULL, nitems, bitsize,
-            base, nitems, nitems * bitsize)
+            base, nitems, nitems * bitsize
+        )
     end
 end
 
@@ -77,7 +81,7 @@ end
     err_llvmir = nothing
     b = @view a[1:5]
 
-    redirect_stdio(stdout=errstream, stderr=errstream, stdin=devnull) do
+    redirect_stdio(stdout = errstream, stderr = errstream, stdin = devnull) do
         try
             gradient(Reverse, func_ccall, Const(0.0), b)
         catch e

[giordano]

I seem to remember LLVM IR isn't very portable across major versions, I presume that'd be a potential problem here when testing different Julia (and llvm) versions?

[ymardoukhi]

I agree with you, and I also didn't feel this was the best test case.

The only way I could reproduce this error was with ccall inside a wrapper function. On the contrary, if I usellvmcall, I could get the gradient correctly. When I inspected the LLVM IR when passing a view of an array, Julia inlines the wrapper function with weakly typed subarray pointers, hence the mismatch at the callsite and the inner ccall.

Also, have a look at this comment in the original issue.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.50%. Comparing base (b02a7b5) to head (0d1f934).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2684      +/-   ##
==========================================
- Coverage   72.58%   72.50%   -0.08%     
==========================================
  Files          58       58              
  Lines       18739    18743       +4     
==========================================
- Hits        13602    13590      -12     
- Misses       5137     5153      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wsmoses]

is this even necessary here?

if you're starting from llvm anyways, why not just use llvmcall?

[ymardoukhi]

Surprisingly, I won't hit that execution path when using llvmcall. Have a look at this comment here
#2664 (comment)

The wrapper function around llvmcall handles the SubArray input with no issues, whereas the same wrapper function around ccall will fail to compute the gradient. I stopped short of investigating the core issue; the only clue I got from the emitted LLVM IR is that Julia inlines the wrapper function, which leads to a mismatch between the types at the callsite.

[vchuravy]

Could we use a small C library instead of using LLVM IR bitcode?

[ymardoukhi]

Could we use a small C library instead of using LLVM IR bitcode?

Great point! Will consider this.