Does it make sense to add a `cuda` function mirroring `@cuda` functionality?

[marius311]

I'm wondering if it makes sense to add something where you can launch kernels by calling a function (called say cuda) rather than using the @cuda macro? Or maybe this has already been discussed elsewhere? I can see two reasons for it:

1. Being a macro, @cuda must exist at compile time, whereas a function need only exist at run-time. The former can be restrictive if (like me) you write packages which lazily load CUDA-specific code via Requires.jl so that CUDA is optional for users. In this case, you cannot use @cuda outside of the lazily loaded code even if your kernel code is otherwise generic, but with a function you could do that.

2. It could let you write some nice Julian-looking CUDA calls, e.g.

x = CuArrays.fill(1f0,10)
cuda(x) do x
    for I in eachindex(x)
        x[I] += 1
    end
end

Another advantage is that the implementation seems trivial, something like:

function cuda(f, args...; threads=256)
    @cuda threads=threads f(args...)
end

I could well be missing a reason why this won't work, but if not, let me know, I could take a stab at a PR.

[maleadt]

We parse certain kwargs during macro expansion, and forward them to the compiler. This isn't possible using a function, and we'd need a different scheme for that (and we don't have access to inferred constant values during expansion of the generator either).

2\. It could let you write some nice Julian-looking CUDA calls, e.g.

x = CuArrays.fill(1f0,10)
cuda(x) do x
    for I in eachindex(x)
        x[I] += 1
    end
end

That's going to be slow, right? You're using a single thread to iterate elements.

[marius311]

We parse certain kwargs during macro expansion, and forward them to the compiler. This isn't possible using a function, and we'd need a different scheme for that

I see, thanks. Sounds like maybe it'd be possible if cuda was a generated function?

That's going to be slow, right?

Yea, that was just meant as a simple syntax example.

[maleadt]

I see, thanks. Sounds like maybe it'd be possible if cuda was a generated function?

No, unless you expect users to Val(...) arguments to the compiler, which is very user unfriendly.

[marius311]

I was thinking something like the following, indeed you do need a helper function to Val(...) the arguments but the user never sees it:

function cuda(f, args...; threads=256)
    _cuda(Val(threads), f, args...)
end

@generated function _cuda(::Val{threads}, f, args...) where {threads}
    quote
        @cuda threads=$threads f(args...)
    end
end

I'm not sure if threads is one of the things that needs to be known at compile time, but in any case the above serves as a proof of concept (I did just check it actually runs).