Skip to content

add Metal extension for batched_mul#614

Draft
mcabbott wants to merge 1 commit intoFluxML:masterfrom
mcabbott:metal1
Draft

add Metal extension for batched_mul#614
mcabbott wants to merge 1 commit intoFluxML:masterfrom
mcabbott:metal1

Conversation

@mcabbott
Copy link
Member

Closes #581

PR Checklist

  • Tests are added
  • Documentation, if applicable

@chengchingwen
Copy link
Member

JuliaGPU/Metal.jl#381

@mcabbott
Copy link
Member Author

mcabbott commented Dec 1, 2024

Thanks I hadn't seen that.

Got a wrong answer in this test on CI (tiny arrays though) but didn't investigate further:

https://github.com/FluxML/NNlib.jl/pull/614/files#diff-df0d2a37225f09d22727651479dc1cd59f2b8358f4eb1e2be98c9b04e215be86R31-R34

@chengchingwen
Copy link
Member

IIRC the bug might also happen on tiny arrays if it's within a sequence of calls. It's really hard to detect though.

@CarloLucibello
Copy link
Member

Can this be revived? If you don't have time I can do it. Merging this would also fix dot_product_attention for Metal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

batched_mul doesn't work with MtlArrays

3 participants