Skip to content

Conversation

@jeremylt
Copy link
Member

@jeremylt jeremylt commented Feb 7, 2025

Ok, there's two main reasons why a gen operator might fail

  1. too much shared memory needed to compile

  2. too many resources needed to run kernel

This PR adds a mechanism to automatically detect these issues and fall back to a shared operator.

@jeremylt
Copy link
Member Author

jeremylt commented Feb 7, 2025

@zatkins-dev can I get your thoughts on the overall approach? This should basically let people use /gpu/[cuda,hip] and all unsupported operators will automatically fallback

@zatkins-dev
Copy link
Collaborator

Yeah, I like the approach. I definitely think that it's good for users to be able to use /gpu/[cuda|hip] without worrying about support for whatever their use case is on a particular implementation

@jeremylt jeremylt force-pushed the jeremy/gen-rollback branch 4 times, most recently from 10cebd5 to d3a5b9e Compare February 7, 2025 20:11
Copy link
Collaborator

@zatkins-dev zatkins-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think this looks good, with a few minor things

@jeremylt jeremylt force-pushed the jeremy/gen-rollback branch from acff3cc to c9192ac Compare February 7, 2025 23:08
@jeremylt jeremylt merged commit ea41f46 into main Feb 7, 2025
27 of 28 checks passed
@jeremylt jeremylt deleted the jeremy/gen-rollback branch February 7, 2025 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants