Skip to content

[KNOWN BUG] gguf + GPU AOTI Inference bug due to PT version, fix in progress #1423

New issue
New issue
Closed
Closed
[KNOWN BUG] gguf + GPU AOTI Inference bug due to PT version, fix in progress#1423
Assignees
Jack-Khuu
Labels
CI InfraIssues related to CI infrastructure and setupCompile / AOTIIssues related to AOT Inductor and torch compileKnown GapsThese are known Gaps/Issues/Bug items in torchchatQuantizationIssues related to Quantization or torchaobugSomething isn't workingtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
@Jack-Khuu

Description

@Jack-Khuu
Jack-Khuu
opened on Dec 14, 2024

🐛 Describe the bug

As of #1367, torchchat/main is failing 3-5 CI jobs related to GPU AOTI inference and GGUF inference

GPU AOTI inference will be fixed with a pinbump to pytorch/pytorch#143236
GGUF AO bug is being addressed in #1404

Versions

bb72b09

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

CI InfraIssues related to CI infrastructure and setupCompile / AOTIIssues related to AOT Inductor and torch compileKnown GapsThese are known Gaps/Issues/Bug items in torchchatQuantizationIssues related to Quantization or torchaobugSomething isn't workingtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions