Skip to content

Does it support running super large models on AMD GPUs? #276

Description

@nodepend-code

The model can output normally when run on the CPU using deepseek_r1_distill_llama_70b_q40. But using a GPU (AMD GPU) will result in abnormal output.May I ask how to adjust the GPU to ensure normal output.(GPU model:AMD RYZEN AI MAX+ 395 w/ Radeon 8060S)
Use GPU:

Image Image

Use CPU:

Image Image

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions