Skip to content

C# Run Program on NPU (OnnxRuntime + DirectML + NPU)? #23375

Open
@Gusha-nye

Description

Hi, big guys!
Recently I tried to run phi3 on NPU inference using DirectML as backend EP under Onnxruntime framework, but I failed!
I loaded the model and tokenizer through the Model class and Tokenizer class, but from the C# API given in the official Onnxruntime docs(https://onnxruntime.ai/docs/genai/api/csharp.html), I can't find any API about being able to set up the NPU, can someone expand on this in detail to explain how to load the model onto the NPU?(It's best if you can explain it through a demo)
Or instead do you know if there are demos on loading Phi3 or other LLM models to NPU?
Looking forward to your reply!
Below is the code I used to load the model and tokenizer using the OnnxruntimeGenai-directml library and enter the question to be answered.

Image

Image

Image

Metadata

Assignees

No one assigned

    Labels

    api:CSharpissues related to the C# APIep:DMLissues related to the DirectML execution provider

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions