NPU GenAI docs update: Image Generation#34077
NPU GenAI docs update: Image Generation#34077RyanMetcalfeInt8 wants to merge 2 commits intoopenvinotoolkit:masterfrom
Conversation
RyanMetcalfeInt8
requested review from
zKulesza
and removed request for
a team
|
|
||
| optimum-cli export openvino --trust-remote-code --model openai/whisper-base whisper-base-int8 --weight-format int8 | ||
|
|
||
|
|
There was a problem hiding this comment.
do we need this update in the new OpenVINO GenAI documentation? Their sources is in OpenVINO GenAI repo.
Regarding GenAI in OpenVINO docs, I am not sure that we are going to develop. We should rather move to the new GenAI documentation
There was a problem hiding this comment.
do we need this update in the new OpenVINO GenAI documentation? Their sources is in OpenVINO GenAI repo.
Yes, probably? The current documentation there looks very CPU / GPU centric.
Regarding GenAI in OpenVINO docs, I am not sure that we are going to develop
There was a request that I add the documentation here. @dmatveev can you chime in? If there is a plan to migrate this documentation to GenAI variant, fine. At the same time, what is the harm in improving this one?
| .. code-block:: python | ||
|
|
||
| # Run Text Encoder on CPU, UNet on NPU, and VAE decoder on GPU | ||
| pipe.compile("CPU", "NPU", "GPU", CACHE_DIR="cache") |
There was a problem hiding this comment.
quite interesting part because it is easy to mix device for each SD part. May be we should pass device with parameter name like "text_encoder_device="CPU"", etc.?
2 participants
This PR adds a section on best practices to run image generation models on NPU.