Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
| @@ -0,0 +1,22 @@ | |||
| # VLM NPU Notebook | |||
|
|
|||
| This folder contains the `vlm-npu.ipynb` notebook for running vision-language models with OpenVINO. The models used in the notebook are specifically optimized to work on Intel NPU, though it may also work on CPU & GPU as well. This notebook is self-sufficient and install all the packages required to run the models within a virtual environment. The notebook downloads the models from HuggingFace (some models might require HF token), quantize and convert to OpenVINO IR format using optimum-cli and then pass an image and a prompt to generate the response using openvino-genai. | |||
There was a problem hiding this comment.
"self-contained" instead of "self-sufficient"
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "## login to huggingfacehub to get access to pretrained model \n", |
There was a problem hiding this comment.
You might want to add a note, similar to other notebooks (e.g. "https://github.com/openvinotoolkit/openvino_notebooks/blob/f0289d39c7881ac7e8451026f1e5ca8a5a0fc4b9/notebooks/llm-chatbot/llm-chatbot.ipynb"):
Note: run model with demo, you will need to accept license agreement. You must be a registered user in Hugging Face Hub. Please visit [HuggingFace model card](???, carefully read terms of usage and click accept button. You will need to use an access token for the code below to run. For more information on access tokens, refer to this section of the documentation. You can login on Hugging Face Hub in notebook environment, using following code:
2 participants
This notebook consists of methodology and environment setup to run multiple VLM models on PTL NPU (NPU5).