Skip to content

Commit 90b4dee

Browse files
committed
Add quick start to readme
1 parent 3755915 commit 90b4dee

File tree

1 file changed

+18
-0
lines changed

1 file changed

+18
-0
lines changed

README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,24 @@ Explore blogs to setup your first hands-on experience with OpenVINO GenAI:
3939

4040
<a id="ai-scenarios"></a>
4141

42+
## Quick Start
43+
44+
1. Install OpenVINO GenAI from PyPI:
45+
```sh
46+
pip install openvino-genai
47+
```
48+
2. Obtain model, e.g. export model to OpenVINO IR format from Hugging Face (see [Model Preparation Guide](https://openvinotoolkit.github.io/openvino.genai/docs/category/model-preparation) for more details):
49+
```sh
50+
optimum-cli export openvino --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --weight-format int4 --trust-remote-code TinyLlama_1_1b_v1_ov
51+
```
52+
3. Run inference:
53+
```python
54+
import openvino_genai as ov_genai
55+
56+
pipe = ov_genai.LLMPipeline("TinyLlama_1_1b_v1_ov", "CPU") # Use CPU or GPU as devices without any other code change
57+
print(pipe.generate("What is OpenVINO?", max_new_tokens=100))
58+
```
59+
4260
## Supported Generative AI Scenarios
4361

4462
OpenVINO™ GenAI library provides very lightweight C++ and Python APIs to run the following Generative AI Scenarios:

0 commit comments

Comments
 (0)