Skip to content

BUILD.md

Latest commit

 

History

History
166 lines (116 loc) · 4.78 KB

BUILD.md

File metadata and controls

166 lines (116 loc) · 4.78 KB

Build OpenVINO™ GenAI Node.js bindings

  1. Build and install OpenVINO from sources following the instructions. In this step, we need to build the OpenVINO Runtime that is required for OpenVINO GenAI. We don't need to configure any environment variables to create JS buildings here. The path to the OpenVINO install directory is referred as <INSTALL_DIR> throughout the document.

  2. Clone OpenVINO GenAI repository and init submodules:

    git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai

  3. Set up the environment:

    Option 1 - using OpenVINO setupvars script:

    Linux and macOS:

    source <INSTALL_DIR>/setupvars.sh

    Windows Command Prompt:

    call <INSTALL_DIR>\setupvars.bat

    Windows PowerShell:

    . <INSTALL_DIR>/setupvars.ps1

    Option 2 - setting environment variables manually:

    Linux:

    export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH

    macOS:

    export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export DYLD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH

    Windows Command Prompt:

    set OpenVINO_DIR=<INSTALL_DIR>\runtime
set PYTHONPATH=<INSTALL_DIR>\python;%CD%\build;%PYTHONPATH%
set OPENVINO_LIB_PATHS=<INSTALL_DIR>\bin\intel64\Release;%OPENVINO_LIB_PATHS%
set PATH=%OPENVINO_LIB_PATHS%;%PATH%

    Windows PowerShell:

    $env:OpenVINO_DIR = "<INSTALL_DIR>\runtime"
$env:PYTHONPATH = "<INSTALL_DIR>\python;$PWD\build;$env:PYTHONPATH"
$env:OPENVINO_LIB_PATHS = "<INSTALL_DIR>\bin\intel64\Release;$env:OPENVINO_LIB_PATHS"
$env:PATH = "$env:OPENVINO_LIB_PATHS;$env:PATH"

  4. Build OpenVINO GenAI Node.js bindings

    cmake -DCMAKE_BUILD_TYPE=Release \
    -DENABLE_JS=ON -DCPACK_GENERATOR=NPM \
    -S . -B ./build
cmake --build ./build --config Release -j
cmake --install ./build/ --config Release --prefix ./src/js/bin

Install and Run

Requirements

  • Node.js v21+

Build Bindings

Build OpenVINO™ GenAI JavaScript Bindings from sources following the instructions.

Using the package from your project

Since the OpenVINO GenAI NodeJS package depends on the OpenVINO NodeJS package, these packages must be of the same version. If you intend to use one of the released versions, please check and install the correct version of the openvino-node package in your project. If you want to use an unstable version of the OpenVINO GenAI NodeJS package, you will also need to build and use OpenVINO™ JavaScript Bindings from source following the instructions to use it correctly.

Then you can use OpenVINO™ GenAI JavaScript Bindings in one of the following ways:

Option 1 - using npm:

To use this package locally use npm link in src/js/ directory and npm link openvino-genai-node in the folder where you want to add this package as a dependency

Option 2 - using package.json:

Add the openvino-genai-node package manually by specifying the path to the src/js/ directory in your package.json:

"openvino-genai-node": "file:*path-to-current-directory*"

Verify the installation:

node -e "const { LLMPipeline } = require('openvino-genai-node'); console.log(LLMPipeline);"

Running Tests

Set Up the Python Environment

To run the JavaScript tests, you need Python 3.10+ and the following Python packages:

  • openvino-genai
  • optimum-intel — for model conversion

Install the required Python dependencies:

python -m pip install openvino-genai optimum-intel

Running the Tests

Run the following npm script to prepare models and run all JavaScript tests:

npm test

Running Tests Manually

If you don't want to set up the Python environment, or you would like to run the JavaScript tests manually, follow these steps:

  1. Obtain a model (for example, export a model to the OpenVINO IR format from Hugging Face). See the Model Preparation Guide for more details.

  2. Set the environment variables with paths to the converted models.

Test Environment Variables

  • LLM_PATH - Path to the LLM model
  • VLM_PATH - Path to the vision-language model
  • EMBEDDING_MODEL_PATH - Path to the text embedding model
  • RERANK_MODEL_PATH - Path to the reranking model
  1. Run the tests:
node --test ./tests/*.test.js