Version 1.0.0 Release
In-Game Inferencing (NVIGI) is an open-sourced cross platform solution that simplifies integration of the latest NVIDIA and other provider's technologies into applications and games. This framework allows developers to easily implement one single integration and enable multiple technologies supported by the hardware vendor or cloud provider. Supported technologies include AI inference but, due to the generic nature of the NVIGI SDK, can be expanded to graphics or any other field of interest.
For high-level details of NVIGI, as well as links to the official downloads for the product, see NVIDIA In-Game Inferencing
This directory contains the main AI plugins for integrating In-Game Inferencing-based AI workflows into your application.
IMPORTANT: For important changes and bug fixes included in the current release, please see the release notes at the end of this document BEFORE use.
- NVIDIA RTX 30x0/A6000 series or (preferably) RTX 4080/4090 or RTX 5080/5090 with a minimum of 8GB and recommendation of 12GB VRAM.
NOTE: Some plugins only support RTX 40x0 and above, and will not be available on RTX 30x0.
- Win10 20H1 (version 2004 - 10.0.19041) or newer
- Install graphics driver r555.85 or newer
- Install VS Code or VS2019/VS2022 with SDK 10.0.19041+
NVIGI consists of several independent components, of which this SDK directory is only one. Normally, several of these components are used together, and in many cases, developers will actually receive a combined, "standard-layout" pack that is easier to use out of the box. This SDK directory contains plugins focused upon AI inference, especially speech recognition, large language models, and embeddings. It relies upon and requires the NVIGI core APIs, which are shipped as nvigi_core
Plugin Development Kit (PDK), either as:
- A downloaded and unzipped stand-alone NVIGI Core PDK binary pack
- A locally cloned and built copy of the
nvigi_core
git repository (generally located in a git repo parallel to the one containing this SDK plugins repo). In general, simply building all configurations of Core from source and then using the NVIGI-Core source tree root as the location of the PDK should work. See the NVIGI Core README for details on how one can build and package a built NVIGI Core source repo into a PDK.
NVIGI core is documented in detail in its own documentation, which should be available in either tree. Developers should read the documentation in nvigi_core
to gain a basic understanding of the overall architecture into which the plugins in this source tree plug into.
As noted, building the NVIGI plugins SDK from source requires access to a fully-built local copy of the NVIGI Core PDK. We will call the location of this PDK <NVIGI_CORE>
.
In the following instructions,
<NVIGI_CORE>
should be replaced with the fully path to the NVIGI Core PDK, either pulled pre-built or built from git source. See above for details.<SDK_PLUGINS>
should be replaced with the full path to your NVIGI SDK directory (the path of this README).<SDK_MODELS>
refers to the location of thenvigi.models
AI models directory.- This is available in
<SDK_PLUGINS>/data/nvigi.models
- This is available in
<SDK_TEST>
refers to the location of thenvigi.test
AI test data directory.- This is available in
<SDK_PLUGINS>/data/nvigi.test
- This is available in
IMPORTANT: The NVIGI SDK plugins tree (this tree) requires that there be a junction link from
<SDK_PLUGINS>/nvigi_core
to<NVIGI_CORE>
Create this by running the following in a command prompt whose current working directory is<SDK_PLUGINS>
:
mklink /j nvigi_core <NVIGI_CORE>
This must be done before running any further setup!
The following sections detail each of the steps of how to set up, rebuild and run the samples in this pack from the debugger or command line. But the basic steps are:
- Generate the project files
- Open a VS2022 Development Command Prompt to the SDK (a.k.a.
<SDK_PLUGINS>
) directory - Run
setup.bat
- Open a VS2022 Development Command Prompt to the SDK (a.k.a.
NOTE: If
setup.bat
fails with an error from thepackman
package downloader, please re-runsetup.bat
again as there are rare but possible issues with link-creation on initial run.
- Build the project, which will build the plugins and samples from source
- Launch VS2022
- Load
<SDK_PLUGINS>/_project/vs2022/nvigi.sln
as the workspace - Build the Release configuration
- Copy the updated binaries into the expected, packaged locations
- Open a (or use an existing) VS2022 Development Command Prompt to the SDK (a.k.a.
<SDK_PLUGINS>
) directory - Run
copy_sdk_binaries.bat Release
- Open a (or use an existing) VS2022 Development Command Prompt to the SDK (a.k.a.
- Run the sample (assumes that the models have been downloaded as per initial setup instructions)
- Open a (or use an existing) VS2022 Development Command Prompt to the SDK (a.k.a.
<SDK_PLUGINS>
) directory - Run
bin\x64\nvigi.basic.exe --models data\nvigi.models
- Open a (or use an existing) VS2022 Development Command Prompt to the SDK (a.k.a.
IMPORTANT: As noted above, there must be a junction link from
<SDK_PLUGINS>/nvigi_core
to your built and packaged copy of the NVIGI Core PDK,<NVIGI_CORE>
.
This should be done via the command
mklink /j nvigi_core <NVIGI_CORE>
To setup the NVIGI repository for building and running the Samples, open a command prompt window or PowerShell to the <SDK_PLUGINS>
directory and simply run the following command:
setup.bat
IMPORTANT: These steps are listed in the following section are done for you by the
setup.bat
script - do not run these steps manually/individually. This is merely to explain what is happening for you.
When you run the setup.bat
script, the script will cause two things to be done for you:
- The NVIDIA tool
packman
will pull all build dependencies to the local machine and cache them in a shared directory. Links will be created fromexternal
in the NVIGI tree to this shared cache for external build dependencies. premake
will generate the project build files in_project\vs20XX
(for Windows)
NOTE: If
setup.bat
fails with an error from thepackman
package downloader, please re-runsetup.bat
again as there are rare but possible issues with link-creation on initial run.
To build the project, simply open _project\vs20XX\nvigi.sln
in Visual Studio, select the desired build configuration and build.
NOTE: To build the project minimal configuration is needed. Any version of Windows 10 will do. Then run the setup and build scripts as described here above. That's it. The specific version of Windows, and NVIDIA driver are all runtime dependencies, not compile/link time dependencies. This allows NVIGI to build on stock virtual machines that require zero configuration.
Once you build your own binaries, either the SDK DLLs or the Sample executable, those will reside in _artifacts
, not bin\x64
. After building any configuration (Release, Debug, etc), you will need to run the copy_sdk_binaries.bat <cfg>
script. If you build the Release config, this would be:
- Open a command prompt to the SDK root
- run
copy_sdk_binaries.bat Release
This will copy the core lib, the DLLs and the exes to bin\x64
and should be done before trying the next section.
Setting up and running the samples as shipped pre-built or built using the instructions above is discussed in detail in Samples
IMPORTANT: Do not edit the MSVC project files directly! Always modify the
premake.lua
or files inpremake
.
When changing an existing project's settings or contents (ie: adding a new source file, changing a compiler setting, linking to a new library, etc), it is necessary to run setup.bat
again for those changes to take effect and MSVC project files and or solution will need to be reloaded in the IDE.
NOTE: NVIDIA does not recommend making changes to the headers in
include
, as these can affect the API itself and can make developer-built components incompatible with NVIDIA-supplied components.
A 3D-rendered sample application using In-Game Inferencing may be found in most SDK packs. If the pack is in standard layout, this will be in a sibling directory of sdk
called sample
. There is a README
in that directory with detailed instructions.