Adds compatibility with NVIDIA AI Workbench #18
Conversation
nv-edwli
commented
Apr 10, 2025
nv-edwli
commented
- Adds AI Workbench specific metadata and setup files, eg. .project directory, variables.env, etc.
- Adds AI Workbench supplemental README to workbench directory
- Adds snippet to main README to redirect users to AI Workbench supplemental README
- Adjusts ingestion and rag notebooks to use correct base URL when working in AI Workbench environment
- Non-workbench users will use original due to env var toggle
- Adds quickstart.ipynb to workbench directory (same as launchable, minus the unneeded env setup sections)
- Adds compose.yml to workbench directory.
- AI Workbench will auto-use this version, all other users will use original under deploy as per main README instructions
- Added pip packages: jupyterlab>3.0 - Added 7 files - Modified files: .gitignore, .project/spec.yaml, requirements.txt
- Added files: variables.env, workbench/compose.yaml - Modified files: .project/spec.yaml
- Modified files: .project/spec.yaml, workbench/compose.yaml
- Modified files: .gitignore, workbench/compose.yaml
- Modified files: .gitignore, .project/spec.yaml, variables.env, workbench/compose.yaml
- Modified files: notebooks/ingestion_api_usage.ipynb, notebooks/retriever_api_usage.ipynb, variables.env
- Added files: workbench/quickstart.ipynb - Modified files: variables.env, workbench/compose.yaml
- Added files: workbench/nim_cache/.gitkeep - Modified files: .gitignore, variables.env, workbench/compose.yaml
use different device ids for in a box where nemotron is on 0,1 0 becomes 4 1 becomes 5 (the 2, 3 remain the same and are used by nim_llm)
use the locally deployed microservices
Update variables.env
move device ids for in-a-box
- Modified files: .gitignore, workbench/compose.yaml
| @@ -22,3 +22,4 @@ opentelemetry-exporter-prometheus==0.50b0 | |||
| opentelemetry-instrumentation-milvus==0.36.0 | |||
| opentelemetry-instrumentation-fastapi==0.50b0 | |||
| opentelemetry-processor-baggage==0.50b0 | |||
| jupyterlab>3.0 | |||
There was a problem hiding this comment.
Every project in AI Workbench is src code that is containerized automatically using a base container with added project-level customizations on top of it, like packages, variables, etc. The main project container environment for this BP project provides a Jupyterlab IDE by default so users can develop, write code, and customize the BP in the project environment.
| @@ -0,0 +1,21 @@ | |||
| # Flag for AI Workbench env | |||
There was a problem hiding this comment.
Can we keep this file inside workbench directory?
There was a problem hiding this comment.
AI Workbench expects environment setup files at the top level of the repo and metadata files under the top level .project hidden folder.
Env setup files include requirements.txt, variables.env, apt.txt, preBuild.bash, postBuild.bash, etc. More details here.
In this case, we only have a need for the former two files, the others are not needed and therefore not included in the repo.
This file does not affect the flow for users running the BP outside of AIWB as they would still run the flow via the main README instructions.
| @@ -0,0 +1,585 @@ | |||
| services: | |||
There was a problem hiding this comment.
Instead of creating a replication of the compose file, can we reuse the one in deploy/compose?
There was a problem hiding this comment.
One current limitation of AI WB is that it expects a single compose file--see compose_file_path field in .project/spec.yaml.
This means unless the deploy/compose gets refactored, it is probably best to organize it as a single compose under the workbench directory where we separate out the services under their own separate compose profiles.
This way users can still spin up portions of the workflow if they would like from by selecting the profiles they want to use, which achieves the same effect as the main flow with multiple compose files.
For example, users using Build endpoints can spin up rag, ingest, vectordb profiles, while other users using locally running NIMs can spin up rag, ingest, vectordb, and local nim profiles.
