Add support for NVIDIA NIM as an LLM provider#79
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new LLM provider option (NVIDIA NIM) alongside the existing OpenAI/Anthropic/Ollama support, following the established provider-selection pattern in turtle_agent and updating dependency extras + type hints.
Changes:
- Extend
get_llm()provider switch to supportLLM_PROVIDER=nvidiavialangchain-nvidia-ai-endpoints/ChatNVIDIA. - Update ROSA’s
ChatModeltype hints to includeChatNVIDIAunderTYPE_CHECKING. - Add
nvidiaoptional dependency (and include it in theallextra) plus.envtemplate variables for NIM configuration.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/turtle_agent/scripts/llm.py |
Adds NVIDIA provider branch that constructs ChatNVIDIA using env-based configuration. |
src/rosa/rosa.py |
Extends static type union and docstring to list NVIDIA as a tested provider. |
pyproject.toml |
Adds nvidia extra and includes it in all for optional installs. |
.env |
Documents LLM_PROVIDER=nvidia and adds NVIDIA NIM env var placeholders/defaults. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| except ImportError: | ||
| raise ImportError( | ||
| "langchain-nvidia-ai-endpoints is required for NVIDIA NIM support. " | ||
| "Install it with: pip install langchain-nvidia-ai-endpoints" |
There was a problem hiding this comment.
The ImportError message recommends installing langchain-nvidia-ai-endpoints directly, which can bypass this repo’s pinned/compatible optional dependency set. Prefer pointing users to install the project extra (e.g., pip install 'jpl-rosa[nvidia]' or pip install '.[nvidia]') so they get a known-good version range.
| "Install it with: pip install langchain-nvidia-ai-endpoints" | |
| "Install the project's NVIDIA extra with: pip install '.[nvidia]'" |
| api_key=get_env_variable("NVIDIA_API_KEY"), | ||
| model=os.getenv("NVIDIA_MODEL", "nvidia/nemotron-3-super-120b-a12b"), | ||
| base_url=os.getenv("NVIDIA_BASE_URL", "https://integrate.api.nvidia.com/v1"), |
There was a problem hiding this comment.
get_env_variable() treats an empty string as “set”, and the .env template sets NVIDIA_API_KEY= (empty). With LLM_PROVIDER=nvidia, this will pass an empty API key into ChatNVIDIA and fail later with a less actionable auth error. Consider treating empty/whitespace values as unset (e.g., if not value or not value.strip(): ...) either here or inside get_env_variable() so misconfiguration is caught early with a clear message.
areporeporepo
force-pushed
the
feat/add-nvidia-nim-provider
branch
from
fb472dc to
fdfb51c
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| value = os.getenv(var_name) | ||
| if value is None: | ||
| if not value or not value.strip(): | ||
| msg = f"Environment variable {var_name} is not set." | ||
| raise ValueError(msg) | ||
| return value |
There was a problem hiding this comment.
get_env_variable now rejects whitespace-only values (good), but it returns the unstripped value. If an env var has accidental leading/trailing whitespace, this will pass validation but can still cause auth/URL failures downstream. Consider returning value.strip() (and optionally adjusting the error text to say “not set or empty”).
| try: | ||
| from langchain_nvidia_ai_endpoints import ChatNVIDIA | ||
| except ImportError: | ||
| raise ImportError( | ||
| "langchain-nvidia-ai-endpoints is required for NVIDIA NIM support. " | ||
| "Install the project's NVIDIA extra with: pip install '.[nvidia]'" | ||
| ) |
There was a problem hiding this comment.
When re-raising on failed optional import, consider preserving or suppressing the original exception (except ImportError as e: ... raise ImportError(...) from e or from None). This keeps tracebacks clearer for debugging dependency issues.
- Use project extra in ImportError message: pip install 'jpl-rosa[nvidia]' - Chain ImportError with `from e` for clearer tracebacks - Return stripped value from get_env_variable() to avoid passing whitespace-padded keys to providers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous default (nvidia/nemotron-3-super-120b-a12b) triggers an AssertionError in langchain-nvidia-ai-endpoints 0.3.x because the NIM /v1/models API returns duplicate entries for that model. Fixed upstream in langchain-ai/langchain-nvidia#282 (April 10, 2026) but not yet in a tagged release (latest v1.2.1 is from March 16, 2026). Switching the default to llama-3.3-nemotron-super-49b-v1.5: - In NIM's hardcoded MODEL_TABLE, so avoids the duplicate-candidates path - Built for agentic AI with native tool-calling (verified) - NVIDIA + Meta origin, appropriate for NASA/JPL context NVIDIA_MODEL env var remains configurable for users who want to override once the upstream fix ships in a released version. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: anh nguyen <anh.nqqq@icloud.com>
|
Hello @RobRoyce, wondering if Rosa would want to support more open models? Nvidia NIM has a pretty good catalog. There's an upstream bug in langchain-nvidia affecting the newest nemotron-3-super (explained in the commit), but everything else seems to work well from my tests. |
Adds NVIDIA NIM support. Uses
langchain-nvidia-ai-endpointswithChatNVIDIA. Backwards compatible.