Skip to content
This repository was archived by the owner on Mar 29, 2026. It is now read-only.

Commit 0064689

Browse files
committed
feat: Add Forge LLM provider support
## Changes - Added ForgeLlmService subclassing OpenAILlmService in src/vanna/integrations/forge/, added forge optional dependency in pyproject.toml - Environment variable: FORGE_API_KEY - Base URL: https://api.forge.tensorblock.co/v1 - Model format: Provider/model-name (e.g., OpenAI/gpt-4o) - Non-breaking: purely additive, existing providers are untouched ## About Forge Forge (https://github.com/TensorBlock/forge) is an open-source middleware that routes inference across 40+ upstream providers (including OpenAI, Anthropic, Gemini, DeepSeek, and OpenRouter). It is OpenAI API compatible — works with the standard OpenAI SDK by changing base_url and api_key. ## Motivation We have seen growing interest from users who standardize on Forge for their model management and want to use it natively with Vanna. This integration aims to bridge that gap. ## Key Benefits - Self-Hosted & Privacy-First: Forge is open-source and designed to be self-hosted, critical for users who require data sovereignty - Future-Proofing: acts as a decoupling layer — instead of maintaining individual adapters for every new provider, Forge users can access them immediately - Compatibility: supports established aggregators (like OpenRouter) as well as direct provider connections (BYOK) ## References - Repo: https://github.com/TensorBlock/forge - Docs: https://www.tensorblock.co/api-docs/overview - Main Page: https://www.tensorblock.co/
1 parent 365d061 commit 0064689

3 files changed

Lines changed: 65 additions & 0 deletions

File tree

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ test = ["pytest>=7.0.0", "pytest-asyncio>=0.21.0", "pytest-mock>=3.10.0", "pytes
5454
dev = ["pytest>=7.0.0", "pytest-asyncio>=0.21.0", "pytest-mock>=3.10.0", "pytest-cov>=4.0.0", "tox>=4.0.0", "mypy", "ruff", "pandas-stubs", "plotly-stubs", "types-PyYAML", "types-requests", "types-tabulate"]
5555
chromadb = ["chromadb>=1.1.0"]
5656
openai = ["openai"]
57+
forge = ["openai"]
5758
azureopenai = ["openai", "azure-identity"]
5859
qianfan = ["qianfan"]
5960
mistralai = ["mistralai>=1.0.0"]
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
"""
2+
Forge integration.
3+
4+
This module provides a Forge LLM service implementation.
5+
"""
6+
7+
from .llm import ForgeLlmService
8+
9+
__all__ = ["ForgeLlmService"]
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
"""
2+
Forge LLM service implementation.
3+
4+
Forge is an OpenAI API compatible LLM router that provides unified access to
5+
multiple AI providers through a single API. This service extends the OpenAI
6+
implementation with Forge-specific defaults.
7+
8+
See https://github.com/TensorBlock/forge for details.
9+
"""
10+
11+
from __future__ import annotations
12+
13+
import os
14+
from typing import Any, Optional
15+
16+
from vanna.integrations.openai.llm import OpenAILlmService
17+
18+
19+
class ForgeLlmService(OpenAILlmService):
20+
"""Forge-backed LLM service.
21+
22+
Extends ``OpenAILlmService`` with Forge-specific defaults for base URL,
23+
API key, and model name. Forge uses the standard OpenAI API format so all
24+
streaming, tool-calling, and response handling from the parent class work
25+
unchanged.
26+
27+
Args:
28+
model: Model name in ``Provider/model-name`` format
29+
(e.g., ``"OpenAI/gpt-4o"``). Falls back to env
30+
``FORGE_MODEL`` then ``"OpenAI/gpt-4o-mini"``.
31+
api_key: Forge API key; falls back to env ``FORGE_API_KEY``.
32+
base_url: Optional custom base URL; falls back to env
33+
``FORGE_API_BASE`` then ``"https://api.forge.tensorblock.co/v1"``.
34+
**extra_client_kwargs: Extra kwargs forwarded to ``openai.OpenAI()``.
35+
"""
36+
37+
def __init__(
38+
self,
39+
model: Optional[str] = None,
40+
api_key: Optional[str] = None,
41+
base_url: Optional[str] = None,
42+
**extra_client_kwargs: Any,
43+
) -> None:
44+
api_key = api_key or os.getenv("FORGE_API_KEY")
45+
base_url = base_url or os.getenv(
46+
"FORGE_API_BASE", "https://api.forge.tensorblock.co/v1"
47+
)
48+
model = model or os.getenv("FORGE_MODEL", "OpenAI/gpt-4o-mini")
49+
50+
super().__init__(
51+
model=model,
52+
api_key=api_key,
53+
base_url=base_url,
54+
**extra_client_kwargs,
55+
)

0 commit comments

Comments
 (0)