Skip to content
29 changes: 28 additions & 1 deletion docs/nav.json
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,34 @@
{ "label": "Export Data", "slug": "how-to-guides/query-api" },
{ "label": "Feature Flags (OFREP)", "slug": "how-to-guides/client-side-feature-flags" },
{ "label": "Convert to Organization", "slug": "how-to-guides/convert-to-organization" },
{ "label": "AI Gateway", "slug": "reference/advanced/gateway/index" }
{
"label": "AI Gateway",
"items": [
{ "label": "Overview", "slug": "reference/advanced/gateway/index" },
{
"label": "Integrations",
"items": [
{ "label": "Overview", "slug": "reference/advanced/gateway/agent-frameworks" },
{ "label": "Agno", "slug": "reference/advanced/gateway/integrations/agno" },
{ "label": "Genkit", "slug": "reference/advanced/gateway/integrations/genkit" },
{ "label": "Google ADK", "slug": "reference/advanced/gateway/integrations/google-adk" },
{ "label": "Haystack", "slug": "reference/advanced/gateway/integrations/haystack" },
{ "label": "Instructor", "slug": "reference/advanced/gateway/integrations/instructor" },
{ "label": "LangChain", "slug": "reference/advanced/gateway/integrations/langchain" },
{ "label": "LlamaIndex", "slug": "reference/advanced/gateway/integrations/llamaindex" },
{ "label": "Mastra", "slug": "reference/advanced/gateway/integrations/mastra" },
{ "label": "Microsoft Agent Framework", "slug": "reference/advanced/gateway/integrations/microsoft-agent-framework" },
{ "label": "OpenAI Agents SDK", "slug": "reference/advanced/gateway/integrations/openai-agents-sdk" },
{ "label": "OpenAI SDK", "slug": "reference/advanced/gateway/integrations/openai-sdk" },
{ "label": "Semantic Kernel", "slug": "reference/advanced/gateway/integrations/semantic-kernel" },
{ "label": "Smolagents", "slug": "reference/advanced/gateway/integrations/smolagents" },
{ "label": "Strands Agents", "slug": "reference/advanced/gateway/integrations/strands-agents" },
{ "label": "Vercel AI SDK", "slug": "reference/advanced/gateway/integrations/vercel-ai-sdk" },
{ "label": "VoltAgent", "slug": "reference/advanced/gateway/integrations/voltagent" }
]
}
]
}
]
},
{
Expand Down
41 changes: 41 additions & 0 deletions docs/reference/advanced/gateway/agent-frameworks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: "AI Gateway integrations"
description: "Connect frameworks and SDKs to the Logfire AI Gateway."
---

# AI Gateway integrations

These guides show how to point each framework or SDK at the Logfire AI Gateway, so every model call flows through one endpoint with usage tracking, spending limits, and provider fallback.

## Before you start

1. In Logfire, open your organization, then go to **AI Engineering** > **Gateway**.
2. Create or copy a gateway API key from the Gateway **API Keys** tab.
3. Set the key in your terminal:

```bash
export LOGFIRE_GATEWAY_API_KEY="..."
```

Each example uses a US gateway URL like `https://gateway-us.pydantic.dev/proxy/openai` (EU users swap in `gateway-eu`). The final path segment selects the provider — OpenAI, Anthropic, Google, and so on. To switch providers, or to point at a routing group that spreads calls across several with fallback, copy the URL you want from the Gateway **Providers** or **Routing** tab.

## Examples

| Framework or SDK | Available in |
| --- | --- |
| Agno | [Python](/logfire/manage/ai-gateway/integrations/agno/) |
| Genkit | [Go](/logfire/manage/ai-gateway/integrations/genkit/#go), [JavaScript](/logfire/manage/ai-gateway/integrations/genkit/#javascript) |
| Google ADK | [Go](/logfire/manage/ai-gateway/integrations/google-adk/#go), [Python](/logfire/manage/ai-gateway/integrations/google-adk/#python) |
| Haystack | [Python](/logfire/manage/ai-gateway/integrations/haystack/) |
| Instructor | [Python](/logfire/manage/ai-gateway/integrations/instructor/) |
| LangChain | [TypeScript](/logfire/manage/ai-gateway/integrations/langchain/#typescript), [Python](/logfire/manage/ai-gateway/integrations/langchain/#python) |
| LlamaIndex | [Python](/logfire/manage/ai-gateway/integrations/llamaindex/) |
| Mastra | [JavaScript](/logfire/manage/ai-gateway/integrations/mastra/) |
| Microsoft Agent Framework | [.NET](/logfire/manage/ai-gateway/integrations/microsoft-agent-framework/#dotnet), [Python](/logfire/manage/ai-gateway/integrations/microsoft-agent-framework/#python) |
| OpenAI Agents SDK | [TypeScript](/logfire/manage/ai-gateway/integrations/openai-agents-sdk/#typescript), [Python](/logfire/manage/ai-gateway/integrations/openai-agents-sdk/#python) |
| OpenAI SDK | [.NET](/logfire/manage/ai-gateway/integrations/openai-sdk/#dotnet), [Go](/logfire/manage/ai-gateway/integrations/openai-sdk/#go), [TypeScript](/logfire/manage/ai-gateway/integrations/openai-sdk/#typescript), [Python](/logfire/manage/ai-gateway/integrations/openai-sdk/#python) |
| Semantic Kernel | [.NET](/logfire/manage/ai-gateway/integrations/semantic-kernel/#dotnet), [Python](/logfire/manage/ai-gateway/integrations/semantic-kernel/#python) |
| Smolagents | [Python](/logfire/manage/ai-gateway/integrations/smolagents/) |
| Strands Agents | [TypeScript](/logfire/manage/ai-gateway/integrations/strands-agents/#typescript), [Python](/logfire/manage/ai-gateway/integrations/strands-agents/#python) |
| Vercel AI SDK | [JavaScript](/logfire/manage/ai-gateway/integrations/vercel-ai-sdk/) |
| VoltAgent | [JavaScript](/logfire/manage/ai-gateway/integrations/voltagent/) |
26 changes: 26 additions & 0 deletions docs/reference/advanced/gateway/integrations/agno.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: "AI Gateway: Agno"
description: "Route Agno model calls through the Logfire AI Gateway."
---

# Agno

[Agno](https://docs.agno.com/) is a Python framework for building multi-modal, multi-agent AI systems. To route its model calls through the Logfire AI Gateway, configure `OpenAIChat` with the gateway URL, using a key from the Gateway **API Keys** tab.

```python title="agno-gateway.py" skip-run="true" skip-reason="external-connection"
import os

from agno.agent import Agent
from agno.models.openai import OpenAIChat

agent = Agent(
name="Weather Agent",
model=OpenAIChat(
id="gpt-5.4-mini",
api_key=os.environ["LOGFIRE_GATEWAY_API_KEY"],
base_url="https://gateway-us.pydantic.dev/proxy/openai",
),
)

agent.print_response("What is the weather in London?")
```
84 changes: 84 additions & 0 deletions docs/reference/advanced/gateway/integrations/genkit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: "AI Gateway: Genkit"
description: "Route Genkit model calls through the Logfire AI Gateway."
---

# Genkit

[Genkit](https://genkit.dev/) is Google's open-source framework for building AI-powered applications. To route its model calls through the Logfire AI Gateway, use Genkit's OpenAI-compatible plugin pointed at the gateway URL, using a key from the Gateway **API Keys** tab.

## JavaScript

```typescript title="genkit-gateway.mts" skip-run="true" skip-reason="external-connection"
import { openAICompatible } from '@genkit-ai/compat-oai';
import { genkit } from 'genkit';
import { z } from 'zod';

const envSchema = z.object({
LOGFIRE_GATEWAY_API_KEY: z.string(),
});

const env = envSchema.parse(process.env);

const ai = genkit({
plugins: [
openAICompatible({
name: 'openai',
apiKey: env.LOGFIRE_GATEWAY_API_KEY,
baseURL: 'https://gateway-us.pydantic.dev/proxy/openai',
}),
],
});

const { text } = await ai.generate({
model: 'openai/gpt-5.4-mini',
prompt: 'What is the weather in London?',
});

console.log(text);
```

## Go

```go title="genkit-gateway.go" skip-run="true" skip-reason="external-connection"
package main

import (
"context"
"log"
"os"
"time"

"github.com/firebase/genkit/go/ai"
"github.com/firebase/genkit/go/genkit"
"github.com/firebase/genkit/go/plugins/compat_oai/openai"
"github.com/openai/openai-go/option"
)

func main() {
apiKey := os.Getenv("LOGFIRE_GATEWAY_API_KEY")
if apiKey == "" {
log.Fatal("LOGFIRE_GATEWAY_API_KEY is required")
}

ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()

g := genkit.Init(ctx, genkit.WithPlugins(&openai.OpenAI{
APIKey: apiKey,
Opts: []option.RequestOption{
option.WithBaseURL("https://gateway-us.pydantic.dev/proxy/openai"),
},
}))

response, err := genkit.Generate(ctx, g,
ai.WithPrompt("What is the weather in London?"),
ai.WithModelName("openai/gpt-5.4-mini"),
)
if err != nil {
log.Fatal(err)
}

log.Println(response.Text())
}
```
117 changes: 117 additions & 0 deletions docs/reference/advanced/gateway/integrations/google-adk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
title: "AI Gateway: Google ADK"
description: "Route Google ADK model calls through the Logfire AI Gateway."
---

# Google ADK

[Google Agent Development Kit (ADK)](https://adk.dev/) is Google's framework for building multi-step, multi-agent AI systems. To route its model calls through the Logfire AI Gateway, set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab. The Python example routes calls via LiteLLM (a library that translates between LLM provider APIs); the Go example connects directly to the gateway's Google Vertex proxy.

## Python

```python title="google-adk-gateway.py" skip-run="true" skip-reason="external-connection"
import asyncio
import os

from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.genai import types

os.environ["OPENAI_API_KEY"] = os.environ["LOGFIRE_GATEWAY_API_KEY"]
os.environ["OPENAI_BASE_URL"] = "https://gateway-us.pydantic.dev/proxy/openai"

agent = LlmAgent(
model=LiteLlm(model="openai/gpt-5.4-mini"),
name="weather_agent",
instruction="You are a concise weather assistant.",
)


async def main() -> None:
runner = InMemoryRunner(agent=agent)
session = await runner.session_service.create_session(
app_name=runner.app_name,
user_id="gateway-example-user",
)
content = types.Content(
role="user",
parts=[types.Part.from_text(text="What is the weather in London?")],
)
async for event in runner.run_async(
user_id=session.user_id,
session_id=session.id,
new_message=content,
):
if event.is_final_response() and event.content and event.content.parts:
print(event.content.parts[0].text)


if __name__ == "__main__":
asyncio.run(main())
```

## Go

The Go example routes Google Vertex model calls through `https://gateway-us.pydantic.dev/proxy/google-vertex`
and uses `gemini-2.5-flash`.

```go title="google-adk-gateway.go" skip-run="true" skip-reason="external-connection"
package main

import (
"context"
"fmt"
"log"
"net/http"
"os"
"time"

adkmodel "google.golang.org/adk/v2/model"
"google.golang.org/adk/v2/model/gemini"
"google.golang.org/genai"
)

func main() {
apiKey := os.Getenv("LOGFIRE_GATEWAY_API_KEY")
if apiKey == "" {
log.Fatal("LOGFIRE_GATEWAY_API_KEY is required")
}

ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()

model, err := gemini.NewModel(ctx, "gemini-2.5-flash", &genai.ClientConfig{
Backend: genai.BackendVertexAI,
HTTPOptions: genai.HTTPOptions{
BaseURL: "https://gateway-us.pydantic.dev/proxy/google-vertex",
Headers: http.Header{
"Authorization": []string{"Bearer " + apiKey},
},
},
})
if err != nil {
panic(err)
}

request := &adkmodel.LLMRequest{
Contents: []*genai.Content{
genai.NewContentFromText("What is the weather in London?", genai.RoleUser),
},
}

for response, err := range model.GenerateContent(ctx, request, false) {
if err != nil {
panic(err)
}
if response.Content == nil {
continue
}
for _, part := range response.Content.Parts {
if part.Text != "" {
fmt.Println(part.Text)
}
}
}
}
```
24 changes: 24 additions & 0 deletions docs/reference/advanced/gateway/integrations/haystack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: "AI Gateway: Haystack"
description: "Route Haystack model calls through the Logfire AI Gateway."
---

# Haystack

[Haystack](https://haystack.deepset.ai/) is an open-source Python framework for building AI search and document processing pipelines. To route its model calls through the Logfire AI Gateway, configure `OpenAIChatGenerator` with the gateway URL, using a key from the Gateway **API Keys** tab.

```python title="haystack-gateway.py" skip-run="true" skip-reason="external-connection"
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

generator = OpenAIChatGenerator(
model="gpt-5.4-mini",
api_key=Secret.from_env_var("LOGFIRE_GATEWAY_API_KEY"),
api_base_url="https://gateway-us.pydantic.dev/proxy/openai",
)

response = generator.run([ChatMessage.from_user("What is the weather in London?")])

print(response["replies"][0].text)
```
37 changes: 37 additions & 0 deletions docs/reference/advanced/gateway/integrations/instructor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: "AI Gateway: Instructor"
description: "Route Instructor model calls through the Logfire AI Gateway."
---

# Instructor

[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from LLM responses — you describe the shape of the data you want and Instructor handles the rest. Because Instructor wraps a standard OpenAI client, routing through the gateway means configuring that underlying client with the gateway URL, using a key from the Gateway **API Keys** tab.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The description paragraph uses a different sentence structure than the other 15 integration pages, which consistently follow an imperative "To route its model calls through the Logfire AI Gateway, configure/set up X..." pattern. Consider aligning this description with that established convention for consistency.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/reference/advanced/gateway/integrations/instructor.md, line 8:

<comment>The description paragraph uses a different sentence structure than the other 15 integration pages, which consistently follow an imperative "To route its model calls through the Logfire AI Gateway, configure/set up X..." pattern. Consider aligning this description with that established convention for consistency.</comment>

<file context>
@@ -5,7 +5,7 @@ description: "Route Instructor model calls through the Logfire AI Gateway."
 # Instructor
 
-[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from large language model (LLM) responses — you describe the shape of the data you want and Instructor handles the rest. Because Instructor wraps a standard OpenAI client, routing through the gateway is just a matter of configuring that underlying client with the gateway URL. Set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab before running this example.
+[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from LLM responses — you describe the shape of the data you want and Instructor handles the rest. Because Instructor wraps a standard OpenAI client, routing through the gateway means configuring that underlying client with the gateway URL, using a key from the Gateway **API Keys** tab.
 
 ```python title="instructor-gateway.py" skip-run="true" skip-reason="external-connection"
</file context>
Suggested change
[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from LLM responses — you describe the shape of the data you want and Instructor handles the rest. Because Instructor wraps a standard OpenAI client, routing through the gateway means configuring that underlying client with the gateway URL, using a key from the Gateway **API Keys** tab.
[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from LLM responses — you describe the shape of the data you want and Instructor handles the rest. To route its model calls through the Logfire AI Gateway, configure the underlying OpenAI client with the gateway URL, using a key from the Gateway **API Keys** tab.


```python title="instructor-gateway.py" skip-run="true" skip-reason="external-connection"
import os

import instructor
from openai import OpenAI
from pydantic import BaseModel


class Weather(BaseModel):
city: str
condition: str


client = instructor.from_openai(
OpenAI(
api_key=os.environ["LOGFIRE_GATEWAY_API_KEY"],
base_url="https://gateway-us.pydantic.dev/proxy/openai",
)
)

weather = client.create(
model="gpt-5.4-mini",
response_model=Weather,
messages=[{"role": "user", "content": "What is the weather in London?"}],
)

print(weather)
```
Loading
Loading