Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 196 additions & 0 deletions src/oss/python/integrations/chat/crusoe.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
---
title: ChatCrusoe
description: Crusoe AI Managed Inference chat model integration for LangChain.
---

# ChatCrusoe

This page will help you get started with Crusoe AI [chat models](/oss/langchain/models). For detailed documentation of all ChatCrusoe features and configurations, head to the [Crusoe managed inference docs](https://docs.crusoecloud.com/managed-inference/overview).

[Crusoe AI](https://www.crusoe.ai/) provides high-performance managed inference for [leading open-source models](https://docs.crusoecloud.com/managed-inference/overview) via the Crusoe Intelligence Foundry, powered by proprietary MemoryAlloy™ technology for ultra-low latency and high throughput.

## Overview

### Integration details

| Class | Package | Local | Serializable | JS support | Downloads | Version |
| :--- | :--- | :---: | :---: | :---: | :---: | :---: |
| [ChatCrusoe](https://docs.crusoecloud.com/managed-inference/overview) | [langchain-crusoe](https://pypi.org/project/langchain-crusoe/) | ❌ | beta | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-crusoe?style=flat-square&label=%20&color=blue) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-crusoe?style=flat-square&label=%20&color=orange) |

### Model features

| [Tool calling](/oss/langchain/tools) | [Structured output](/oss/langchain/structured-output) | JSON mode | [Image input](/oss/langchain/messages#multimodal) | Audio input | Video input | [Token-level streaming](/oss/langchain/streaming#llm-tokens) | Native async | [Token usage](/oss/langchain/models#token-usage) | [Logprobs](/oss/langchain/models#log-probabilities) |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |

## Setup

To access Crusoe models you'll need to create a Crusoe Cloud account, get an Inference API key, and install the `langchain-crusoe` integration package.

### Credentials

Head to the [Crusoe Cloud Console](https://console.crusoecloud.com/) to sign up. Then navigate to the **Security** tab and select **Inference API Key** to generate your key. Once you've done this, set the CRUSOE_API_KEY environment variable:

```python
import getpass
import os

if "CRUSOE_API_KEY" not in os.environ:
os.environ["CRUSOE_API_KEY"] = getpass.getpass("Enter your Crusoe API key: ")
```

To enable automated tracing of your model calls, set your [LangSmith](https://docs.langchain.com/langsmith/home) API key:

```python
os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
os.environ["LANGSMITH_TRACING"] = "true"
```

### Installation

The LangChain Crusoe integration is included in the `langchain-crusoe` package:

```python pip
pip install -qU langchain-crusoe
```

```python uv
uv add langchain-crusoe
```

## Instantiation

Now we can instantiate our model object and generate chat completions:

```python
from langchain_crusoe import ChatCrusoe

llm = ChatCrusoe(
model="meta-llama/Llama-3.3-70B-Instruct",
temperature=0,
max_tokens=None,
timeout=None,
max_retries=2,
# api_key="...", # if not set via CRUSOE_API_KEY env var
# other params...
)
```

### Available models

Crusoe serves leading open-source models via the Intelligence Foundry. See the [full model list](https://docs.crusoecloud.com/managed-inference/overview) for the latest availability.

| Model | Provider | Context Length |
| :--- | :--- | :---: |
| `meta-llama/Llama-3.3-70B-Instruct` | Meta | 128k |
| `openai/gpt-oss-120b` | OpenAI | 128k |
| `deepseek-ai/DeepSeek-V3-0324` | DeepSeek | 160k |
| `deepseek-ai/DeepSeek-R1-0528` | DeepSeek | 160k |
| `deepseek-ai/DeepSeek-V3.1` | DeepSeek | 160k |
| `Qwen/Qwen3-235B-A22B` | Qwen | 131k |
| `google/gemma-3-12b-it` | Google | 128k |
| `moonshotai/Kimi-K2-Thinking` | Moonshot AI | 131k |

## Invocation

```python
messages = [
(
"system",
"You are a helpful assistant that translates English to French. Translate the user sentence.",
),
("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
```

```python
AIMessage(content="J'adore la programmation.", response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 35, 'total_tokens': 44}, 'model_name': 'meta-llama/Llama-3.3-70B-Instruct', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-...', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})
```

```python
print(ai_msg.content)
```

```
J'adore la programmation.
```

## Chaining

We can [chain](/oss/langchain/overview) our model with a prompt template like so:

```python
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant that translates {input_language} to {output_language}.",
),
("human", "{input}"),
]
)

chain = prompt | llm
chain.invoke(
{
"input_language": "English",
"output_language": "German",
"input": "I love programming.",
}
)
```

```python
AIMessage(content='Ich liebe Programmieren.', response_metadata={...}, id='run-...')
```

## Streaming

```python
for chunk in llm.stream(messages):
print(chunk.content, end="", flush=True)
```

## Tool calling

```python
from pydantic import BaseModel, Field

class GetWeather(BaseModel):
"""Get the current weather in a given location."""
location: str = Field(description="City and state, e.g. San Francisco, CA")

llm_with_tools = llm.bind_tools([GetWeather])
ai_msg = llm_with_tools.invoke("What's the weather like in San Francisco?")
print(ai_msg.tool_calls)
```

## Structured output

```python
from pydantic import BaseModel, Field
from typing import Optional

class Joke(BaseModel):
"""Joke to tell user."""
setup: str = Field(description="The setup of the joke")
punchline: str = Field(description="The punchline to the joke")
rating: Optional[int] = Field(description="How funny the joke is, from 1 to 10")

structured_llm = llm.with_structured_output(Joke)
structured_llm.invoke("Tell me a joke about cats")
```

```python
Joke(setup='Why was the cat sitting on the computer?', punchline='To keep an eye on the mouse!', rating=7)
```

---

## API reference

For detailed documentation of all ChatCrusoe features and configurations, head to the [Crusoe managed inference docs](https://docs.crusoecloud.com/managed-inference/overview).
Loading