Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,15 +144,13 @@ export async function POST(req: Request) {

import { useChat } from '@ai-sdk/react';
import { useChannel } from 'ably/react';
import { useClientTransport, useActiveTurns, useView } from '@ably/ai-transport/react';
import { useActiveTurns, useView } from '@ably/ai-transport/react';
import { useChatTransport, useMessageSync } from '@ably/ai-transport/vercel/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';

function Chat({ chatId, clientId }: { chatId: string; clientId?: string }) {
const { channel } = useChannel({ channelName: chatId });

const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
const chatTransport = useChatTransport(transport);
const { chatTransport, transport } = useChatTransport({ channel, clientId });

const { messages, setMessages, sendMessage, stop } = useChat({
id: chatId,
Expand Down
6 changes: 2 additions & 4 deletions demo/vercel/react/use-chat/src/app/chat.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@

import { useChat } from '@ai-sdk/react';
import { useChannel } from 'ably/react';
import { useClientTransport, useActiveTurns, useView, useAblyMessages } from '@ably/ai-transport/react';
import { useActiveTurns, useView, useAblyMessages } from '@ably/ai-transport/react';
import { useChatTransport, useMessageSync } from '@ably/ai-transport/vercel/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';
import { useState } from 'react';
import { MessageList } from './components/message-list';
import { DebugPane } from './components/debug-pane';
Expand All @@ -18,8 +17,7 @@ export function Chat({ chatId, clientId, historyLimit }: { chatId: string; clien
const { channel } = useChannel({ channelName: chatId });

// Create transport immediately (subscribes before attach — RTL7g)
const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
const chatTransport = useChatTransport(transport);
const { chatTransport, transport } = useChatTransport({ channel, clientId });

const {
setMessages,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ export function Chat({ chatId, clientId, historyLimit }: ChatProps) {
channel,
codec: UIMessageCodec,
clientId,
api: '/api/chat',
body: () => ({ id: chatId }),
});

Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ In React, the hooks handle subscriptions and state management:
import { useClientTransport, useView } from '@ably/ai-transport/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';

const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId, api: '/api/chat' });
const { nodes, send } = useView(transport);
```

Expand Down
2 changes: 1 addition & 1 deletion docs/features/branching.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ With a single view, navigating to a different branch in one part of the UI chang
```typescript
import { useClientTransport, useCreateView, useView } from '@ably/ai-transport/react';

const transport = useClientTransport({ channel, codec, clientId });
const transport = useClientTransport({ channel, codec, clientId, api: '/api/chat' });

// Default view for the left pane
const left = useView(transport, { limit: 50 });
Expand Down
4 changes: 2 additions & 2 deletions docs/features/multi-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ No special API is needed. Connect two clients to the same channel name, and mess

```typescript
// Client A
const transportA = useClientTransport({ channel, codec: UIMessageCodec, clientId: 'user-a' });
const transportA = useClientTransport({ channel, codec: UIMessageCodec, clientId: 'user-a', api: '/api/chat' });

// Client B (different browser tab, device, or user)
const transportB = useClientTransport({ channel, codec: UIMessageCodec, clientId: 'user-b' });
const transportB = useClientTransport({ channel, codec: UIMessageCodec, clientId: 'user-b', api: '/api/chat' });

// When Client A sends a message and the server streams a response,
// Client B sees both the user message and the assistant response
Expand Down
9 changes: 3 additions & 6 deletions docs/frameworks/vercel-ai-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,10 @@ The Vercel AI SDK provides model abstraction, streaming primitives, and React ho
Wrap the transport in a `ChatTransport` adapter and pass it to Vercel's `useChat()`. Message state is managed by `useChat()` - the transport delivers messages over Ably instead of HTTP.

```typescript
import { useClientTransport } from '@ably/ai-transport/react';
import { useChatTransport, useMessageSync } from '@ably/ai-transport/vercel/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';
import { useChat } from '@ai-sdk/react';

const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
const chatTransport = useChatTransport(transport);
const { chatTransport, transport } = useChatTransport({ channel, clientId });

const { messages, setMessages, sendMessage, stop } = useChat({
id: chatId,
Expand All @@ -37,7 +34,7 @@ const { messages, setMessages, sendMessage, stop } = useChat({
useMessageSync(transport, setMessages);
```

`useChatTransport()` wraps the core transport into the `ChatTransport` interface that `useChat()` expects. `useMessageSync()` pushes the transport's authoritative message list into `useChat()`'s state - this is how messages from other clients appear.
`useChatTransport()` creates the core transport and wraps it into the `ChatTransport` interface that `useChat()` expects, returning both as `{ chatTransport, transport }`. `useMessageSync()` pushes the transport's authoritative message list into `useChat()`'s state - this is how messages from other clients appear.

### Generic hooks path (more control)

Expand All @@ -47,7 +44,7 @@ Use the generic React hooks directly. You manage message state through the trans
import { useClientTransport, useView, useActiveTurns } from '@ably/ai-transport/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';

const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId, api: '/api/chat' });
const {
nodes,
hasOlder,
Expand Down
31 changes: 13 additions & 18 deletions docs/get-started/vercel-use-chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,36 +155,32 @@ Wire up `useChat()` with the AI Transport hooks:

import { useChat } from '@ai-sdk/react';
import { useChannel, ChannelProvider } from 'ably/react';
import { useClientTransport, useActiveTurns, useView } from '@ably/ai-transport/react';
import { useActiveTurns, useView } from '@ably/ai-transport/react';
import { useChatTransport, useMessageSync } from '@ably/ai-transport/vercel/react';
import { UIMessageCodec } from '@ably/ai-transport/vercel';
import { useState } from 'react';

function ChatInner({ chatId, clientId }: { chatId: string; clientId?: string }) {
const { channel } = useChannel({ channelName: chatId });
const [input, setInput] = useState('');

// 1. Create the core transport - subscribes to the Ably channel and decodes
// incoming messages through UIMessageCodec
const transport = useClientTransport({ channel, codec: UIMessageCodec, clientId });
// 1. Create the transport and wrap it for useChat - subscribes to the
// Ably channel and decodes incoming messages through UIMessageCodec
const { chatTransport, transport } = useChatTransport({ channel, clientId });

// 2. Wrap it for useChat compatibility
const chatTransport = useChatTransport(transport);

// 3. Use Vercel's useChat with the wrapped transport
// 2. Use Vercel's useChat with the wrapped transport
const { messages, setMessages, sendMessage, stop } = useChat({
id: chatId,
transport: chatTransport,
});

// 4. Sync transport messages into useChat's state (for observer messages)
// 3. Sync transport messages into useChat's state (for observer messages)
useMessageSync(transport, setMessages);

// 5. Track active turns for loading state
// 4. Track active turns for loading state
const activeTurns = useActiveTurns(transport);
const isStreaming = activeTurns.size > 0;

// 6. Load history on mount
// 5. Load history on mount
useView(transport, { limit: 30 });

return (
Expand Down Expand Up @@ -245,12 +241,11 @@ Open `http://localhost:3000`. Type a message - you'll see tokens stream in real

## What's happening

1. `useClientTransport()` creates a transport that subscribes to the Ably channel before it attaches - no messages are lost.
2. `useChatTransport()` wraps the transport into Vercel's `ChatTransport` interface, which `useChat()` expects.
3. When you send a message, `useChat()` calls the chat transport's `sendMessages()`, which fires an HTTP POST to `/api/chat` and opens a stream on the Ably channel.
4. The server creates a turn, publishes user messages, streams the LLM response through the encoder to the channel, and publishes a turn-end event.
5. The client transport decodes incoming Ably messages through `UIMessageCodec` and routes them to the stream.
6. `useMessageSync()` syncs messages from the transport (including messages from other clients) into `useChat()`'s state.
1. `useChatTransport()` creates a transport that subscribes to the Ably channel before it attaches - no messages are lost. It wraps the transport into Vercel's `ChatTransport` interface, which `useChat()` expects, and returns both.
2. When you send a message, `useChat()` calls the chat transport's `sendMessages()`, which fires an HTTP POST to `/api/chat` and opens a stream on the Ably channel.
3. The server creates a turn, publishes user messages, streams the LLM response through the encoder to the channel, and publishes a turn-end event.
4. The client transport decodes incoming Ably messages through `UIMessageCodec` and routes them to the stream.
5. `useMessageSync()` syncs messages from the transport (including messages from other clients) into `useChat()`'s state.

For the conceptual details, see [Client and server transport](../concepts/transport.md) and [Turns](../concepts/turns.md).

Expand Down
1 change: 1 addition & 0 deletions docs/get-started/vercel-use-client-transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ function ChatInner({ chatId, clientId }: { chatId: string; clientId?: string })
channel,
codec: UIMessageCodec,
clientId,
api: '/api/chat',
body: () => ({ id: chatId }),
});

Expand Down
15 changes: 10 additions & 5 deletions docs/reference/react-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ const transport = useClientTransport<TEvent, TMessage>(options: ClientTransportO
| `options.channel` | `Ably.RealtimeChannel` | The Ably channel to subscribe to |
| `options.codec` | `Codec<TEvent, TMessage>` | The codec for encoding/decoding |
| `options.clientId` | `string?` | Client identity, sent to the server in POST body |
| `options.api` | `string?` | Server endpoint URL. Default: `"/api/chat"` |
| `options.api` | `string` | Server endpoint URL (required) |
| `options.headers` | `Record<string, string> \| (() => Record<string, string>)?` | HTTP POST headers. Function form for dynamic values |
| `options.body` | `Record<string, unknown> \| (() => Record<string, unknown>)?` | Additional POST body fields. Function form for dynamic values |
| `options.credentials` | `RequestCredentials?` | Fetch credentials mode |
Expand Down Expand Up @@ -243,10 +243,10 @@ Import from `@ably/ai-transport/vercel/react`.

### useChatTransport

Create and memoize a `ChatTransport` for Vercel's `useChat()` hook.
Create and memoize a `ChatTransport` and its underlying `ClientTransport` for Vercel's `useChat()` hook.

```typescript
const chatTransport = useChatTransport(
const { chatTransport, transport } = useChatTransport(
transportOrOptions: ClientTransport<UIMessageChunk, UIMessage> | VercelClientTransportOptions,
chatOptions?: ChatTransportOptions,
);
Expand All @@ -257,7 +257,12 @@ const chatTransport = useChatTransport(
| `transportOrOptions` | `ClientTransport \| VercelClientTransportOptions` | An existing transport, or options to create one |
| `chatOptions` | `ChatTransportOptions?` | Optional hooks for customizing request construction |

**Returns:** `ChatTransport` - compatible with `useChat()`'s `transport` option.
**Returns:** `ChatTransportHandle`

| Property | Type | Description |
| --------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------- |
| `chatTransport` | `ChatTransport` | Compatible with `useChat()`'s `transport` option |
| `transport` | `ClientTransport<UIMessageChunk, UIMessage>` | The underlying transport for use with `useMessageSync`, `useActiveTurns`, `useView`, etc. |

Two usage patterns:
Comment thread
lawrence-forooghian marked this conversation as resolved.

Expand All @@ -267,7 +272,7 @@ Two usage patterns:
`ChatTransportOptions.prepareSendMessagesRequest` lets you customize the HTTP POST body and headers:

```typescript
const chatTransport = useChatTransport(transport, {
const { chatTransport } = useChatTransport(transport, {
prepareSendMessagesRequest: (context) => ({
body: { history: context.history, sessionId: mySessionId },
headers: { 'x-custom': 'value' },
Expand Down
2 changes: 1 addition & 1 deletion src/core/transport/client-transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ class DefaultClientTransport<TEvent, TMessage> implements ClientTransport<TEvent
this._channel = options.channel;
this._codec = options.codec;
this._clientId = options.clientId;
this._api = options.api ?? '/api/chat';
this._api = options.api;
this._credentials = options.credentials;
// CAST: TS can't narrow options.headers/body inside a closure because the outer
// object is mutable. The truthiness check on the preceding line guarantees non-nullish.
Expand Down
4 changes: 2 additions & 2 deletions src/core/transport/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -239,8 +239,8 @@ export interface ClientTransportOptions<TEvent, TMessage> {
/** The client's identity. Sent to the server in the POST body. */
clientId?: string;

/** Server endpoint URL for the HTTP POST. Defaults to `"/api/chat"`. */
api?: string;
/** Server endpoint URL for the HTTP POST. */
api: string;

/** Headers for the HTTP POST. Function form for dynamic values (e.g. auth tokens). */
headers?: Record<string, string> | (() => Record<string, string>);
Expand Down
1 change: 1 addition & 0 deletions src/vercel/react/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Vercel-specific React hooks
export type { ChatTransport } from '../transport/chat-transport.js';
export type { ChatTransportHandle } from './use-chat-transport.js';
export { useChatTransport } from './use-chat-transport.js';
export { useMessageSync } from './use-message-sync.js';
38 changes: 30 additions & 8 deletions src/vercel/react/use-chat-transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
* Both forms accept an optional second argument for ChatTransportOptions
* (e.g. prepareSendMessagesRequest for the persistence pattern).
*
* Returns a {@link ChatTransportHandle} containing both the ChatTransport
* (for useChat) and the underlying ClientTransport (for useMessageSync,
* useActiveTurns, useView, etc.).
*
* The hook does NOT auto-close the transport on unmount. Channel lifecycle is
* managed by the Ably provider (useChannel). Auto-closing would break React
* Strict Mode. Call chatTransport.close() explicitly if needed.
Expand All @@ -23,6 +27,18 @@ import { createChatTransport } from '../transport/chat-transport.js';
import type { VercelClientTransportOptions } from '../transport/index.js';
import { createClientTransport as createCoreClientTransport } from '../transport/index.js';

/**
* Handle returned by {@link useChatTransport}, providing both the
* Vercel-compatible {@link ChatTransport} and the underlying
* {@link ClientTransport} for use with generic hooks.
*/
export interface ChatTransportHandle {
/** The ChatTransport adapter for Vercel's useChat hook. */
chatTransport: ChatTransport;
/** The underlying ClientTransport for useMessageSync, useActiveTurns, useView, etc. */
transport: ClientTransport<AI.UIMessageChunk, AI.UIMessage>;
}

/**
* Type guard: distinguish an existing ClientTransport from options.
* @param x - Either a transport instance or options object.
Expand All @@ -33,28 +49,34 @@ const isClientTransport = (
): x is ClientTransport<AI.UIMessageChunk, AI.UIMessage> => 'view' in x;

/**
* Create and memoize a {@link ChatTransport} for Vercel's useChat hook.
* Create and memoize a {@link ChatTransportHandle} for Vercel's useChat hook.
*
* Pass an existing `ClientTransport` to wrap it, or pass
* `VercelClientTransportOptions` to create one internally with UIMessageCodec.
* @param transportOrOptions - An existing ClientTransport, or options to create one.
* @param chatOptions - Optional hooks for customizing request construction.
* @returns A {@link ChatTransport} compatible with Vercel's useChat hook.
* @returns A {@link ChatTransportHandle} containing the ChatTransport and underlying ClientTransport.
*/
export const useChatTransport = (
transportOrOptions: ClientTransport<AI.UIMessageChunk, AI.UIMessage> | VercelClientTransportOptions,
chatOptions?: ChatTransportOptions,
): ChatTransport => {
const chatTransportRef = useRef<ChatTransport | null>(null);
): ChatTransportHandle => {
const handleRef = useRef<ChatTransportHandle | null>(null);

if (chatTransportRef.current === null) {
if (handleRef.current === null) {
if (isClientTransport(transportOrOptions)) {
chatTransportRef.current = createChatTransport(transportOrOptions, chatOptions);
handleRef.current = {
chatTransport: createChatTransport(transportOrOptions, chatOptions),
transport: transportOrOptions,
};
} else {
const transport = createCoreClientTransport(transportOrOptions);
chatTransportRef.current = createChatTransport(transport, chatOptions);
handleRef.current = {
chatTransport: createChatTransport(transport, chatOptions),
transport,
};
}
}

return chatTransportRef.current;
return handleRef.current;
};
15 changes: 12 additions & 3 deletions src/vercel/transport/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,11 @@ import type {
} from '../../core/transport/types.js';
import { UIMessageCodec } from '../codec/index.js';

/** Options for creating a Vercel client transport. Same as core options but without the codec field. */
export type VercelClientTransportOptions = Omit<ClientTransportOptions<AI.UIMessageChunk, AI.UIMessage>, 'codec'>;
/** Core client transport options with Vercel AI SDK types pre-applied. */
type CoreClientOpts = ClientTransportOptions<AI.UIMessageChunk, AI.UIMessage>;

/** Options for creating a Vercel client transport. Same as core options but without the codec field, and with `api` optional (defaults to `"/api/chat"`). */
export type VercelClientTransportOptions = Omit<CoreClientOpts, 'codec' | 'api'> & Partial<Pick<CoreClientOpts, 'api'>>;

/** Options for creating a Vercel server transport. Same as core options but without the codec field. */
export type VercelServerTransportOptions = Omit<ServerTransportOptions<AI.UIMessageChunk, AI.UIMessage>, 'codec'>;
Expand All @@ -42,7 +45,13 @@ export type VercelServerTransportOptions = Omit<ServerTransportOptions<AI.UIMess
*/
export const createClientTransport = (
options: VercelClientTransportOptions,
): ClientTransport<AI.UIMessageChunk, AI.UIMessage> => createCoreClientTransport({ ...options, codec: UIMessageCodec });
): ClientTransport<AI.UIMessageChunk, AI.UIMessage> =>
createCoreClientTransport({
...options,
codec: UIMessageCodec,
// Mirrors the Vercel AI SDK's DefaultChatTransport default.
api: options.api ?? '/api/chat',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this /api/chat string default seems a bit buried now.. should it be some constant in this file or something?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious what value you think this would add? It's only used in this place and isn't intended as a reusable value. If you feel strongly I'm happy to change though

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's pretty hidden / unclear what would happen if you don't pass options.api. How would a developer understand the behaviour of the system?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's documented on VercelClientTransportOptions:

(defaults to "/api/chat")

});

/**
* Create a server-side transport pre-configured with the Vercel AI SDK codec.
Expand Down
Loading
Loading