Replies: 5 comments 11 replies
-
|
I like this. We need an end-to-end way to give to developers so they can build great interfaces. Currently, when using Livewire, I am basically abstracting a way to pass through chunks to the frontend to then process them in JS. It makes a lot of sense to give the dev an end-to-end way of consuming the events in the frontend. I've struggled myself moving from just streaming the text to streaming whole events. I tried to keep the frontend processing to a minimum, which was wrong. Today, with thinking and tool calls, you really have to process the full stream in the frontend. There is no good in-between imo. I'd also like to see a default frontend implementation for people to reference in different stacks, kind of like @pushpak1300 or my starter kits. |
Beta Was this translation helpful? Give feedback.
-
|
That's an excellent proposal. I completely agree that an event-based system makes a lot more sense and would provide a solid, standardized architecture. |
Beta Was this translation helpful? Give feedback.
-
|
Streaming-focused feedback from production use I’m squarely focused on production apps in Laravel, not cloning OpenRouter/LiteLLM in PHP. Prism was a great starting point—I built a small lib on top that focuses only on Streaming and Structured Outputs (kept structured outputs unchanged from Prism). A few lessons from running this at scale in PHP/Laravel:
On the proposal itself: it feels closer to TS/Python streaming patterns than to PHP’s operational constraints. That’s not a knock—just a note that production-ready PHP may need adapted patterns vs. ports. Questions that would help validate the design for PHP/Laravel:
If the design can lock down the above (especially canonical usage and resumability) and provide Laravel-first guidance (queues/events/persistence), it’ll land much closer to what PHP teams need in production. |
Beta Was this translation helpful? Give feedback.
-
|
Great work @sixlive! The proposed
For example, just this week this was released: https://streamdown.ai/, which is built on those streaming standards and AI Elements (https://ai-sdk.dev/elements/overview/usage). If Prisms default worked out of the box with all of those packages I think that would be a boon to all. |
Beta Was this translation helpful? Give feedback.
-
Streaming OutputWant to show AI responses to your users in real-time? Prism provides multiple ways to handle streaming AI responses, from simple Server-Sent Events to WebSocket broadcasting for real-time applications. Warning When using Laravel Telescope or other packages that intercept Laravel's HTTP client events, they may consume the stream before Prism can emit the stream events. This can cause streaming to appear broken or incomplete. Consider disabling such interceptors when using streaming functionality, or configure them to ignore Prism's HTTP requests. Quick StartServer-Sent Events (SSE)The simplest way to stream AI responses to a web interface: Route::get('/chat', function () {
return Prism::text()
->using('anthropic', 'claude-3-7-sonnet')
->withPrompt(request('message'))
->asEventStreamResponse();
});const eventSource = new EventSource('/chat');
eventSource.addEventListener('text_delta', (event) => {
const data = JSON.parse(event.data);
document.getElementById('output').textContent += data.delta;
});
eventSource.addEventListener('stream_end', (event) => {
const data = JSON.parse(event.data);
console.log('Stream ended:', data.finish_reason);
eventSource.close();
});Vercel AI SDK IntegrationFor apps using Vercel's AI SDK, use the Data Protocol adapter which provides compatibility with the Vercel AI SDK UI: Route::post('/api/chat', function () {
return Prism::text()
->using('openai', 'gpt-4')
->withPrompt(request('message'))
->asDataStreamResponse();
});Client-side with the import { useChat } from 'ai/react';
export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
api: '/api/chat',
});
return (
<div>
<div>
{messages.map(m => (
<div key={m.id}>
{m.role}: {m.content}
</div>
))}
</div>
<form onSubmit={handleSubmit}>
<input
value={input}
placeholder="Say something..."
onChange={handleInputChange}
disabled={isLoading}
/>
<button type="submit" disabled={isLoading}>
Send
</button>
</form>
</div>
);
}For more advanced usage, including tool support and custom options, see the Vercel AI SDK UI documentation. WebSocket Broadcasting with Background JobsFor real-time multi-user applications that need to process AI requests in the background: // Job Class
<?php
namespace App\Jobs;
use Illuminate\Broadcasting\Channel;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Prism\Prism\Prism;
class ProcessAiStreamJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public function __construct(
public string $message,
public string $channel,
public string $model = 'claude-3-7-sonnet'
) {}
public function handle(): void
{
Prism::text()
->using('anthropic', $this->model)
->withPrompt($this->message)
->asBroadcast(new Channel($this->channel));
}
}
// Controller
Route::post('/chat-broadcast', function () {
$sessionId = request('session_id') ?? 'session_' . uniqid();
ProcessAiStreamJob::dispatch(
request('message'),
"chat.{$sessionId}",
request('model', 'claude-3-7-sonnet')
);
return response()->json(['status' => 'processing', 'session_id' => $sessionId]);
});Client-side with React and useEcho: import { useEcho } from '@/hooks/useEcho';
import { useState } from 'react';
function ChatComponent() {
const [currentMessage, setCurrentMessage] = useState('');
const [currentMessageId, setCurrentMessageId] = useState('');
const [isComplete, setIsComplete] = useState(false);
const sessionId = 'session_' + Date.now();
// Listen for streaming events
useEcho(`chat.${sessionId}`, {
'.stream_start': (data) => {
console.log('Stream started:', data);
setCurrentMessage('');
setIsComplete(false);
},
'.text_start': (data) => {
console.log('Text start event received:', data);
setCurrentMessage('');
setCurrentMessageId(data.message_id || Date.now().toString());
},
'.text_delta': (data) => {
console.log('Text delta received:', data);
setCurrentMessage(prev => prev + data.delta);
},
'.text_complete': (data) => {
console.log('Text complete:', data);
},
'.tool_call': (data) => {
console.log('Tool called:', data.tool_name, data.arguments);
},
'.tool_result': (data) => {
console.log('Tool result:', data.result);
},
'.stream_end': (data) => {
console.log('Stream ended:', data.finish_reason);
setIsComplete(true);
},
'.error': (data) => {
console.error('Stream error:', data.message);
}
});
const sendMessage = async (message) => {
await fetch('/chat-broadcast', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
message,
session_id: sessionId,
model: 'claude-3-7-sonnet'
})
});
};
return (
<div>
<div className="message-display">
{currentMessage}
{!isComplete && <span className="cursor">|</span>}
</div>
<button onClick={() => sendMessage("What's the weather in Detroit?")}>
Send Message
</button>
</div>
);
}Event TypesAll streaming approaches emit the same core events with consistent data structures: Available Events
Event Data ExamplesBased on actual streaming output: // stream_start event
{
"id": "anthropic_evt_SSrB7trNIXsLkbUB",
"timestamp": 1756412888,
"model": "claude-3-7-sonnet-20250219",
"provider": "anthropic",
"metadata": {
"request_id": "msg_01BS7MKgXvUESY8yAEugphV2",
"rate_limits": []
}
}
// text_start event
{
"id": "anthropic_evt_8YI9ULcftpFtHzh3",
"timestamp": 1756412888,
"message_id": "msg_01BS7MKgXvUESY8yAEugphV2",
"turn_id": null
}
// text_delta event
{
"id": "anthropic_evt_NbS3LIP0QDl5whYu",
"timestamp": 1756412888,
"delta": "💠🌐 Well hello there! You want to know",
"message_id": "msg_01BS7MKgXvUESY8yAEugphV2",
"turn_id": null
}
// tool_call event
{
"id": "anthropic_evt_qXvozT6OqtmFPgkG",
"timestamp": 1756412889,
"tool_id": "toolu_01NAbzpjGxv2mJ8gJRX5Bb8m",
"tool_name": "search",
"arguments": {"query": "current date and time in Detroit Michigan"},
"message_id": "msg_01BS7MKgXvUESY8yAEugphV2",
"reasoning_id": null
}
// stream_end event
{
"id": "anthropic_evt_BZ3rqDYyprnywNyL",
"timestamp": 1756412898,
"finish_reason": "Stop",
"usage": {
"prompt_tokens": 3448,
"completion_tokens": 192,
"cache_write_input_tokens": 0,
"cache_read_input_tokens": 0,
"thought_tokens": 0
}
}Advanced UsageCustom Event ProcessingAccess raw events for complete control over handling: $events = Prism::text()
->using('openai', 'gpt-4')
->withPrompt('Explain quantum physics')
->asStream();
foreach ($events as $event) {
match ($event->type()) {
StreamEventType::TextDelta => handleTextChunk($event),
StreamEventType::ToolCall => handleToolCall($event),
StreamEventType::StreamEnd => handleCompletion($event),
default => null,
};
}Streaming with ToolsStream responses that include tool interactions: use Prism\Prism\Facades\Tool;
$searchTool = Tool::as('search')
->for('Search for information')
->withStringParameter('query', 'Search query')
->using(function (string $query) {
return "Search results for: {$query}";
});
return Prism::text()
->using('anthropic', 'claude-3-7-sonnet')
->withTools([$searchTool])
->withPrompt("What's the weather in Detroit?")
->asEventStreamResponse();Data Protocol OutputThe Vercel AI SDK format provides structured streaming data: Configuration OptionsStreaming supports all the same configuration options as regular text generation, including temperature, max tokens, and provider-specific settings. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hey Prism community! 👋
I've been thinking a lot about our streaming implementation lately, and I want to share a proposal for a pretty significant refactor that could make building real-time AI applications way better. But first, I need your feedback.
The Current State (And Why I Think We Can Do Better)
Right now, our streaming system works around "Chunks" – objects that contain mixed data types. If you've built streaming UIs with Prism, you've probably written code like this:
This works, but I've been hearing from developers that it creates some challenges:
Question for you: Does this match your experience? What other pain points have you hit with the current streaming approach?
The Proposal: Event-Driven Streaming
What I'm proposing is a complete refactor to an event-based system. Instead of chunks with mixed data, you'd get specific events for specific things happening in the stream.
Here's what a typical conversation would look like:
Key Changes
event:fields and snake_case namingThe Magic: asEventStreamResponse()
The part I'm most excited about is this new method:
This automatically handles all the SSE formatting, headers, and streaming mechanics.
Tool Calls Get First-Class Treatment
For those of you building tool-enabled apps, tool calls would get proper event sequences:
You can watch tool arguments being built incrementally and handle the execution lifecycle properly.
Thinking Events for Reasoning Models
With reasoning models becoming more common, we'd have dedicated events for chain-of-thought:
The Developer Experience
On the frontend, you could build UIs like this:
No more weird state management or hunting for usage data across different chunk types.
Backward Compatibility
I want to be transparent: this would be a breaking change. The
asStream()method would returnStreamEventobjects instead ofChunkobjects. However, the method signature stays the same, and the migration path is pretty straightforward:The Vision
My goal is to make Prism the best way to build streaming AI applications. This event system would give you:
But I don't want to build this in a vacuum. Your feedback will shape how this actually works.
This is a proposal, not a commitment. Based on community feedback, the actual implementation might look different. But I wanted to share the direction I'm thinking and get your input before diving in.
Beta Was this translation helpful? Give feedback.
All reactions