feat(anthropic): text stream support

[sentiasa]

Description

• Basic stream handling
• Tool call processing and handling
• Thinking content extraction
• Citation handling
• Rate limit processing

Breaking Changes

none

[kinsta]

Preview deployments for prism ⚡️

Status	Branch preview	Commit preview
❌ Failed to deploy	N/A	N/A

Commit: 04a0aff955203c9bfba5f6b15db4e1a242355f05

Deployment ID: fb34d344-5f3b-409f-85e3-056bd7c42845

Static site name: prism-97nz9

[sentiasa]

Hi @ChrisB-TL

Thanks for the feedback. I’ve made some changes, but I could use some guidance on the remaining parts.

1. For thinking, how should we handle the record/return of thinking and thinking signature values? Should we send them through stream? One way I could think of is returning it in yield new Chunk(content: !empty($additionalContent) ? json_encode($additionalContent) : null. Not sure if this is the right direction.

2. For RateLimits, I saw how it('adds rate limit data to the responseMeta') handles the RateLimiting, however I am not sure how to test it with Stream. Do you know how I can test it?

3. I don't fully grasp the Step concept (AnthropicText) and how it should play with Stream.

This turned out to be more complex than I anticipated, but with some guidance I believe I can crack it

[sentiasa]

I applied some fixes and refactoring based on your feedback, and fixed the formatting. The implementation now handles rate limits correctly, and properly processes citations and thinking signatures.

Any feedback would be very helpful.

[ChrisB-TL]

@sentiasa would it be possible to add me to your repo so I can make some changes directly?

[sentiasa]

@ChrisB-TL added you to the repo

[ChrisB-TL]

@ChrisB-TL added you to the repo

Thanks - I'll push a few commits by "topic" - hopefully making my changes make sense!

[ChrisB-TL]

Changes made directly.

I think I am done with this. In summary I have:

• Shared error handling code across the text/structured base handler and streaming.
• Added the concept of a meta chunk - the first and last - which the dev probably won't want to forward to the frontend. This should lay the groundwork as/when we add usage data also.
• Refactored the processStream method to use a match statement and defer to separate handler methods, and added comments to explain what is going on given its complexity.
• Clarified what is going on with state.
• Added yielding of thinking chunks with a chunkType of Thinking (so dev can decide whether or not to forward to frontend).
• Fixed citations - this was knarly as it does not work quite the same way as a standard response. I've added a note to the docs to explain.
• Re-used the text handler's build payload method, as its the same other than the stream property.

It is a pretty complex feature for Anthropic given citations, thinking etc. and I think it could do with further cleaning up for readability / maintenance. But I think good enough for now if @sixlive agrees?

[sentiasa]

@ChrisB-TL

Thanks a lot for all the refactoring on the PR! You've really shaped the code to align well with the rest of the library. Sorry if I caused some pain due to my lack of knowledge with the rest of the library (and I did underestimate the complexity of Anthropic Stream) 🤦‍♂️

The ChunkType approach you implemented is quite smart. I'm now thinking about how we could extend this to provide better signalling for tool execution to the frontend - specifically for "tool_started" and "tool_finished" events that would help to display tool execution status in the frontend. Would ChunkType be the foundation bricks to achieve this?

[ChrisB-TL]

Not at all - it's a tricky one as Anthropic is so feature rich!

That's a great idea on using the chunkType for tools. You could add metadata about the tool - name, input data etc. - to additionalData.

I'd suggest getting this PR done first, then we can loop back over the stream handlers to do that if @sixlive likes the idea too!

[sentiasa]

@ChrisB-TL I finally had chance to test it further.

I realised an issue with the handleMessageStop's Chunk. The stop_reason is not in the message_stop event, but in the last message_delta event instead. I introduced $this->stopReason

Also within FinishReasonMap 'message_stop' was introduced by me and is a mistake, because the actual stop reason should be message_delta's stop_reason. I pushed a fix

[sentiasa]

I found some more room for improvements.

I was trying to setup history and I had difficulty with multiple tool-call case. Specifically;

assistant:
"doing X text"
[tool_use]
"doing Y text"
[tool_use]

1. The Stop Chunk (Stream Line 305) is fired for each subsequent request, so it is difficult to catch the true final event. Is returning the Chunk only if ($depth == 0) a good idea?

2. Once the Stop Chunk is received on the application side, it doesn't contain any information to craft the history (e.g. text, tool_use, text, tool_use). As I understand, the Step approach is built for this use-case. Would it make sense to initiate ResponseBuilder and create a method addStep() similar to Anthropic/Text?