Skip to content

Commit bfaf222

Browse files
committed
fix(client): stream terminator echoes handler-supplied finish_reason
_handle_chat's two streaming paths (async-gen + sync-gen) hardcoded finish_reason="stop" on the synthetic done=True chat-chunk terminator. A handler that yielded a chunk with finish_reason="length" (max-tokens truncation) had it silently rewritten to "stop" on the wire, so the client couldn't distinguish a clean stop from a truncation. The non-streaming accumulate paths already carry the handler's finish_reason through _finalize_accumulated; the JS port was fixed for the streaming paths in 7427ed4. Track the last finish_reason seen via _chunk_fields and echo it on the terminator, falling back to "stop" when the handler never sets one.
1 parent c45fdca commit bfaf222

1 file changed

Lines changed: 10 additions & 2 deletions

File tree

zhub/client.py

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -359,9 +359,13 @@ async def _handle_chat(pub: ZhubPublication, ws, env: Envelope) -> None:
359359
# streaming HTTP callers don't time out.
360360
if inspect.isasyncgen(result):
361361
if streaming_requested:
362+
final_finish: Optional[str] = None
362363
async for chunk in result:
364+
_, _, finish = _chunk_fields(chunk)
365+
if finish:
366+
final_finish = finish
363367
await ws.send(_serialize_stream_chunk(chunk, env.request_id))
364-
await ws.send(chat_chunk("", env.request_id, done=True, finish_reason="stop").to_json())
368+
await ws.send(chat_chunk("", env.request_id, done=True, finish_reason=final_finish or "stop").to_json())
365369
return
366370
else:
367371
text_parts: list[str] = []
@@ -380,9 +384,13 @@ async def _handle_chat(pub: ZhubPublication, ws, env: Envelope) -> None:
380384
return
381385
if inspect.isgenerator(result):
382386
if streaming_requested:
387+
final_finish = None
383388
for chunk in result:
389+
_, _, finish = _chunk_fields(chunk)
390+
if finish:
391+
final_finish = finish
384392
await ws.send(_serialize_stream_chunk(chunk, env.request_id))
385-
await ws.send(chat_chunk("", env.request_id, done=True, finish_reason="stop").to_json())
393+
await ws.send(chat_chunk("", env.request_id, done=True, finish_reason=final_finish or "stop").to_json())
386394
return
387395
else:
388396
text_parts = []

0 commit comments

Comments
 (0)