Skip to content

fix: 处理 streaming 响应中 choices 为空的情况#377

Open
KangweiZhu wants to merge 1 commit intodatawhalechina:mainfrom
KangweiZhu:main
Open

fix: 处理 streaming 响应中 choices 为空的情况#377
KangweiZhu wants to merge 1 commit intodatawhalechina:mainfrom
KangweiZhu:main

Conversation

@KangweiZhu
Copy link

使用NVIDIA NIM提供的minimax-m2.5 API,运行python llm_client.py 会报

content = chunk.choices[0].delta.content or ''
              ~~~~~~~~~~~~~^^^ IndexError: list index out of range

Debug后发现,一些模型的最后一个chunk会是token usage报告, 其choices为空

ChatCompletionChunk(id='chatcmpl-81e6962c93ce26f7', choices=[Choice(delta=ChoiceDelta(content='性能。祝 你编码顺利!', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason='stop', index=0, logprobs=None, stop_reason=None, token_ids=None)], created=1772561517, model='minimaxai/minimax-m2.5', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)

ChatCompletionChunk(id='chatcmpl-81e6962c93ce26f7', choices=[], created=1772561517, model='minimaxai/minimax-m2.5', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=1368, prompt_tokens=43, total_tokens=1411, completion_tokens_details=None, prompt_tokens_details=None))

模型完成响应

<think>The user asks in Chinese: "写一个快速排序算法" i.e., "Write a quicksort algorithm." This is a straightforward request for code. The policy: It's allowed content. The user asks for an algorithm; we can provide code in some language, maybe Python, perhaps also discuss algorithm.

加一条额外判断就可以解决问题。

使用NVIDIA NIM提供的minimax-m2.5 API,运行`python llm_client.py` 会报
```
content = chunk.choices[0].delta.content or ''
              ~~~~~~~~~~~~~^^^ IndexError: list index out of range
```

Debug后发现,一些模型的最后一个chunk会是token usage报告, 其choices为空
```
ChatCompletionChunk(id='chatcmpl-81e6962c93ce26f7', choices=[Choice(delta=ChoiceDelta(content='性能。祝 你编码顺利!', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason='stop', index=0, logprobs=None, stop_reason=None, token_ids=None)], created=1772561517, model='minimaxai/minimax-m2.5', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None)

ChatCompletionChunk(id='chatcmpl-81e6962c93ce26f7', choices=[], created=1772561517, model='minimaxai/minimax-m2.5', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=1368, prompt_tokens=43, total_tokens=1411, completion_tokens_details=None, prompt_tokens_details=None))

模型完成响应

<think>The user asks in Chinese: "写一个快速排序算法" i.e., "Write a quicksort algorithm." This is a straightforward request for code. The policy: It's allowed content. The user asks for an algorithm; we can provide code in some language, maybe Python, perhaps also discuss algorithm.
```
加一条额外判断就可以解决问题。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant