Skip to content

Commit e2a9cd0

Browse files
Add nscale support for streaming (#10698)
1 parent 1a42245 commit e2a9cd0

File tree

2 files changed

+24
-0
lines changed

2 files changed

+24
-0
lines changed

docs/my-website/docs/providers/nscale.md

+16
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,22 @@ response = completion(
6767
print(response)
6868
```
6969

70+
```python showLineNumbers title="Nscale Text Generation - Streaming"
71+
from litellm import completion
72+
import os
73+
74+
os.environ["NSCALE_API_KEY"] = "" # your Nscale API key
75+
stream = completion(
76+
model="nscale/meta-llama/Llama-4-Scout-17B-16E-Instruct",
77+
messages=[{"role": "user", "content": "What is LiteLLM?"}],
78+
stream=True
79+
)
80+
81+
for chunk in stream:
82+
if chunk.choices[0].delta.content is not None:
83+
print(chunk.choices[0].delta.content, end="")
84+
```
85+
7086
### Image Generation
7187

7288
```python showLineNumbers title="Nscale Image Generation"

litellm/llms/nscale/chat/transformation.py

+8
Original file line numberDiff line numberDiff line change
@@ -41,4 +41,12 @@ def get_supported_openai_params(self, model: str) -> list:
4141
"n",
4242
"temperature",
4343
"top_p",
44+
"stream",
45+
"logprobs",
46+
"top_logprobs",
47+
"frequency_penalty",
48+
"presence_penalty",
49+
"response_format",
50+
"stop",
51+
"logit_bias",
4452
]

0 commit comments

Comments
 (0)