Skip to content

Commit f9267cb

Browse files
ushaketsjmonson
authored andcommitted
update resolve output signature
Signed-off-by: Uri Shaket <ushaket@redhat.com>
1 parent f676845 commit f9267cb

File tree

2 files changed

+4
-3
lines changed

2 files changed

+4
-3
lines changed

src/guidellm/backends/openai/http.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -279,7 +279,7 @@ async def resolve( # type: ignore[override]
279279
request: GenerationRequest,
280280
request_info: RequestInfo,
281281
history: list[tuple[GenerationRequest, GenerationResponse]] | None = None,
282-
) -> AsyncIterator[tuple[GenerationResponse, RequestInfo]]:
282+
) -> AsyncIterator[tuple[GenerationResponse | None, RequestInfo]]:
283283
"""
284284
Process generation request and yield progressive responses.
285285

src/guidellm/scheduler/schemas.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -119,14 +119,15 @@ async def resolve(
119119
request: RequestT,
120120
request_info: RequestInfo,
121121
history: list[tuple[RequestT, ResponseT]] | None = None,
122-
) -> AsyncIterator[tuple[ResponseT, RequestInfo]]:
122+
) -> AsyncIterator[tuple[ResponseT | None, RequestInfo]]:
123123
"""
124124
Process a request and yield incremental response updates.
125125
126126
:param request: The request object to process
127127
:param request_info: Scheduling metadata and timing information
128128
:param history: Conversation history for multi-turn requests
129-
:yield: Tuples of (response, updated_request_info) for each response chunk
129+
:yield: Tuples of (response, updated_request_info) for each response chunk.
130+
Response may be None for intermediate updates (e.g., first token arrival).
130131
:raises Exception: Implementation-specific exceptions for processing failures
131132
"""
132133

0 commit comments

Comments
 (0)