Skip to content

Commit a963ae9

Browse files
authored
Revert back to iterating over lines (#680)
## Summary Partially reverts #663 to iterating over lines, but keeps the skipping of blank newlines. ## Details #663 switched the HTTP backend to iterating over byte strings. The problem is that is did not handle the case where a line was split over multiple iterations. ## Test Plan Run a benchmark with known errored request rate (preferably 0) and ensure that there are no failed requests due to `orjson.JSONDecodeError: unexpected end of data`. --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
2 parents 64c40c4 + 155efd1 commit a963ae9

File tree

1 file changed

+4
-5
lines changed
  • src/guidellm/backends/openai

1 file changed

+4
-5
lines changed

src/guidellm/backends/openai/http.py

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -473,11 +473,10 @@ async def _aiter_lines(self, stream: httpx.Response) -> AsyncIterator[str]:
473473
:param stream: HTTP response object with streaming content
474474
:yield: Lines of text from the response stream
475475
"""
476-
async for part in stream.aiter_bytes():
477-
for line in part.split(b"\n\n"):
478-
if not line:
479-
continue # Skip blank lines
480-
yield line.decode("utf-8").strip()
476+
async for line in stream.aiter_lines():
477+
if not line.strip():
478+
continue # Skip blank lines
479+
yield line
481480

482481
def _build_headers(
483482
self, existing_headers: dict[str, str] | None = None

0 commit comments

Comments
 (0)