Skip to content

Commit 9c98aa4

Browse files
hanlaurRobertSamoilescu
authored andcommitted
Fix AdaptiveBatcher timeout calculation (#2093)
Before the fix, the timeout calculation subtracted the passed time from the timeout calculated for the previous round instead of subtracting it from the configured timeout value, causing time to pass quicker than expected.
1 parent 8e05777 commit 9c98aa4

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

mlserver/batching/adaptive.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ async def _batch_requests(self) -> AsyncIterator[BatchedRequests]:
136136

137137
# Update remaining timeout
138138
current = time.time()
139-
timeout = timeout - (current - start)
139+
timeout = self._max_batch_time - (current - start)
140140
except asyncio.TimeoutError:
141141
# NOTE: Hit timeout, continue
142142
pass

0 commit comments

Comments
 (0)