generated from oracle/template-repo
-
Notifications
You must be signed in to change notification settings - Fork 33
Open
Description
Some components such as LLMNode using a remote LLM, or remote tools, MCP tools do network communications.
Network communications can fail transiently (and quite often do, thanks to Murphy's law), so it would make sense to be able to configure a retry-policy on those components.
A retry-policy could look as follows:
class RetryPolicy(BaseModel):
max_attempts: int= 2
"""Maximum number of retries for a request that fails with a recoverable status. 0 allowed meaning no retry passed the max_wait"""
min_wait: float = 1
"""Minimum amount of time to wait between retries in seconds"""
max_wait: float = 8
"""Maximum amount of time to wait between 2 retries in seconds"""
backoff_factor: float = 2
"""Back-off factor to increase the amount of time between 2 retries: t(n+1) = backoff_factor * t(n)"""
jitter: Optional[Literal['equal', 'full', 'full_and_equal_for_throttle', 'decorrelated']] = "full_and_equal_for_throttle"
"""Method to add randomness to the retry time. Supported methods are:
- None: t = min(min_wait * (backoff_factor ** attempts), max_wait)
- full: t = min(random(0, min_wait * (backoff_factor ** attempts)), max_wait)
- equal: t = min(min_wait * (backoff_factor ** attempts), max_wait) * (1 + random(0, 1)) / 2)
- full_and_equal_for_throttle: full for 5xx errors and equal for 4xx errors
- decorrelated: t = min(min_wait * (backoff_factor ** attempts) + random(0, 1), max_wait)
"""
service_error_retry_on_any_5xx: bool = True
"""Whether to retry on all 5xx errors (network errors, except 501)"""
recoverable_statuses: Dict[int, List[str]] = {409: [], 429: []}
"""
Some additional statuses considered as recoverable.
By default retries on:
- 409: ...
- 429: throttling (retry after x time)
"""Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels