Skip to content

feat: make it possible to configure a retry policy on components doing network communications #119

@dhilloulinoracle

Description

@dhilloulinoracle

Some components such as LLMNode using a remote LLM, or remote tools, MCP tools do network communications.

Network communications can fail transiently (and quite often do, thanks to Murphy's law), so it would make sense to be able to configure a retry-policy on those components.

A retry-policy could look as follows:

class RetryPolicy(BaseModel):

    max_attempts: int= 2
    """Maximum number of retries for a request that fails with a recoverable status. 0 allowed meaning no retry passed the max_wait"""

    min_wait: float = 1
    """Minimum amount of time to wait between retries in seconds"""

    max_wait: float = 8
    """Maximum amount of time to wait between 2 retries in seconds"""

    backoff_factor: float = 2
    """Back-off factor to increase the amount of time between 2 retries: t(n+1) = backoff_factor * t(n)"""

    jitter: Optional[Literal['equal', 'full', 'full_and_equal_for_throttle', 'decorrelated']] = "full_and_equal_for_throttle"
	"""Method to add randomness to the retry time. Supported methods are:
	- None: t = min(min_wait * (backoff_factor ** attempts), max_wait)
	- full: t = min(random(0, min_wait * (backoff_factor ** attempts)), max_wait)
	- equal: t = min(min_wait * (backoff_factor ** attempts), max_wait) * (1 + random(0, 1)) / 2)
	- full_and_equal_for_throttle: full for 5xx errors and equal for 4xx errors
	- decorrelated: t = min(min_wait * (backoff_factor ** attempts) + random(0, 1), max_wait)
	"""
	
    service_error_retry_on_any_5xx: bool = True
	"""Whether to retry on all 5xx errors (network errors, except 501)"""

    recoverable_statuses: Dict[int, List[str]] = {409: [], 429: []}
    """
	Some additional statuses considered as recoverable.
    By default retries on:
	- 409: ...
    - 429: throttling (retry after x time)
    """

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions