Skip to content

[Feature]: Support Retry for SendStreaming #534

@willy0416

Description

@willy0416

Is your feature request related to a problem? Please describe.
When utilizing SendStreaming, we noticed that our conversation will often terminate prematurely due to rate-limiting from Bedrock. There does not seem to be Retry support for SendStreaming as there is for Send.

Describe the solution you'd like

  • Utilizing Retry in SendStreaming as it is utilized in Send
  • Properly parsing any Bedrock converse errors in bedrock.go, as DefaultIsRetryableError seems to be unable to detect a 429 Status Code in its current state
  • Only committing tool calls to the conversation history if they are successful (otherwise, the number of expected tool call results expected after the retry differs from the number of tools successfully made)

Describe alternatives you've considered
Any alternative solutions or features you’ve thought of.

Additional context
Add any other context, links, or screenshots here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions