Skip to content

Feature: Allow Disabling Streaming for AWS Bedrock #12

@panyuenlau

Description

@panyuenlau

Have you searched for this feature request?

  • I searched but did not find similar requests

Problem Statement

The streaming configuration is currently hard-coded to True: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L255, which might create issues for users that don't have streaming supported in their infrastructure.

Possible Solution

Similar to other the Azure and OpenAI providers, we shall allow users to control this streaming option via a flag: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L342-L343

Alternatives Considered

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions