Skip to content

Feature/bedrock structured outputs#2081

Closed
deepakdalakoti wants to merge 2 commits intoconfident-ai:mainfrom
deepakdalakoti:feature/bedrock-structured-outputs
Closed

Feature/bedrock structured outputs#2081
deepakdalakoti wants to merge 2 commits intoconfident-ai:mainfrom
deepakdalakoti:feature/bedrock-structured-outputs

Conversation

@deepakdalakoti
Copy link
Copy Markdown

This PR adds support for structured outputs in AmazonBedrockModel. Main changes are described below:

  1. Adds an optional parameter to AmazonBedrockModel for structured output. When set as true, uses tool calling to enforce structured outputs. Added functionality to convert a pydantic output schema (in a new amazon_bedrock_utils.py file) to a Bedrock Converse API compatible tool specification
  2. Removed hardcoded max_tokens and top_p from generation_kwargs for bedrock models. These default values can cause tests to artifically fail. For example, when the test output exceeds max_tokens, the generation is stopped which causes the schema parsing to fail and thus the test fails. Setting these values should be upto the user. If no values are provided, bedrock uses the default values for the model and max_tokens are set to maximum tokens allowed for a particular model
  3. Updated documentation

@vercel
Copy link
Copy Markdown

vercel Bot commented Sep 21, 2025

Someone is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

@deepakdalakoti deepakdalakoti marked this pull request as ready for review September 21, 2025 23:11
@trevor-cai
Copy link
Copy Markdown
Contributor

Thanks a lot for this contribution, @deepakdalakoti! Structured outputs for Bedrock is definitely something we want to support, and your PR highlights both the approach and some important cleanup (like removing hardcoded generation params). We’re currently working on a broader refactor to make structured outputs opt-in across all models and metrics, and Bedrock will be included in that effort. Closing this for now so the changes can be integrated as part of that work, but I really appreciate you surfacing this and giving us a solid starting point.

@trevor-cai trevor-cai closed this Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants