Pass through cachePoint and other Bedrock-native content blocks#2078
Open
debu-sinha wants to merge 1 commit into567-labs:mainfrom
Open
Pass through cachePoint and other Bedrock-native content blocks#2078debu-sinha wants to merge 1 commit into567-labs:mainfrom
debu-sinha wants to merge 1 commit into567-labs:mainfrom
Conversation
Fixes 567-labs#1954. The _to_bedrock_content_items function was raising ValueError for unrecognized Bedrock-native dict content blocks like cachePoint, guardContent, audio, and video. These are valid Bedrock Converse API content types that should be passed through to the API unchanged, just like text, image, and document blocks. Instead of maintaining an explicit allowlist that breaks whenever AWS adds new content types, pass through any dict that does not match the OpenAI-style format (no "type" key). The Bedrock API itself validates content blocks, so instructor does not need to be the gatekeeper here. Signed-off-by: debu-sinha <debusinha2009@gmail.com>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1954
What changed
_to_bedrock_content_itemswas raisingValueErrorfor any Bedrock-native dict content block it didn't explicitly recognize. This broke caching functionality (cachePoint) introduced in Bedrock's Converse API, and would also breakguardContent,audio,video, and any other content block types AWS adds in the future.The fix replaces the
ValueErrorwith a pass-through for unrecognized Bedrock-native dicts (dicts without a"type"key, which distinguishes them from OpenAI-style parts). The Bedrock API validates content blocks on its end, so instructor doesn't need to maintain a hardcoded allowlist that falls behind API changes.Changes
instructor/providers/bedrock/utils.py: ReplaceValueErrorwith pass-through for unrecognized Bedrock-native content blocks. Updated docstring to documentcachePointand other supported types.tests/llm/test_bedrock/test_bedrock_native_passthrough.py: Added regression tests forcachePoint(with and without TTL),guardContent,audio,video, and mixed content with cache points matching the exact reproduction from [bedrock] - Bedrock caching broken in v1.13.0 - ValueError with cachePoint dict content #1954.Testing
All 32 bedrock tests pass (11 new, 21 existing):
Also verified against the exact reproduction code from #1954 -- the
ValueErroris gone and cachePoint dicts pass through correctly.