Skip to content

[Enhancement] Activate Structured Outputs for AWS models #691

@iwr-redmond

Description

@iwr-redmond

Motivation

The Amazon Web Services NLP adapter currently does not use structured outputs, which may result in errors during inference. AWS has recently introduced a Structured Outputs feature, which requires models to adhere to provided JSON schemas.

Solution Proposal

Modify the AWS NLP adapter to use Structured Outputs instead of standard prompts as described in

Discussion

This FR is paired with #678, which would provide the same constrained sampling feature for user-provided open weights models.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions