Motivation
The Amazon Web Services NLP adapter currently does not use structured outputs, which may result in errors during inference. AWS has recently introduced a Structured Outputs feature, which requires models to adhere to provided JSON schemas.
Solution Proposal
Modify the AWS NLP adapter to use Structured Outputs instead of standard prompts as described in
Discussion
This FR is paired with #678, which would provide the same constrained sampling feature for user-provided open weights models.
Motivation
The Amazon Web Services NLP adapter currently does not use structured outputs, which may result in errors during inference. AWS has recently introduced a Structured Outputs feature, which requires models to adhere to provided JSON schemas.
Solution Proposal
Modify the AWS NLP adapter to use Structured Outputs instead of standard prompts as described in
Discussion
This FR is paired with #678, which would provide the same constrained sampling feature for user-provided open weights models.