Conversation
massi-ang
left a comment
massi-ang
left a comment
There was a problem hiding this comment.
Nice feature. Please add the possibility to choose the models to be enabled via the config tool instead of hard coded values in the CDK construct.
| super(scope, id); | ||
|
|
||
| const models: SageMakerModelEndpoint[] = []; | ||
| const bedrockEnabledModels: string[] = ['anthropic.claude-3-haiku-20240307-v1:0', 'anthropic.claude-3-sonnet-20240229-v1:0']; |
There was a problem hiding this comment.
This value should come from the config.json file. You need to add a new property to the SystemConfig interface to store these values and add a multiselect option to the magic config to let the user choose which models to enable.
There was a problem hiding this comment.
Should we add a free text with comma seperated model ids or a multi-selected to select from all the available foundation models ? I would vote for free text, since it doesn't need maintance - adding new models - and a bit more user-friendly - user will type instead of scrolling through the models to select the one's needed.
There was a problem hiding this comment.
I you just want to allow-list few models, a text field could work fine, but you would still need to validate the modelIds. I think a multiselect is more user friendly and you can always populate the values dynamically by querying the bedrock API. Anyway, to add new models we also need to add them to the rest of the code.
2 participants
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.