feat: filter model list by permissions

[clorichel]

The bedrock:ListFoundationModels and bedrock:ListInferenceProfiles IAM permission model requires resource to be *, so in effect all available models are returned in the list.

But the user/role listing those models could actually be limited to invoke only a subset of those, through resources like arn:aws:bedrock:*::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0 and arn:aws:bedrock:*:ACTUAL_ACCOUNT_ID:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0

These changes automatically filter out any model that is available, but which the user couldn't invoke anyway with their current permissions.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[clorichel]

Be aware if your tool makes frequent/lots of GET /api/v1/models requests to this gateway using the changes of this MR (Open WebUI users 👋), that tool will feel laggy because of the permissions requests those changes introduced. A naive approach could be to cache the model list, but remember to restart your gateway when you add models or change permissions!

diff --git a/bedrock-access-gateway/api/models/bedrock.py b/bedrock-access-gateway/api/models/bedrock.py
index eddd19887d5b2317ce33ba45ddfc6fdc999bef34..c66d5694a0ec9dacb1dd42fedf5b734ab9315a14 100644
--- a/bedrock-access-gateway/api/models/bedrock.py
+++ b/bedrock-access-gateway/api/models/bedrock.py
@@ -296,15 +296,25 @@ def list_bedrock_models() -> dict:
     return model_list
 
 
-# Initialize the model list.
-bedrock_model_list = list_bedrock_models()
+# Initialize model list once at startup (cached permanently)
+bedrock_model_list = None
+
+def initialize_model_cache():
+    """Initialize model list once at application startup"""
+    global bedrock_model_list
+    if bedrock_model_list is None:
+        logger.info("Initializing model list cache at startup...")
+        bedrock_model_list = list_bedrock_models()
+        logger.info(f"Model cache initialized with {len(bedrock_model_list)} models")
 
+initialize_model_cache()
 
 class BedrockModel(BaseChatModel):
     def list_models(self) -> list[str]:
-        """Always refresh the latest model list"""
+        """Return cached model list (no refresh)"""
         global bedrock_model_list
-        bedrock_model_list = list_bedrock_models()
+        if bedrock_model_list is None:
+            raise Exception("Model list not initialized")
         return list(bedrock_model_list.keys())
 
     def validate(self, chat_request: ChatRequest):

[brian-mayer]

[like] Mayer, Brian M. reacted to your message:

…

________________________________ From: Pierre-Alexandre ***@***.***> Sent: Monday, August 18, 2025 10:08:50 AM To: aws-samples/bedrock-access-gateway ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [aws-samples/bedrock-access-gateway] feat: filter model list by permissions (PR #123) CAUTION: This email originated from outside of Thermo Fisher Scientific. If you believe it to be suspicious, report using the 'Report this email' button, which is displayed with a shield icon in Outlook, or send to ***@***.*** [https://avatars.githubusercontent.com/u/12629433?s=20&v=4]clorichel left a comment (aws-samples/bedrock-access-gateway#123)<https://urldefense.com/v3/__https://github.com/aws-samples/bedrock-access-gateway/pull/123*issuecomment-3196027856__;Iw!!HLrAl2XzZ3iCLg!BmqP2YaeEVKZk0yEXF8W_5vzQb3N36yTxewXp0urhQasMVOMIxIiXWkbDKlsmJUckcYY-BRkoZcNQnuujOzBAaolpOfrVNs$> Be aware if your tool makes frequent/lots of GET /api/v1/models requests to this gateway using the changes of this MR (Open WebUI users 👋), that tool will feel laggy because of the permissions requests those changes introduced. A naive approach could be to cache the model list, but remember to restart your gateway when you add models or change permissions! diff --git a/apps/aws-cdk-stack/src/constructs/eks/bedrock-access-gateway/api/models/bedrock.py b/apps/aws-cdk-stack/src/constructs/eks/bedrock-access-gateway/api/models/bedrock.py index eddd19887d5b2317ce33ba45ddfc6fdc999bef34..c66d5694a0ec9dacb1dd42fedf5b734ab9315a14 100644 --- a/apps/aws-cdk-stack/src/constructs/eks/bedrock-access-gateway/api/models/bedrock.py +++ b/apps/aws-cdk-stack/src/constructs/eks/bedrock-access-gateway/api/models/bedrock.py @@ -296,15 +296,25 @@ def list_bedrock_models() -> dict: return model_list -# Initialize the model list. -bedrock_model_list = list_bedrock_models() +# Initialize model list once at startup (cached permanently) +bedrock_model_list = None + +def initialize_model_cache(): + """Initialize model list once at application startup""" + global bedrock_model_list + if bedrock_model_list is None: + logger.info("Initializing model list cache at startup...") + bedrock_model_list = list_bedrock_models() + logger.info(f"Model cache initialized with {len(bedrock_model_list)} models") +initialize_model_cache() class BedrockModel(BaseChatModel): def list_models(self) -> list[str]: - """Always refresh the latest model list""" + """Return cached model list (no refresh)""" global bedrock_model_list - bedrock_model_list = list_bedrock_models() + if bedrock_model_list is None: + raise Exception("Model list not initialized") return list(bedrock_model_list.keys()) def validate(self, chat_request: ChatRequest): — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/aws-samples/bedrock-access-gateway/pull/123*issuecomment-3196027856__;Iw!!HLrAl2XzZ3iCLg!BmqP2YaeEVKZk0yEXF8W_5vzQb3N36yTxewXp0urhQasMVOMIxIiXWkbDKlsmJUckcYY-BRkoZcNQnuujOzBAaolpOfrVNs$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AY75GR74Q66KI4WFT23CDDD3OGQ3FAVCNFSM6AAAAABZDESKW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOJWGAZDOOBVGY__;!!HLrAl2XzZ3iCLg!BmqP2YaeEVKZk0yEXF8W_5vzQb3N36yTxewXp0urhQasMVOMIxIiXWkbDKlsmJUckcYY-BRkoZcNQnuujOzBAaolQSgQnEI$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>