feat(platform/backend): auto discover credentials in all block modules #10028

Swiftyos · 2025-05-23T20:30:06Z

This pull request introduces a refactoring of the credential management system for various provider integrations, aiming to streamline credential discovery and eliminate hardcoded credentials. It introduces a dynamic mechanism for discovering default credentials and simplifies the codebase by removing redundant definitions. Additionally, it updates the logic for handling credentials in block configurations.

Refactoring of Credential Management

Dynamic Credential Discovery: Introduced a discover_default_credentials function that dynamically collects default credentials from block modules, replacing the previous hardcoded credential definitions. This change centralizes credential discovery and reduces redundancy. (autogpt_platform/backend/backend/integrations/credentials_store.py, [1] [2]
Iterative Module Loading: Added an iter_block_modules utility to recursively load all modules within the backend.blocks package, enabling the discovery of credentials defined in each block. (autogpt_platform/backend/backend/integrations/credentials_store.py, autogpt_platform/backend/backend/integrations/credentials_store.pyL35-R62)

Updates to Block-Specific Credential Definitions

Default Credential Functions: Added default_credentials functions to multiple block _auth.py files (e.g., apollo, exa, fal, etc.) to provide default API key credentials dynamically. These functions use environment variables and fall back to placeholder values if no key is provided. [1] [2] [3] [4] [5] [6] [7]

Updates to Block Cost Configuration

Credential Dependency Check: Updated block cost configurations to include checks for the presence of credentials before associating costs with models, ensuring that only valid credentials are used. (autogpt_platform/backend/backend/data/block_cost_config.py, [1] [2] [3] [4] [5]
Fallback Credential Handling: Added a _fallback function to provide a default empty credential object for providers without discovered credentials. (autogpt_platform/backend/backend/data/block_cost_config.py, autogpt_platform/backend/backend/data/block_cost_config.pyL23-R66)

General Improvements

Removed Redundant Imports: Cleaned up unused imports and added necessary ones, such as SecretStr from pydantic. (autogpt_platform/backend/backend/data/block_cost_config.py, autogpt_platform/backend/backend/data/block_cost_config.pyR3-R4)
Simplified Default Credential List: Consolidated the DEFAULT_CREDENTIALS list to include only the dynamically discovered credentials and the special case for ollama. (autogpt_platform/backend/backend/integrations/credentials_store.py, autogpt_platform/backend/backend/integrations/credentials_store.pyL35-R62)## Changes

discover provider defaults from any block module
document new discovery behavior

Checklist

poetry run format
poetry run test (fails: FileNotFoundError: 'docker')

netlify · 2025-05-23T20:30:14Z

✅ Deploy Preview for auto-gpt-docs-dev canceled.

Name	Link
🔨 Latest commit	`96deb08`
🔍 Latest deploy log	https://app.netlify.com/projects/auto-gpt-docs-dev/deploys/6835ae3cbb29680008352324

qodo-merge-pro · 2025-05-23T20:30:31Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Logic Error The condition `if not key and ENV_VAR` in the default_credentials function may not work as intended. If ENV_VAR is set but empty, it will still return None instead of using the FAKE_API_KEY fallback. Credentials, OAuth2Credentials, Potential Bug The code checks for credentials existence in conditional statements but doesn't handle the case where _DEFAULTS.get() returns None but _fallback() is called, which could lead to unexpected behavior. anthropic_credentials: APIKeyCredentials = _DEFAULTS.get("anthropic") or _fallback( "anthropic" ) did_credentials: APIKeyCredentials = _DEFAULTS.get("d_id") or _fallback("d_id") groq_credentials: APIKeyCredentials = _DEFAULTS.get("groq") or _fallback("groq") ideogram_credentials: APIKeyCredentials = _DEFAULTS.get("ideogram") or _fallback( "ideogram" ) jina_credentials: APIKeyCredentials = _DEFAULTS.get("jina") or _fallback("jina") llama_api_credentials: APIKeyCredentials = _DEFAULTS.get("llama_api") or _fallback( "llama_api" ) open_router_credentials: APIKeyCredentials = _DEFAULTS.get("open_router") or _fallback( "open_router" ) openai_credentials: APIKeyCredentials = _DEFAULTS.get("openai") or _fallback("openai") replicate_credentials: APIKeyCredentials = _DEFAULTS.get("replicate") or _fallback( "replicate" ) revid_credentials: APIKeyCredentials = _DEFAULTS.get("revid") or _fallback("revid") unreal_credentials: APIKeyCredentials = _DEFAULTS.get("unreal") or _fallback("unreal")

deepsource-io · 2025-05-23T20:30:47Z

Here's the code health analysis summary for commits 17e973a..96deb08. View details on DeepSource ↗.

Analysis Summary

Analyzer	Status	Summary	Link
JavaScript	✅ Success		View Check ↗
Python	✅ Success	❗ 4 occurences introduced 🎯 2 occurences resolved	View Check ↗

💡 If you’re a repository administrator, you can configure the quality gates from the settings.

netlify · 2025-05-23T20:31:41Z

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	`96deb08`
🔍 Latest deploy log	https://app.netlify.com/projects/auto-gpt-docs/deploys/6835ae3cd102da0008f3a82c

Copilot

Pull Request Overview

This PR adds automatic discovery of default API key credentials across all block modules and updates related logic, tests, and documentation.

Introduce default_credentials helper functions in block auth modules for auto-discovering provider credentials.
Implement discover_default_credentials to dynamically collect defaults and update the credentials store and cost config.
Update documentation and tests to use the new discovery behavior.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
docs/content/platform/new_blocks.md	Document the new `default_credentials` helper usage
autogpt_platform/backend/test/data/test_credit.py	Update test to use `discover_default_credentials`
autogpt_platform/backend/test/data/test_credentials_store.py	Add tests for default credentials discovery in store
autogpt_platform/backend/backend/server/v2/store/image_gen.py	Replace static `ideogram_credentials` with dynamic discovery
autogpt_platform/backend/backend/integrations/credentials_store.py	Add `discover_default_credentials` and simplify default list
autogpt_platform/backend/backend/data/block_cost_config.py	Use discovered credentials in cost configuration
backend/blocks/*/_auth.py (several modules)	Add `default_credentials` helpers in block auth modules

Comments suppressed due to low confidence (8)

docs/content/platform/new_blocks.md:398

The snippet uses SecretStr but does not include an import for it. Consider adding from pydantic import SecretStr at the top of the example.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/zerobounce/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/smartlead/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/nvidia/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/jina/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/fal/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/exa/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

autogpt_platform/backend/backend/blocks/apollo/_auth.py:29

Missing import for SecretStr, causing a NameError at runtime. Add from pydantic import SecretStr.

api_key=SecretStr(key),

Copilot · 2025-05-23T20:34:04Z

docs/content/platform/new_blocks.md

+    if not key and ENV_VAR:
+        return None


The condition if not key and ENV_VAR: will always be true when key is empty because ENV_VAR is a non-empty string. This prevents the fallback to FAKE_API_KEY. Consider removing and ENV_VAR or adjusting the logic to check if the environment variable name is intended.

Suggested change

if not key and ENV_VAR:

return None

Copilot · 2025-05-23T20:34:05Z

autogpt_platform/backend/backend/integrations/credentials_store.py

-        if settings.secrets.google_maps_api_key:
-            all_credentials.append(google_maps_credentials)
-        return all_credentials
+        return [*users_credentials, *DEFAULT_CREDENTIALS]


Merging users_credentials and DEFAULT_CREDENTIALS directly may introduce duplicate entries if a user already has a default credential. Consider deduplicating by credential ID or provider.

Suggested change

return [*users_credentials, *DEFAULT_CREDENTIALS]

all_credentials = {cred.id: cred for cred in DEFAULT_CREDENTIALS}

all_credentials.update({cred.id: cred for cred in users_credentials})

return list(all_credentials.values())

Copilot · 2025-05-23T20:34:05Z

autogpt_platform/backend/backend/server/v2/store/image_gen.py

+_defaults = {c.provider: c for c in discover_default_credentials()}
+ideogram_credentials = _defaults.get("ideogram") or APIKeyCredentials(


[nitpick] Calling discover_default_credentials at import time triggers recursive module walks on startup, which could slow down loading. Consider deferring discovery to when credentials are first needed or explicitly caching the result.

Suggested change

_defaults = {c.provider: c for c in discover_default_credentials()}

ideogram_credentials = _defaults.get("ideogram") or APIKeyCredentials(

_defaults = None

def get_defaults():

global _defaults

if _defaults is None:

_defaults = {c.provider: c for c in discover_default_credentials()}

return _defaults

ideogram_credentials = get_defaults().get("ideogram") or APIKeyCredentials(

ntindle · 2025-05-23T20:42:28Z

autogpt_platform/backend/backend/data/block_cost_config.py

+_DEFAULTS = {c.provider: c for c in discover_default_credentials()}
+
+
+def _fallback(provider: str) -> APIKeyCredentials:


We should not have a fallback here

ntindle · 2025-05-23T20:42:57Z

autogpt_platform/backend/backend/data/block_cost_config.py

+    )
+
+
+anthropic_credentials: APIKeyCredentials = _DEFAULTS.get("anthropic") or _fallback(


This doesn’t do what we want. We should do this similar to getting avail blocks

- Fixed credit tests to use TEST_CREDENTIALS instead of trying to discover OpenAI credentials - Updated library db test to use correct include structure (library_agent_include) - Fixed credential discovery by adding __init__.py files to auth module directories - Updated all auth modules to use Settings class instead of os.getenv for consistency - Fixed auth modules to create Settings instance inside default_credentials function - Updated test runner to use 'prisma migrate deploy' instead of interactive 'dev' command - Skipped environment-based credential tests due to Settings caching limitations All 352 non-skipped tests now pass successfully.

…dential-functions

Swiftyos · 2025-06-02T13:07:42Z

closed in favour of #10074

feat(platform/backend): broaden credential discovery

f1fdfe1

Swiftyos requested a review from a team as a code owner May 23, 2025 20:30

Swiftyos requested review from ntindle and Pwuts and removed request for a team May 23, 2025 20:30

Swiftyos added the codex label May 23, 2025 — with ChatGPT Connector

github-project-automation bot added this to AutoGPT development kanban May 23, 2025

github-project-automation bot moved this to 🆕 Needs initial review in AutoGPT development kanban May 23, 2025

github-actions bot added documentation Improvements or additions to documentation platform/backend AutoGPT Platform - Back end platform/blocks size/xl labels May 23, 2025

qodo-merge-pro bot added the Review effort 3/5 label May 23, 2025

Swiftyos requested a review from Copilot May 23, 2025 20:31

Copilot AI reviewed May 23, 2025

View reviewed changes

ntindle reviewed May 23, 2025

View reviewed changes

Swiftyos marked this pull request as draft May 23, 2025 20:57

Swiftyos added 2 commits May 24, 2025 07:46

Merge branch 'dev' into swiftyos/add-provider-metadata-and-update-cre…

96deb08

…dential-functions

Swiftyos closed this Jun 2, 2025

github-project-automation bot moved this from 🆕 Needs initial review to ✅ Done in AutoGPT development kanban Jun 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(platform/backend): auto discover credentials in all block modules #10028

feat(platform/backend): auto discover credentials in all block modules #10028

Uh oh!

Swiftyos commented May 23, 2025 •

edited

Loading

Uh oh!

netlify bot commented May 23, 2025 •

edited

Loading

Uh oh!

qodo-merge-pro bot commented May 23, 2025

Uh oh!

deepsource-io bot commented May 23, 2025 •

edited

Loading

Analysis Summary

Uh oh!

netlify bot commented May 23, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 23, 2025

Uh oh!

Copilot AI May 23, 2025

Uh oh!

Copilot AI May 23, 2025

Uh oh!

ntindle May 23, 2025

Uh oh!

ntindle May 23, 2025

Uh oh!

Swiftyos commented Jun 2, 2025

Uh oh!

Uh oh!

-        return [*users_credentials, *DEFAULT_CREDENTIALS]
+        all_credentials = {cred.id: cred for cred in DEFAULT_CREDENTIALS}
+        all_credentials.update({cred.id: cred for cred in users_credentials})
+        return list(all_credentials.values())

		_defaults = {c.provider: c for c in discover_default_credentials()}
		ideogram_credentials = _defaults.get("ideogram") or APIKeyCredentials(

-_defaults = {c.provider: c for c in discover_default_credentials()}
-ideogram_credentials = _defaults.get("ideogram") or APIKeyCredentials(
+_defaults = None
+def get_defaults():
+    global _defaults
+    if _defaults is None:
+        _defaults = {c.provider: c for c in discover_default_credentials()}
+    return _defaults
+ideogram_credentials = get_defaults().get("ideogram") or APIKeyCredentials(

		_DEFAULTS = {c.provider: c for c in discover_default_credentials()}


		def _fallback(provider: str) -> APIKeyCredentials:

		)


		anthropic_credentials: APIKeyCredentials = _DEFAULTS.get("anthropic") or _fallback(

feat(platform/backend): auto discover credentials in all block modules #10028

feat(platform/backend): auto discover credentials in all block modules #10028

Uh oh!

Conversation

Swiftyos commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Refactoring of Credential Management

Updates to Block-Specific Credential Definitions

Updates to Block Cost Configuration

General Improvements

Checklist

Uh oh!

netlify bot commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for auto-gpt-docs-dev canceled.

Uh oh!

qodo-merge-pro bot commented May 23, 2025

PR Reviewer Guide 🔍

Uh oh!

deepsource-io bot commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Analysis Summary

Uh oh!

netlify bot commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for auto-gpt-docs canceled.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 23, 2025

Choose a reason for hiding this comment

Uh oh!

ntindle May 23, 2025

Choose a reason for hiding this comment

Uh oh!

ntindle May 23, 2025

Choose a reason for hiding this comment

Uh oh!

Swiftyos commented Jun 2, 2025

Uh oh!

Uh oh!

Swiftyos commented May 23, 2025 •

edited

Loading

netlify bot commented May 23, 2025 •

edited

Loading

deepsource-io bot commented May 23, 2025 •

edited

Loading

netlify bot commented May 23, 2025 •

edited

Loading