feat: add get_connector_manifest command to retrieve connector metadata

[aaronsteers]

feat: add get_connector_manifest command to retrieve connector metadata

Summary

This PR adds a new MCP tool get_connector_manifest to the connector-builder-mcp project that allows retrieval of connector metadata YAML files from the connectors.airbyte.com endpoint. The function accepts a connector name (required) and version (optional, defaults to 'latest') and returns the raw metadata content as a string.

Key implementation details:

• Uses the URL pattern: https://connectors.airbyte.com/metadata/airbyte/{connector_name}/{version}/metadata.yaml
• Supports version prefix cleaning (removes 'v' if present)
• Includes proper HTTP error handling and timeout (30s)
• Follows existing code patterns for tool registration and HTTP requests
• Returns descriptive error messages for failed requests

Note: The function name suggests it returns "manifest" files, but it actually fetches "metadata.yaml" files from the connectors.airbyte.com endpoint structure.

Review & Testing Checklist for Human

• Test with multiple connector types: Verify the URL pattern works for both source and destination connectors (e.g., source-github, destination-postgres)
• Test version handling: Try with specific versions, 'latest', and versions with 'v' prefix to ensure proper handling
• Verify returned content utility: Check that the returned metadata.yaml content contains useful information for connector building workflows
• Test error scenarios: Try with invalid connector names, network issues, and non-existent versions to ensure graceful error handling
• Consider function naming: Evaluate if get_connector_manifest is the appropriate name given it returns metadata.yaml files, not traditional manifest.yaml files

Recommended test plan:

1. Test the new tool via MCP client with known connectors like source-faker, destination-postgres
2. Verify content format and usefulness for connector development workflows
3. Test edge cases: invalid names, network failures, malformed versions

---

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    Server["connector_builder_mcp/server.py"]:::context
    ConnectorBuilder["connector_builder_mcp/_connector_builder.py"]:::major-edit
    FastMCP["FastMCP App"]:::context
    ExternalAPI["connectors.airbyte.com"]:::context
    
    Server --> |registers tools| FastMCP
    ConnectorBuilder --> |register_connector_builder_tools| FastMCP
    ConnectorBuilder --> |HTTP GET requests| ExternalAPI
    
    subgraph "New Function"
        GetManifest["get_connector_manifest()"]:::major-edit
    end
    
    ConnectorBuilder --> GetManifest
    GetManifest --> |"GET /metadata/airbyte/{name}/{version}/metadata.yaml"| ExternalAPI
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit  
        L3["Context/No Edit"]:::context
    end

classDef major-edit fill:#90EE90
classDef minor-edit fill:#87CEEB
classDef context fill:#FFFFFF

Loading

Notes

• URL Pattern Discovery: During development, I initially used an incorrect URL pattern (files/metadata/airbyte/...) and had to correct it to metadata/airbyte/... after investigating the actual connectors.airbyte.com bucket structure
• Limited Testing: Only tested with destination-amazon-sqs during development - broader testing across connector types is needed
• Function vs File Naming: There's a semantic mismatch where the function is named get_connector_manifest but fetches metadata.yaml files rather than traditional manifest.yaml files

Link to Devin run: https://app.devin.ai/sessions/050f37720b8f485a86ad1fbe51d5e013
Requested by: AJ Steers (@aaronsteers)

[devin-ai-integration]

Original prompt from AJ Steers

@Devin - for connector-builder-mcp, please add a command: get_connector_manifest. Given a connector name, it should return the 'raw' github content from master, assuming the (approximate) file path: airbyte/integrations/airbyte-connectors/{connector-name}/manifest.yaml

Or ( better), pull from the versioned <http://connectors.airbyte.com|connectors.airbyte.com> endpoint, which could accept an optional version, defaulting to 'latest'.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This Branch via MCP

To test the changes in this specific branch with an MCP client like Claude Desktop, use the following configuration:

{
  "mcpServers": {
    "connector-builder-mcp-dev": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/airbytehq/connector-builder-mcp.git@devin/1754439247-add-get-connector-manifest", "connector-builder-mcp"]
    }
  }
}

Testing This Branch via CLI

You can test this version of the MCP Server using the following CLI snippet:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/connector-builder-mcp.git@devin/1754439247-add-get-connector-manifest#egg=airbyte-connector-builder-mcp' --help

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poe <command> - Runs any poe command in the uv virtual environment

📝 Edit this welcome message.

[Copilot]

Pull Request Overview

This PR adds a new MCP tool get_connector_manifest that retrieves connector metadata YAML files from the Airbyte connector registry. The tool enables users to fetch metadata information for any connector by name and optionally by version.

Key changes:

• New function to fetch connector metadata from connectors.airbyte.com endpoint
• Support for version handling with automatic prefix cleaning
• Integration with existing FastMCP tool registration system

Comments suppressed due to low confidence (1)

connector_builder_mcp/_connector_builder.py:455

• The function name 'get_connector_manifest' is misleading since it fetches metadata.yaml files, not manifest files. Consider renaming to 'get_connector_metadata' to better reflect its actual behavior.

def get_connector_manifest(

[github-actions]

PyTest Results (Full)

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit 8431759. ± Comparison against base commit 36252a7.

♻️ This comment has been updated with latest results.

[github-actions]

PyTest Results (Fast)

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit 8431759. ± Comparison against base commit 36252a7.

♻️ This comment has been updated with latest results.