Skip to content

Conversation

aaronsteers
Copy link
Contributor

@aaronsteers aaronsteers commented Aug 1, 2025

feat: prefer Docker images in MCP server when Docker is available

Summary

This PR implements Docker preference functionality in the PyAirbyte MCP server by updating all get_source() calls to pass docker_image=is_docker_installed() when Docker is detected on the system. This change leverages the existing cached Docker detection utility and maintains backward compatibility with graceful fallback to other installation methods.

Key Changes:

  • Added is_docker_installed() import to MCP server modules
  • Updated 6 get_source() calls across MCP local operations and connector registry
  • Leverages existing @lru_cache decorator for performance optimization
  • Maintains existing fallback logic for cases where Docker images aren't available

Files Modified:

  • airbyte/mcp/_local_ops.py - 5 get_source() calls updated
  • airbyte/mcp/_connector_registry.py - 1 get_source() call updated

Review & Testing Checklist for Human

  • Test Docker preference end-to-end: Verify MCP server actually uses Docker images for connectors when Docker is available (test with a known connector like source-pokeapi)
  • Test fallback behavior: Confirm graceful fallback when Docker is detected but specific connector images aren't available
  • Verify non-Docker environments: Test that MCP server still works correctly in environments without Docker installed
  • Performance validation: Confirm the Docker detection caching works correctly and doesn't introduce performance regressions
  • Integration testing: Test core MCP operations (validate_config, sync_source_to_cache, etc.) work with Docker preference enabled

Recommended Test Plan:

  1. Run MCP server in Docker-enabled environment and verify connector operations use Docker images
  2. Test in Docker-disabled environment to ensure fallback works
  3. Test with connectors that have/don't have Docker images available
  4. Monitor performance of repeated MCP operations to verify caching effectiveness

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    MCP["MCP Server<br/>airbyte/mcp/server.py"]:::context
    LocalOps["Local Operations<br/>airbyte/mcp/_local_ops.py"]:::major-edit
    Registry["Connector Registry<br/>airbyte/mcp/_connector_registry.py"]:::major-edit
    Meta["Docker Detection<br/>airbyte/_util/meta.py"]:::context
    GetSource["get_source()<br/>airbyte/sources/util.py"]:::context
    Executor["get_connector_executor()<br/>airbyte/_executors/util.py"]:::context
    
    MCP --> LocalOps
    MCP --> Registry
    LocalOps --> GetSource
    Registry --> GetSource
    LocalOps --> Meta
    Registry --> Meta
    GetSource --> Executor
    
    LocalOps -.->|"docker_image=is_docker_installed()"| GetSource
    Registry -.->|"docker_image=is_docker_installed()"| GetSource
    Meta -.->|"@lru_cache cached result"| LocalOps
    Meta -.->|"@lru_cache cached result"| Registry
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit  
        L3["Context/No Edit"]:::context
    end
    
    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF
Loading

Notes

  • This implementation reuses the existing is_docker_installed() function which already has proper caching via @lru_cache
  • The get_source() function already supports the docker_image parameter, so this change integrates cleanly with existing architecture
  • Executor selection logic already handles graceful fallback when Docker images aren't available for specific connectors
  • All linting, formatting, and type checking passed successfully

Session Info: Requested by AJ Steers (@aaronsteers) in Devin session: https://app.devin.ai/sessions/5aa5c0fa6cd347808bab340f53cc603d

Summary by CodeRabbit

  • New Features
    • Improved detection of Docker installation status, enhancing connector source selection and related operations.

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

- Add is_docker_installed() import to MCP modules
- Update all get_source() calls to pass docker_image=is_docker_installed()
- Leverages existing cached Docker detection for performance
- Maintains backward compatibility with graceful fallback

Co-Authored-By: AJ Steers <[email protected]>
Copy link
Contributor

Original prompt from AJ Steers
@Devin - In our PyAirbyte MCP server, please update the MCP module to prefer Docker images if docker is available. You can cache the success/fail of "which docker" (or something better) and then just send 'docker_image=True' during get_source().

@Copilot Copilot AI review requested due to automatic review settings August 1, 2025 02:55
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements Docker preference functionality in the PyAirbyte MCP server by automatically detecting Docker availability and using Docker images when possible. The change improves performance and reliability by leveraging containerized connectors while maintaining backward compatibility.

  • Updates all get_source() calls to pass docker_image=is_docker_installed() parameter
  • Leverages existing cached Docker detection for performance optimization
  • Maintains graceful fallback to other installation methods when Docker images aren't available

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
airbyte/mcp/_local_ops.py Updated 5 get_source() calls to use Docker preference when available
airbyte/mcp/_connector_registry.py Updated 1 get_source() call to use Docker preference when available

Copy link

github-actions bot commented Aug 1, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1754007779-prefer-docker-images' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1754007779-prefer-docker-images'

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test-pr - Runs tests with the updated PyAirbyte

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

Copy link
Contributor

coderabbitai bot commented Aug 1, 2025

📝 Walkthrough

Walkthrough

The changes update the logic in two modules to pass a new docker_image keyword argument—determined by the is_docker_installed() utility—to all calls of get_source(). This adjustment ensures that the presence of Docker on the system influences how connector sources are retrieved. No other logic or public API signatures are modified.

Changes

Cohort / File(s) Change Summary
Connector Registry Docker Awareness
airbyte/mcp/_connector_registry.py
Imports is_docker_installed and passes its result as docker_image to get_source in get_connector_info.
Local Ops Docker Awareness
airbyte/mcp/_local_ops.py
Updates all internal calls to get_source to include docker_image set via is_docker_installed().

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~7 minutes

Would you like me to suggest some inline comments to clarify the purpose of the new docker_image argument for future maintainers, wdyt?


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 023892f and a122082.

📒 Files selected for processing (1)
  • airbyte/mcp/_local_ops.py (6 hunks)
✅ Files skipped from review due to trivial changes (1)
  • airbyte/mcp/_local_ops.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Pytest (All, Python 3.10, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.11, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.11, Windows)
  • GitHub Check: Pytest (All, Python 3.10, Windows)
  • GitHub Check: Pytest (No Creds)
  • GitHub Check: Pytest (Fast)
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch devin/1754007779-prefer-docker-images

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

github-actions bot commented Aug 1, 2025

PyTest Results (Fast Tests Only, No Creds)

256 tests  ±0   256 ✅ ±0   3m 29s ⏱️ +3s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 1d926df. ± Comparison against base commit 121d34f.

♻️ This comment has been updated with latest results.

- Apply consistent pattern across all get_source() calls
- Addresses GitHub comment feedback for explicit boolean values
- Ensures docker_image parameter is always True or False, never None

Co-Authored-By: AJ Steers <[email protected]>
- Convert remaining single-line get_source() calls to multiline format
- Addresses GitHub comment feedback for consistent formatting
- Improves code readability and maintainability

Co-Authored-By: AJ Steers <[email protected]>
Copy link

github-actions bot commented Aug 1, 2025

PyTest Results (Full)

318 tests  ±0   304 ✅ ±0   17m 56s ⏱️ -17s
  1 suites ±0    14 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 1d926df. ± Comparison against base commit 121d34f.

♻️ This comment has been updated with latest results.

@aaronsteers aaronsteers enabled auto-merge (squash) August 1, 2025 04:12
@aaronsteers aaronsteers merged commit 1a9c411 into main Aug 1, 2025
21 checks passed
@aaronsteers aaronsteers deleted the devin/1754007779-prefer-docker-images branch August 1, 2025 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant