feat(agents): Replace openai-agents with pydantic-ai implementation #139
Conversation
- Migrated from openai-agents to pydantic-ai framework - Replaced Agent class from agents with Agent from pydantic_ai - Updated agent creation to use pydantic-ai patterns - Replaced handoff mechanism with tool-based delegation - Updated manager agent to delegate to developer using @agent.tool decorator - Replaced Runner.run with Agent.run methods - Replaced session-based management with message_history tracking - Updated MCP server integration to use pydantic_ai.mcp.MCPServerStdio - Removed function_tool decorators (pydantic-ai tools are plain functions) - Updated dependencies in pyproject.toml to use pydantic-ai and pydantic-ai-slim - Preserved manager-developer architecture and 3-phase workflow - All tests passing (98 passed, 2 skipped, 1 xfailed) Co-Authored-By: AJ Steers <[email protected]>
Original prompt from AJ Steers |
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
📝 WalkthroughWalkthroughReplaces OpenAI-specific agents and handoff models with pydantic_ai Agents and tool-based delegation/reporting, standardizes MCP integration to MCPServerStdio, simplifies run/session orchestration with session_state and message_history, changes default models to "openai:gpt-4o", and adds pydantic-ai and emoji dependencies. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant Manager as Manager Agent
participant Dev as Developer Agent
participant MCP as MCP Servers
User->>Manager: submit assignment (title, description)
Manager->>Manager: plan / decide to delegate
Manager->>Manager: call delegate_to_developer(...)
Manager->>Dev: invoke developer_agent.run(ctx, task)
activate Dev
Dev->>MCP: call MCP tools (filesystem/browser/connector)
MCP-->>Dev: return tool results
Dev->>Manager: call report_back_to_manager(status, summary)
deactivate Dev
Manager->>Manager: update session_state / message_history
alt not complete
Manager->>Dev: delegate_to_developer(...)
else complete
Manager-->>User: final outputs/status
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
👋 Welcome to the Airbyte Connector Builder MCP!Thank you for your contribution! Here are some helpful tips and reminders for your convenience. Testing This Branch via MCPTo test the changes in this specific branch with an MCP client like Claude Desktop, use the following configuration: {
"mcpServers": {
"connector-builder-mcp-dev": {
"command": "uvx",
"args": ["--from", "git+https://github.com/airbytehq/connector-builder-mcp.git@devin/1760288645-replace-openai-agents-with-pydantic-ai", "connector-builder-mcp"]
}
}
}Testing This Branch via CLIYou can test this version of the MCP Server using the following CLI snippet: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/connector-builder-mcp.git@devin/1760288645-replace-openai-agents-with-pydantic-ai#egg=airbyte-connector-builder-mcp' --helpPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
AI Builder EvaluationsAI builder evaluations run automatically under the following conditions:
A set of standardized evaluations also run on a schedule (Mon/Wed/Fri at midnight UTC) and can be manually triggered via workflow dispatch. Helpful ResourcesIf you have any questions, feel free to ask in the PR comments or join our Slack community. |
PyTest Results (Full)0 tests 0 ✅ 0s ⏱️ Results for commit 7d21084. ♻️ This comment has been updated with latest results. |
- Imported duckduckgo_search from pydantic_ai.tools - Added to developer agent's tools list - Replaces the previous WebSearchTool from openai-agents - Addresses web search capability checklist item from PR description Co-Authored-By: AJ Steers <[email protected]>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
pyproject.toml (1)
30-31: Remove duplicated dev dependency.
poethepoetis listed twice. Keep the newer pin only.- "poethepoet>=0.29.0", "poethepoet>=0.37.0",
🧹 Nitpick comments (9)
pyproject.toml (1)
69-69: Align mypy target python with runtime.
requires-pythonis ">=3.11" but mypy is set to "3.10". Set to 3.11 to avoid mismatches in type inference.Suggested:
[tool.mypy] python_version = "3.11"connector_builder_agents/src/run.py (4)
28-31: Use a collision‑free session id.
int(time.time())can collide under concurrency. Prefer UUID.-def generate_session_id() -> str: - """Generate a unique session ID based on current timestamp.""" - return f"unified-mcp-session-{int(time.time())}" +import uuid + +def generate_session_id() -> str: + """Generate a unique session ID.""" + return f"unified-mcp-session-{uuid.uuid4()}"
98-105: Avoid CWD‑relative prompt path; use the module constant.This will fail if run outside repo root. Reuse
ROOT_PROMPT_FILE_STR.- prompt_file = Path("./prompts") / "root-prompt.md" - prompt = prompt_file.read_text(encoding="utf-8") + "\n\n" - prompt += instructions + from .constants import ROOT_PROMPT_FILE_STR + prompt = ROOT_PROMPT_FILE_STR + "\n\n" + instructions
200-234: Add safety limits and trim message history to prevent unbounded growth.
- Loop can run indefinitely if agents never mark completion.
message_historygrows without bounds.- while not is_complete(session_state): + from .constants import MAX_CONNECTOR_BUILD_STEPS + MAX_HISTORY = 200 # local cap; consider promoting to constants + while not is_complete(session_state) and iteration_count < MAX_CONNECTOR_BUILD_STEPS: @@ - session_state.message_history.extend(run_result.new_messages()) + session_state.message_history.extend(run_result.new_messages()) + if len(session_state.message_history) > MAX_HISTORY: + session_state.message_history = session_state.message_history[-MAX_HISTORY:] @@ - return all_run_results + if not is_complete(session_state): + update_progress_log("⏹ Reached max iterations; stopping.", session_state) + return all_run_results
236-241: Avoidsys.exitin run_manager_developer_build.Return cleanly so callers control process termination.
- except KeyboardInterrupt: + except KeyboardInterrupt: update_progress_log("\n🛑 Build terminated (ctrl+c input received).", session_state) - sys.exit(0) + return all_run_results if 'all_run_results' in locals() else []connector_builder_agents/src/agents.py (2)
106-121: Handle errors from delegated developer runs.If the developer toolchain fails (tool error/timeout), log via
log_tool_failureand propagate a concise message to the manager to decide next steps.- result = await developer_agent.run( + try: + result = await developer_agent.run( assignment_description, message_history=ctx.deps.message_history, deps=ctx.deps, ) + except Exception as e: + update_progress_log( + f"🛠️ Delegation failed for '{assignment_title}': {e}", ctx.deps + ) + return f"Delegation failed: {e}"
118-120: Cap shared message history to avoid ballooning.Same concern as in run.py; trim after extending.
- ctx.deps.message_history.extend(result.new_messages()) + ctx.deps.message_history.extend(result.new_messages()) + if len(ctx.deps.message_history) > 200: + ctx.deps.message_history = ctx.deps.message_history[-200:]connector_builder_agents/src/tools.py (2)
22-31: Avoid mutable default formessage_history.Use
Field(default_factory=list)and drop manual kwargs handling.- message_history: list = [] + message_history: list = Field(default_factory=list) @@ - message_history = kwargs.get("message_history", []) @@ super().__init__( workspace_dir=workspace_dir, execution_log_file=execution_log_file, - message_history=message_history, + # Let Pydantic default_factory create a fresh list per instance start_time=start_time, **kwargs, )Also applies to: 40-44
58-73: Duplicate connector‑builder MCP servers; reuse a single instance.Two identical
MCPServerStdio("uv", ["run","airbyte-connector-builder-mcp"])are spawned, contradicting the “reuse instances” docstring and wasting resources.-connector_builder_dev = MCP_CONNECTOR_BUILDER_FOR_DEVELOPER() -connector_builder_manager = MCP_CONNECTOR_BUILDER_FOR_MANAGER() +connector_builder = MCP_CONNECTOR_BUILDER_FOR_DEVELOPER() @@ - all_servers = [ + all_servers = [ # MCP_PLAYWRIGHT_WEB_BROWSER(), - connector_builder_dev, - connector_builder_manager, + connector_builder, filesystem_server, ] @@ - manager_servers = [ - connector_builder_manager, + manager_servers = [ + connector_builder, filesystem_server, ] @@ - developer_servers = [ + developer_servers = [ # MCP_PLAYWRIGHT_WEB_BROWSER(), - connector_builder_dev, + connector_builder, filesystem_server, ]Or drop the separate lambdas and define a single
MCP_CONNECTOR_BUILDERfactory.Also applies to: 106-127
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (6)
connector_builder_agents/src/agents.py(3 hunks)connector_builder_agents/src/constants.py(1 hunks)connector_builder_agents/src/guidance.py(0 hunks)connector_builder_agents/src/run.py(5 hunks)connector_builder_agents/src/tools.py(5 hunks)pyproject.toml(1 hunks)
💤 Files with no reviewable changes (1)
- connector_builder_agents/src/guidance.py
🧰 Additional context used
🧬 Code graph analysis (2)
connector_builder_agents/src/run.py (3)
connector_builder_agents/src/_util.py (1)
get_secrets_dotenv(29-88)connector_builder_agents/src/agents.py (2)
create_developer_agent(23-51)create_manager_agent(54-151)connector_builder_agents/src/tools.py (4)
SessionState(17-50)update_progress_log(144-168)is_complete(48-50)is_complete(132-134)
connector_builder_agents/src/agents.py (2)
connector_builder_agents/src/tools.py (5)
SessionState(17-50)create_log_progress_milestone_from_developer_tool(284-291)create_log_problem_encountered_by_developer_tool(264-271)create_log_tool_failure_tool(220-251)update_progress_log(144-168)connector_builder_agents/src/guidance.py (2)
get_default_developer_prompt(76-93)get_default_manager_prompt(58-73)
🪛 GitHub Actions: Dependency Analysis
pyproject.toml
[error] 1-1: DEP002 'pydantic-ai' defined as a dependency but not used in the codebase.
[error] 1-1: DEP002 'pydantic-ai-slim' defined as a dependency but not used in the codebase.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Pytest (Fast)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
🔇 Additional comments (2)
connector_builder_agents/src/agents.py (1)
4-6: Verify availability ofduckduckgo_searchtool.Ensure
duckduckgo_searchworks without extra install. If it requires an extra, add it to project deps or the newagentsextra.If needed, add:
[project.optional-dependencies] agents = [ "pydantic-ai[duckduckgo]", # or the specific package providing duckduckgo search ]connector_builder_agents/src/constants.py (1)
16-18: Clarify model ID format for pydantic-aiAgent shorthand supports provider-prefixed IDs (e.g.
Agent("openai:gpt-4o")), but direct OpenAI model instantiation (e.g.OpenAIChatModel) expects the raw name ("gpt-4o"). EnsureDEFAULT_DEVELOPER_MODELandDEFAULT_MANAGER_MODELare only used withAgent; otherwise strip theopenai:prefix.
| update_progress_log("\n⚙️ AI Agent is working...", session_state) | ||
| try: | ||
| result = await agent.run( | ||
| input_prompt, | ||
| message_history=session_state.message_history, | ||
| deps=session_state, | ||
| ) | ||
|
|
||
| session_state.message_history.extend(result.new_messages()) | ||
|
|
||
| update_progress_log(f"\n🤖 AI Agent: {result.data}", session_state) | ||
|
|
||
| input_prompt = input("\n👤 You: ") | ||
| if input_prompt.lower() in {"exit", "quit"}: | ||
| update_progress_log("☑️ Ending conversation...", session_state) | ||
| break | ||
|
|
||
| except KeyboardInterrupt: | ||
| update_progress_log( | ||
| "\n🛑 Conversation terminated (ctrl+c input received).", session_state | ||
| ) | ||
| sys.exit(0) | ||
|
|
||
| return None |
There was a problem hiding this comment.
Don’t block the event loop or exit the process in a library function.
input()blocks in an async function.sys.exit(0)will terminate the whole process; prefer returning gracefully.
- input_prompt = input("\n👤 You: ")
+ # Non-blocking input in async context
+ import asyncio
+ loop = asyncio.get_event_loop()
+ input_prompt = await loop.run_in_executor(None, input, "\n👤 You: ")
if input_prompt.lower() in {"exit", "quit"}:
update_progress_log("☑️ Ending conversation...", session_state)
break
- except KeyboardInterrupt:
+ except KeyboardInterrupt:
update_progress_log(
"\n🛑 Conversation terminated (ctrl+c input received).", session_state
)
- sys.exit(0)
+ return None📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| update_progress_log("\n⚙️ AI Agent is working...", session_state) | |
| try: | |
| result = await agent.run( | |
| input_prompt, | |
| message_history=session_state.message_history, | |
| deps=session_state, | |
| ) | |
| session_state.message_history.extend(result.new_messages()) | |
| update_progress_log(f"\n🤖 AI Agent: {result.data}", session_state) | |
| input_prompt = input("\n👤 You: ") | |
| if input_prompt.lower() in {"exit", "quit"}: | |
| update_progress_log("☑️ Ending conversation...", session_state) | |
| break | |
| except KeyboardInterrupt: | |
| update_progress_log( | |
| "\n🛑 Conversation terminated (ctrl+c input received).", session_state | |
| ) | |
| sys.exit(0) | |
| return None | |
| update_progress_log("\n⚙️ AI Agent is working...", session_state) | |
| try: | |
| result = await agent.run( | |
| input_prompt, | |
| message_history=session_state.message_history, | |
| deps=session_state, | |
| ) | |
| session_state.message_history.extend(result.new_messages()) | |
| update_progress_log(f"\n🤖 AI Agent: {result.data}", session_state) | |
| # Non-blocking input in async context | |
| import asyncio | |
| loop = asyncio.get_event_loop() | |
| input_prompt = await loop.run_in_executor(None, input, "\n👤 You: ") | |
| if input_prompt.lower() in {"exit", "quit"}: | |
| update_progress_log("☑️ Ending conversation...", session_state) | |
| break | |
| except KeyboardInterrupt: | |
| update_progress_log( | |
| "\n🛑 Conversation terminated (ctrl+c input received).", session_state | |
| ) | |
| return None | |
| return None |
🤖 Prompt for AI Agents
In connector_builder_agents/src/run.py around lines 133 to 156, the code uses
blocking input() inside an async function and calls sys.exit(0) in the
KeyboardInterrupt handler; replace blocking input() with an async-friendly
approach (either accept an injected async input callback from caller, or wrap
input() with asyncio.to_thread to avoid blocking the event loop) and remove
sys.exit(0) so the function returns gracefully on KeyboardInterrupt (return None
or raise a controlled exception up to caller). Also ensure update_progress_log
is called before returning so the caller can continue/cleanup.
| env={}, | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
Confirm env={} semantics on MCPServerStdio.
If env replaces rather than merges with os.environ, this can strip needed variables (e.g., credentials). Prefer omitting env or explicitly merging.
If replace: construct via:
import os
MCPServerStdio(cmd, args, env=os.environ.copy() | extra_env)Also applies to: 72-73, 81-82, 93-94
🌐 Web query:
In pydantic-ai MCPServerStdio, does passing env={} replace the entire environment or merge with the current environment?
💡 Result:
Passing env={} replaces the subprocess environment with exactly the mapping you provide (so an empty env yields no inherited vars). To inherit the parent env, pass env=os.environ. [1]
Preserve inherited environment in all MCPServerStdio invocations
Passing env={} replaces the subprocess environment and strips inherited variables. At lines 64-65, 72-73, 81-82, and 93-94, either omit the env argument or explicitly merge with the parent environment (e.g. env=os.environ.copy() | extra_env).
🤖 Prompt for AI Agents
In connector_builder_agents/src/tools.py around lines 64-65 (also addressing
similar calls at 72-73, 81-82 and 93-94), the MCPServerStdio invocations pass
env={} which replaces the subprocess environment and strips inherited variables;
instead preserve the parent environment by omitting the env argument or merging
with os.environ (e.g., create a copy of os.environ and update it with any extra
env values) so subprocesses inherit existing variables, then pass that merged
dict as the env parameter.
pyproject.toml
Outdated
| "pydantic-ai>=0.0.14,<1.0", | ||
| "pydantic-ai-slim[openai]>=0.0.14,<1.0", |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
🧩 Analysis chain
Fix deptry failures: move pydantic‑ai deps to an optional extra or include agents package in the wheel.
Deptry flags these as unused because connector_builder_agents/ is excluded from analysis and not in wheel packages. Choose one:
- Preferred: move to an optional extra (e.g.,
agents) and keep[tool.deptry]as‑is. - Or: include
connector_builder_agentsin[tool.hatch.build.targets.wheel].packagesand remove it from deptry’s exclude.
Apply this diff to remove the direct deps now:
- "pydantic-ai>=0.0.14,<1.0",
- "pydantic-ai-slim[openai]>=0.0.14,<1.0",Then add this section (outside the changed lines) to declare the extra:
[project.optional-dependencies]
agents = [
"pydantic-ai>=0.0.14,<1.0",
"pydantic-ai-slim[openai]>=0.0.14,<1.0",
]Alternatively, include the package and un‑exclude it:
# In [tool.hatch.build.targets.wheel]
packages = ["connector_builder_mcp", "connector_builder_agents"]
# In [tool.deptry]
exclude = [
".venv/",
"tests/",
]Run deptry locally after either change to confirm the error is gone.
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "1. Checking for connector_builder_agents directory..."
fd --type d connector_builder_agents .
echo ""
echo "2. Inspecting wheel build targets in pyproject.toml..."
rg -n 'tool.hatch.build.targets.wheel' -A5 -B2 pyproject.toml
echo ""
echo "3. Inspecting deptry exclude configuration in pyproject.toml..."
rg -n 'tool.deptry' -A5 -B2 pyproject.tomlLength of output: 506
Move pydantic-ai deps into an optional agents extra or include connector_builder_agents in the wheel build.
- Preferred: in pyproject.toml, remove
"pydantic-ai>=0.0.14,<1.0",
"pydantic-ai-slim[openai]>=0.0.14,<1.0",
from the base[project]deps and add under[project.optional-dependencies] agents = [ "pydantic-ai>=0.0.14,<1.0", "pydantic-ai-slim[openai]>=0.0.14,<1.0", ]
- Alternatively: in
[tool.hatch.build.targets.wheel].packagesadd"connector_builder_agents",
and remove"connector_builder_agents/"from[tool.deptry].exclude.
Re-run deptry to confirm the warning is resolved.
🤖 Prompt for AI Agents
In pyproject.toml around lines 16-17, the pydantic-ai dependencies are listed in
the base [project] deps; move them into an optional extra named "agents" by
removing the two lines from [project] and adding them under
[project.optional-dependencies] agents = [...] with the same version
constraints, or alternatively ensure the package "connector_builder_agents" is
included in [tool.hatch.build.targets.wheel].packages and remove
"connector_builder_agents/" from [tool.deptry].exclude so the wheel build
includes that module; after making either change, re-run deptry to confirm the
warning is resolved.
- Corrected import path from pydantic_ai.common_tools.duckduckgo - Changed to function call: duckduckgo_search_tool() - Added [duckduckgo] optional group to pydantic-ai-slim dependency - Added emoji as direct dependency (used in tools.py) - Excluded connector_builder_agents/src/evals/ from Deptry scanning - Fixes Deptry false positives for pydantic-ai package usage Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
connector_builder_agents/src/agents.py (1)
120-120: Consider type-safe result data handling.The conversion
str(result.data)assumesresult.datais present and meaningful. Ifresult.dataisNoneor an unexpected type, the conversion succeeds silently but may return unhelpful values like"None".For better type safety, consider explicit handling:
- return str(result.data) + if result.data is None: + return "Task completed (no data returned)" + return str(result.data)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (2)
connector_builder_agents/src/agents.py(3 hunks)pyproject.toml(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
connector_builder_agents/src/agents.py (2)
connector_builder_agents/src/tools.py (5)
SessionState(17-50)create_log_progress_milestone_from_developer_tool(284-291)create_log_problem_encountered_by_developer_tool(264-271)create_log_tool_failure_tool(220-251)update_progress_log(144-168)connector_builder_agents/src/guidance.py (2)
get_default_developer_prompt(76-93)get_default_manager_prompt(58-73)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (Fast)
🔇 Additional comments (4)
connector_builder_agents/src/agents.py (4)
4-5: LGTM: Clean migration to pydantic-ai framework.The imports correctly replace the openai-agents framework with pydantic-ai's Agent and RunContext, and adopt the built-in duckduckgo_search_tool.
23-51: LGTM: Developer agent correctly configured.The developer agent is properly initialized with pydantic-ai's Agent class, includes appropriate tools (progress logging, problem reporting, and web search), and correctly integrates MCP servers via the toolsets pattern.
54-85: LGTM: Manager agent properly initialized.The manager agent configuration follows the same pydantic-ai patterns as the developer agent, with appropriate tools for orchestration and MCP server integration.
122-149: LGTM: Well-structured progress reporting mechanism.The
report_back_to_managertool provides a clean interface for the developer agent to communicate status back to the manager with structured flags (is_full_success,is_partial_success,is_blocked) and detailed updates.
| result = await developer_agent.run( | ||
| assignment_description, | ||
| message_history=ctx.deps.message_history, | ||
| deps=ctx.deps, | ||
| ) | ||
|
|
||
| def create_on_manager_handback(session_state: SessionState): | ||
| """Create an on_manager_handback callback bound to a specific session state.""" | ||
| update_progress_log( | ||
| f"🤝 [DEVELOPER → MANAGER] Developer completed task: {assignment_title}" | ||
| f"\n Result: {result.data}", | ||
| ctx.deps, | ||
| ) | ||
|
|
||
| async def on_manager_handback(ctx, input_data: ManagerHandoffInput) -> None: | ||
| ctx.deps.message_history.extend(result.new_messages()) | ||
|
|
||
| return str(result.data) |
There was a problem hiding this comment.
Add error handling for developer agent invocation.
The developer_agent.run() call at line 106 lacks error handling. If the developer agent encounters an exception, it will propagate uncaught, potentially causing the manager to fail without proper logging or recovery.
Consider wrapping the invocation in a try-except block:
- result = await developer_agent.run(
- assignment_description,
- message_history=ctx.deps.message_history,
- deps=ctx.deps,
- )
-
- update_progress_log(
- f"🤝 [DEVELOPER → MANAGER] Developer completed task: {assignment_title}"
- f"\n Result: {result.data}",
- ctx.deps,
- )
-
- ctx.deps.message_history.extend(result.new_messages())
-
- return str(result.data)
+ try:
+ result = await developer_agent.run(
+ assignment_description,
+ message_history=ctx.deps.message_history,
+ deps=ctx.deps,
+ )
+
+ update_progress_log(
+ f"🤝 [DEVELOPER → MANAGER] Developer completed task: {assignment_title}"
+ f"\n Result: {result.data}",
+ ctx.deps,
+ )
+
+ ctx.deps.message_history.extend(result.new_messages())
+
+ return str(result.data)
+ except Exception as e:
+ error_msg = f"Developer agent failed during task '{assignment_title}': {str(e)}"
+ update_progress_log(f"❌ {error_msg}", ctx.deps)
+ return f"ERROR: {error_msg}"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| result = await developer_agent.run( | |
| assignment_description, | |
| message_history=ctx.deps.message_history, | |
| deps=ctx.deps, | |
| ) | |
| def create_on_manager_handback(session_state: SessionState): | |
| """Create an on_manager_handback callback bound to a specific session state.""" | |
| update_progress_log( | |
| f"🤝 [DEVELOPER → MANAGER] Developer completed task: {assignment_title}" | |
| f"\n Result: {result.data}", | |
| ctx.deps, | |
| ) | |
| async def on_manager_handback(ctx, input_data: ManagerHandoffInput) -> None: | |
| ctx.deps.message_history.extend(result.new_messages()) | |
| return str(result.data) | |
| try: | |
| result = await developer_agent.run( | |
| assignment_description, | |
| message_history=ctx.deps.message_history, | |
| deps=ctx.deps, | |
| ) | |
| update_progress_log( | |
| f"🤝 [DEVELOPER → MANAGER] Developer completed task: {assignment_title}" | |
| f"\n Result: {result.data}", | |
| ctx.deps, | |
| ) | |
| ctx.deps.message_history.extend(result.new_messages()) | |
| return str(result.data) | |
| except Exception as e: | |
| error_msg = f"Developer agent failed during task '{assignment_title}': {str(e)}" | |
| update_progress_log(f"❌ {error_msg}", ctx.deps) | |
| return f"ERROR: {error_msg}" |
| ) | ||
|
|
||
| async def on_manager_handback(ctx, input_data: ManagerHandoffInput) -> None: | ||
| ctx.deps.message_history.extend(result.new_messages()) |
There was a problem hiding this comment.
❓ Verification inconclusive
Monitor unbounded message history growth.
The message history is extended indefinitely without pruning or size limits. In long-running sessions with many delegations, this could lead to excessive memory consumption and slower performance as the context grows.
As noted in the PR objectives checklist, message history growth should be monitored. Consider implementing a mechanism to prune old messages or set a maximum history size.
Verify whether there's a history management mechanism elsewhere in the codebase:
🏁 Script executed:
#!/bin/bash
# Search for message_history pruning, cleanup, or size management
rg -n -C5 'message_history.*(?:prune|clear|trim|limit|max|size)' --type pyLength of output: 0
#!/bin/bash
# Search for all occurrences of message_history and any pruning/size-management logic
rg -n 'message_history' --type py
rg -nP '(?:message_history|history).*\b(prune|trim|limit|max|size)\b' --type py -C5Limit message history growth.
In connector_builder_agents/src/agents.py:118, ctx.deps.message_history.extend(result.new_messages()) is unbounded. Add pruning or enforce a maximum history size to prevent unbounded memory usage.
🤖 Prompt for AI Agents
In connector_builder_agents/src/agents.py around line 118,
ctx.deps.message_history.extend(result.new_messages()) allows unbounded growth
of the message history; after extending, enforce a maximum history size (e.g., a
configurable MAX_HISTORY constant or setting) and prune older entries to keep
only the most recent N messages, or replace the list with a
collections.deque(maxlen=...) to automatically discard oldest entries — update
code to extend/add new messages then trim the list to the max length (or convert
to deque) so memory usage is bounded.
|
/build-connector
|
The evals command was failing with ModuleNotFoundError for pydantic_ai because the connector_builder_agents subdirectory has its own separate package configuration that still referenced openai-agents. - Replaced openai-agents>=0.3.3 with pydantic-ai>=0.0.14,<1.0 - Added pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0 - Removed mcp-agent (was transitive dependency of openai-agents) - Removed openinference-instrumentation-openai-agents (openai-agents specific) - Updated connector_builder_agents/uv.lock via uv sync Fixes: /build-connector evals failure Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
connector_builder_agents/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (1)
connector_builder_agents/pyproject.toml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Pytest (Fast)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
| "pydantic-ai>=0.0.14,<1.0", | ||
| "pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0", |
There was a problem hiding this comment.
Remove the duplicate Pydantic AI distributions.
pydantic-ai and pydantic-ai-slim[...] both ship the same pydantic_ai package; installing both forces pip to pick one wheel and the other will repeatedly reinstall/overwrite during builds. Choose a single distribution (likely the slim extra) to avoid installation failures.
"python-dotenv>=1.1.1",
- "pydantic-ai>=0.0.14,<1.0",
"pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "pydantic-ai>=0.0.14,<1.0", | |
| "pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0", | |
| "python-dotenv>=1.1.1", | |
| "pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0", |
🤖 Prompt for AI Agents
In connector_builder_agents/pyproject.toml around lines 9-10, the file lists
both "pydantic-ai" and "pydantic-ai-slim[openai,duckduckgo]" which are the same
package and conflict during installs; remove the duplicate "pydantic-ai" entry
and keep the slim distribution with the needed extras
(pydantic-ai-slim[openai,duckduckgo]>=0.0.14,<1.0), then update the lockfile
(poetry lock / pip-compile / reinstall deps) so the dependency graph reflects
the change.
The pydantic-ai AgentRunResult object uses 'output' attribute, not 'data'. This was causing the evals CI check to fail with: AttributeError: 'AgentRunResult' object has no attribute 'data' Fixes the delegate_to_developer function in agents.py. Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (2)
connector_builder_agents/src/agents.py (2)
106-110: Add error handling for developer agent invocation.The
developer_agent.run()call lacks error handling. If the developer agent encounters an exception, it will propagate uncaught, potentially causing the manager to fail without proper logging or recovery.The previous review comment on this issue remains valid and unresolved. Consider wrapping the invocation in a try-except block as suggested in the earlier review.
118-118: Limit message history growth.The
ctx.deps.message_history.extend(result.new_messages())call allows unbounded growth. In long-running sessions with many delegations, this could lead to excessive memory consumption and performance degradation.The previous review comment on this issue remains valid and unresolved. Consider implementing a mechanism to prune old messages or set a maximum history size as suggested in the earlier review.
🧹 Nitpick comments (1)
connector_builder_agents/src/agents.py (1)
122-150: Consider enhancing the return value.The tool returns a static string
"Status reported to manager"regardless of the status content. While functional, a more informative return value could include a summary of the reported status.For example:
- return "Status reported to manager" + return f"Reported to manager: {short_status} (full_success={is_full_success}, blocked={is_blocked})"This would provide better context in the message history and agent logs.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
connector_builder_agents/src/agents.py(3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
connector_builder_agents/src/agents.py (2)
connector_builder_agents/src/tools.py (5)
SessionState(17-50)create_log_progress_milestone_from_developer_tool(284-291)create_log_problem_encountered_by_developer_tool(264-271)create_log_tool_failure_tool(220-251)update_progress_log(144-168)connector_builder_agents/src/guidance.py (2)
get_default_developer_prompt(76-93)get_default_manager_prompt(58-73)
🔇 Additional comments (4)
connector_builder_agents/src/agents.py (4)
4-5: LGTM!The import changes correctly reflect the migration from openai-agents to pydantic-ai framework.
23-51: LGTM!The developer agent setup correctly uses pydantic-ai patterns:
- Proper return type annotation with
Agent- Correct constructor parameters including
deps_type- MCP servers attached via
toolsets.append()- Web search tool updated to pydantic-ai's
duckduckgo_search_tool()
54-85: LGTM!The manager agent setup follows the same correct pydantic-ai patterns as the developer agent, with proper type annotations and MCP server attachment.
114-114: Confirm.outputis correct
Pydantic-AI’s RunResult object exposes its output via the.outputattribute, not.data.
Found three additional locations in run.py that were using result.data instead of result.output: - Line 143 in run_interactive_build function - Line 224 in run_manager_developer_build function - Line 231 in run_manager_developer_build function This completes the migration from openai-agents to pydantic-ai. Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
connector_builder_agents/src/run.py (1)
145-145: Previous critical issue not addressed: blocking I/O and process termination.The blocking
input()call at Line 145 andsys.exit(0)at Line 154 were flagged in the previous review but remain unresolved.
- Line 145:
input()blocks the async event loop, preventing concurrent operations.- Line 154:
sys.exit(0)terminates the entire process ungracefully, preventing cleanup and making the function non-reusable.Apply this diff to address both issues:
+import asyncio + async def run_interactive_build( prompt: str, model: str, session_id: str, ) -> None: """Run the agent using interactive mode with conversation loop.""" workspace_dir = get_workspace_dir(session_id) session_state = create_session_state(workspace_dir) all_mcp_servers, _, _ = create_session_mcp_servers(session_state) agent = Agent( model, name="MCP Connector Builder", deps_type=SessionState, system_prompt=( "You are a helpful assistant with access to MCP tools for building Airbyte connectors." ), ) for mcp_server in all_mcp_servers: agent.toolsets.append(mcp_server) input_prompt: str = prompt while True: update_progress_log("\n⚙️ AI Agent is working...", session_state) try: result = await agent.run( input_prompt, message_history=session_state.message_history, deps=session_state, ) session_state.message_history.extend(result.new_messages()) update_progress_log(f"\n🤖 AI Agent: {result.output}", session_state) - input_prompt = input("\n👤 You: ") + loop = asyncio.get_event_loop() + input_prompt = await loop.run_in_executor(None, input, "\n👤 You: ") if input_prompt.lower() in {"exit", "quit"}: update_progress_log("☑️ Ending conversation...", session_state) break except KeyboardInterrupt: update_progress_log( "\n🛑 Conversation terminated (ctrl+c input received).", session_state ) - sys.exit(0) + return None return NoneAlso applies to: 150-154
🧹 Nitpick comments (1)
connector_builder_agents/src/run.py (1)
50-50: Consider preserving type information in return signatures.The return types have been loosened from
list[RunResult]tolist(Line 165) and fromlist[RunResult] | Nonetolist | None(Line 50). This removes valuable type information and reduces type safety.If
RunResultis no longer available due to the framework migration, consider defining a replacement type or using a more specific type hint.If pydantic-ai provides a result type, use it:
+from pydantic_ai import RunResult # or equivalent + -) -> list | None: +) -> list[RunResult] | None:-) -> list: +) -> list[RunResult]:Also applies to: 165-165
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
connector_builder_agents/src/run.py(5 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
connector_builder_agents/src/run.py (3)
connector_builder_agents/src/_util.py (1)
get_secrets_dotenv(29-88)connector_builder_agents/src/agents.py (2)
create_developer_agent(23-51)create_manager_agent(54-151)connector_builder_agents/src/tools.py (4)
SessionState(17-50)update_progress_log(144-168)is_complete(48-50)is_complete(132-134)
| except KeyboardInterrupt: | ||
| update_progress_log("\n🛑 Build terminated (ctrl+c input received).", session_state) | ||
| sys.exit(0) |
There was a problem hiding this comment.
Don't terminate the process in a library function.
Line 238 uses sys.exit(0) which terminates the entire process. This prevents cleanup and makes the function non-reusable as a library component.
Apply this diff to return gracefully:
except KeyboardInterrupt:
update_progress_log("\n🛑 Build terminated (ctrl+c input received).", session_state)
- sys.exit(0)
+ return []📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| except KeyboardInterrupt: | |
| update_progress_log("\n🛑 Build terminated (ctrl+c input received).", session_state) | |
| sys.exit(0) | |
| except KeyboardInterrupt: | |
| update_progress_log("\n🛑 Build terminated (ctrl+c input received).", session_state) | |
| return [] |
🤖 Prompt for AI Agents
In connector_builder_agents/src/run.py around lines 236-238, the except block
currently calls sys.exit(0) which force-terminates the whole process; instead
remove the sys.exit call and return gracefully to the caller (or return an
appropriate sentinel value) after logging the termination message so the
function can be used as a library component and allows callers to perform
cleanup.
…_output The pydantic-ai AgentRunResult object uses 'output' attribute, not 'final_output'. This was causing the evals CI check to fail with: AttributeError: 'AgentRunResult' object has no attribute 'final_output' Completes the migration from openai-agents to pydantic-ai by fixing the last remaining incompatible attribute access in the evals code. Co-Authored-By: AJ Steers <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
connector_builder_agents/src/evals/task.py (1)
54-54: Rename dictionary key to match attribute. The code now usesfinal_result.outputbut the result dict still uses"final_output"—consider renaming it to"output"for consistency.
1 participant
Replace openai-agents with pydantic-ai implementation
Summary
Migrated the connector builder MCP from
openai-agentstopydantic-ai, updating the agent architecture, tool delegation mechanisms, and dependencies across both the main package and theconnector_builder_agentssubpackage.Key changes:
openai-agents.Agentwithpydantic_ai.Agenthandoff()classes to tool-based delegation usingRunContextWebSearchTool()toduckduckgo_search_tool()from pydantic-aimessage_historytoSessionStatefor pydantic-ai's message managementmcp_serversparameter totoolsets.append()for attaching MCP serverspyproject.tomlfiles (root andconnector_builder_agents/) to use pydantic-ai packagesReview & Testing Checklist for Human
/build-connectorcommand to confirm evals work with pydantic-ai (initial failure was fixed by adding pydantic-ai to connector_builder_agents/pyproject.toml)Notes
evals/subdirectory to prevent false positivesLink to Devin run: https://app.devin.ai/sessions/0011f26c17d1458b81190e5bcfe5e0f7
Requested by: @aaronsteers
Summary by CodeRabbit
New Features
Improvements
Refactor
Chores