-
-
Notifications
You must be signed in to change notification settings - Fork 172
[Agent] Introducing fundation for human-in-the-loop mechanism for tool calling #1548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
99283f0 to
08c6600
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request introduces a human-in-the-loop confirmation system for tool execution in the Agent component. The system allows developers to require user approval before potentially dangerous tool operations are performed, using a policy-based approach with event-driven architecture.
Changes:
- Adds
ToolCallRequestedevent dispatched before tool execution to enable interception - Implements confirmation system with policies (DefaultPolicy, AlwaysAllowPolicy), handlers, and a subscriber
- Provides comprehensive documentation and a working CLI example
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/agent/src/Toolbox/Toolbox.php | Integrates ToolCallRequested event dispatch into tool execution flow |
| src/agent/src/Toolbox/Event/ToolCallRequested.php | New stoppable event for tool execution confirmation with denial and custom result support |
| src/agent/src/Toolbox/Confirmation/PolicyInterface.php | Interface for decision policies on whether tools should be allowed, denied, or require confirmation |
| src/agent/src/Toolbox/Confirmation/PolicyDecision.php | Enum defining Allow, Deny, and AskUser policy decisions |
| src/agent/src/Toolbox/Confirmation/DefaultPolicy.php | Default policy implementation that auto-allows read operations and remembers user decisions |
| src/agent/src/Toolbox/Confirmation/AlwaysAllowPolicy.php | Permissive policy that bypasses all confirmation checks |
| src/agent/src/Toolbox/Confirmation/ConfirmationHandlerInterface.php | Interface for implementing user confirmation mechanisms |
| src/agent/src/Toolbox/Confirmation/ConfirmationResult.php | Result object carrying confirmation decision and remember preference |
| src/agent/src/Toolbox/Confirmation/ConfirmationSubscriber.php | Event subscriber that orchestrates the confirmation workflow |
| src/agent/tests/Toolbox/Event/ToolCallRequestedTest.php | Comprehensive unit tests for the ToolCallRequested event |
| src/agent/tests/Toolbox/Confirmation/*.php | Unit tests for all confirmation components with excellent coverage |
| src/agent/tests/Toolbox/Confirmation/HumanInTheLoopIntegrationTest.php | Integration tests verifying the complete confirmation workflow |
| src/agent/tests/Toolbox/ToolboxEventDispatcherTest.php | Updated to include ToolCallRequested in event dispatch assertions |
| src/agent/tests/Fixtures/Tool/*.php | New fixture tools for testing read/write/delete operations |
| docs/components/agent.rst | Detailed documentation of the confirmation system with usage examples |
| examples/toolbox/confirmation.php | Working CLI example demonstrating the human-in-the-loop pattern |
| src/agent/CHANGELOG.md | Documents the new features for version 0.4 |
src/agent/tests/Toolbox/Confirmation/HumanInTheLoopIntegrationTest.php
Outdated
Show resolved
Hide resolved
ac5b01f to
9461c1c
Compare
9461c1c to
a73aa24
Compare
Screencast.from.2026-02-05.21-50-19.webm