feat: added HITL feature#769
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis PR adds human-in-the-loop (HITL) voice confirmation to the Breeze Buddy voice agent. Templates can now specify HITL config to require voice-based user approval before executing certain functions. The system renders a confirmation prompt, waits for TTS completion, listens to transcription, and gates execution on keyword matching against configured accepted/rejected responses. ChangesVoice HITL confirmation for Breeze Buddy
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/ai/voice/agents/breeze_buddy/features/hitl/manager.py`:
- Line 176: The current HITL logging prints raw transcription via
logger.info(f"HITL: Heard '{text}'), which may expose PII; update the HITL flow
(the logger.info call in manager.py that references variable text) to stop
logging full user speech—instead log only the classification/result (e.g.,
"approved", "rejected", "timeout") or a redacted indicator (e.g.,
"[REDACTED_TRANSCRIPTION]") and include any relevant non-sensitive metadata
(e.g., confidence or decision reason) so that logger.info no longer contains the
raw text variable.
- Around line 186-199: Event handlers added with
task.add_event_handler("on_frame_reached_upstream", on_frame_received) and
task.add_event_handler("on_frame_reached_downstream", on_bot_stopped) are never
removed and will accumulate across confirmation requests; in the finally block
of the function, detect the Task type (PipelineTask) and call
task.remove_event_handler with the same event names and handler references
(on_frame_received and on_bot_stopped) to unregister them before returning,
keeping the existing restoration logic for user_aggregator/original_config
unchanged.
In `@app/ai/voice/agents/breeze_buddy/template/transition.py`:
- Line 88: Remove the unnecessary f-string prefix on the dictionary value for
the "error" key in transition.py (the entry currently set as f"Function
execution rejected"); replace it with a plain string literal "Function execution
rejected" to fix Ruff F541 while keeping the surrounding structure and semantics
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 408713f9-e9a2-4462-8cc2-0c80448010e5
📒 Files selected for processing (8)
app/ai/voice/agents/breeze_buddy/features/hitl/__init__.pyapp/ai/voice/agents/breeze_buddy/features/hitl/manager.pyapp/ai/voice/agents/breeze_buddy/template/builder.pyapp/ai/voice/agents/breeze_buddy/template/context.pyapp/ai/voice/agents/breeze_buddy/template/global_function.pyapp/ai/voice/agents/breeze_buddy/template/transition.pyapp/ai/voice/agents/breeze_buddy/template/types.pyapp/core/config/static.py
|
|
||
| if isinstance(frame, TranscriptionFrame): | ||
| text = frame.text.strip() | ||
| logger.info(f"HITL: Heard '{text}'") |
There was a problem hiding this comment.
Avoid logging raw transcription content in HITL path.
Line 176 logs full user speech (HITL: Heard ...). Please log only classification/result (approved/rejected/timeout) or redact text to avoid sensitive utterance exposure.
Based on learnings: “ensure that no PII/secrets are logged, transmitted, or persisted, and that payload handling enforces this constraint at every access point.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/ai/voice/agents/breeze_buddy/features/hitl/manager.py` at line 176, The
current HITL logging prints raw transcription via logger.info(f"HITL: Heard
'{text}'), which may expose PII; update the HITL flow (the logger.info call in
manager.py that references variable text) to stop logging full user
speech—instead log only the classification/result (e.g., "approved", "rejected",
"timeout") or a redacted indicator (e.g., "[REDACTED_TRANSCRIPTION]") and
include any relevant non-sensitive metadata (e.g., confidence or decision
reason) so that logger.info no longer contains the raw text variable.
| task.add_event_handler("on_frame_reached_upstream", on_frame_received) | ||
|
|
||
| # ------------------------ | ||
| # TTS Phase | ||
| # ------------------------ | ||
|
|
||
| tts_complete = asyncio.Event() | ||
|
|
||
| async def on_bot_stopped(task_obj, frame): | ||
| if isinstance(frame, BotStoppedSpeakingFrame): | ||
| tts_complete.set() | ||
|
|
||
| task.add_event_handler("on_frame_reached_downstream", on_bot_stopped) | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify whether add/remove event handler usage is balanced in HITL flow.
rg -n -C2 'add_event_handler|remove_event_handler' app/ai/voice/agents/breeze_buddy/features/hitl/manager.py
rg -n -C2 'add_event_handler|remove_event_handler' app/ai/voice/agents/breeze_buddyRepository: juspay/clairvoyance
Length of output: 2569
🏁 Script executed:
# Get the full context around lines 186-199 and 222-234
head -n 250 app/ai/voice/agents/breeze_buddy/features/hitl/manager.py | tail -n 100Repository: juspay/clairvoyance
Length of output: 3508
🏁 Script executed:
# Get the function signature and verify the full context
head -n 240 app/ai/voice/agents/breeze_buddy/features/hitl/manager.py | tail -n 150Repository: juspay/clairvoyance
Length of output: 5138
🏁 Script executed:
# Verify the exact line numbers for the finally block and check if there's any handler cleanup elsewhere
sed -n '220,245p' app/ai/voice/agents/breeze_buddy/features/hitl/manager.pyRepository: juspay/clairvoyance
Length of output: 907
🏁 Script executed:
# Check if Pipecat's task object has a remove_event_handler method
rg -n 'remove_event_handler' app/ai/voice/agents/breeze_buddy/Repository: juspay/clairvoyance
Length of output: 45
🏁 Script executed:
# Check Pipecat API and task object documentation/imports
rg -n 'from pipecat' app/ai/voice/agents/breeze_buddy/features/hitl/manager.py
head -n 30 app/ai/voice/agents/breeze_buddy/features/hitl/manager.pyRepository: juspay/clairvoyance
Length of output: 961
🏁 Script executed:
# Search for any task lifecycle/cleanup patterns or remove_event_handler in the entire codebase
rg -n 'remove_event_handler|clear.*handler|unregister.*handler' app/ai/voice/Repository: juspay/clairvoyance
Length of output: 45
🏁 Script executed:
# Verify that _voice_confirmation is called multiple times per call (per confirmation request)
rg -n '_voice_confirmation' app/ai/voice/agents/breeze_buddy/Repository: juspay/clairvoyance
Length of output: 290
🏁 Script executed:
# Check the method containing line 101 and understand the call pattern
sed -n '80,110p' app/ai/voice/agents/breeze_buddy/features/hitl/manager.pyRepository: juspay/clairvoyance
Length of output: 950
Remove registered event handlers in the finally block to prevent accumulation across multiple confirmation requests.
Event handlers registered at lines 186 and 198 are appended per confirmation request and never deregistered. Over long calls with multiple confirmation requests, this causes handler accumulation. The finally block (lines 222–234) should remove both handlers before returning.
Fix location (lines 222–234)
finally:
# Remove event handlers to prevent accumulation across requests
if isinstance(task, PipelineTask):
task.remove_event_handler("on_frame_reached_upstream", on_frame_received)
task.remove_event_handler("on_frame_reached_downstream", on_bot_stopped)
if user_aggregator and original_config:
await _apply_interruption_config(
user_aggregator,
original_config,
has_vad=context.bot.vad_analyzer is not None,
call_sid=context.call_sid or "unknown",
label="hitl_restore",
bot=context.bot,
)
logger.info("HITL: Interruptions restored")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/ai/voice/agents/breeze_buddy/features/hitl/manager.py` around lines 186 -
199, Event handlers added with
task.add_event_handler("on_frame_reached_upstream", on_frame_received) and
task.add_event_handler("on_frame_reached_downstream", on_bot_stopped) are never
removed and will accumulate across confirmation requests; in the finally block
of the function, detect the Task type (PipelineTask) and call
task.remove_event_handler with the same event names and handler references
(on_frame_received and on_bot_stopped) to unregister them before returning,
keeping the existing restoration logic for user_aggregator/original_config
unchanged.
| ) | ||
| return { | ||
| "status": "rejected", | ||
| "error": f"Function execution rejected", |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the exact offending pattern in this file.
rg -n 'f"Function execution rejected"' app/ai/voice/agents/breeze_buddy/template/transition.pyRepository: juspay/clairvoyance
Length of output: 123
Remove unnecessary f-string prefix on Line 88.
The f-string f"Function execution rejected" contains no placeholders and should be a regular string literal. This triggers Ruff F541.
Suggested fix
- "error": f"Function execution rejected",
+ "error": "Function execution rejected",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "error": f"Function execution rejected", | |
| "error": "Function execution rejected", |
🧰 Tools
🪛 Ruff (0.15.12)
[error] 88-88: f-string without any placeholders
Remove extraneous f prefix
(F541)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@app/ai/voice/agents/breeze_buddy/template/transition.py` at line 88, Remove
the unnecessary f-string prefix on the dictionary value for the "error" key in
transition.py (the entry currently set as f"Function execution rejected");
replace it with a plain string literal "Function execution rejected" to fix Ruff
F541 while keeping the surrounding structure and semantics unchanged.
There was a problem hiding this comment.
Pull request overview
This PR adds Human-in-the-Loop confirmation support for Breeze Buddy function execution, wiring hitl template config through flow schema building, context injection, and transition handling.
Changes:
- Adds Breeze Buddy HITL config models and timeout configuration.
- Adds a voice confirmation manager that asks users for verbal approval before executing configured flow functions.
- Passes HITL config through builder/context/transition layers; global function changes are currently comment-only.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
app/core/config/static.py |
Adds Breeze Buddy HITL timeout environment setting. |
app/ai/voice/agents/breeze_buddy/template/types.py |
Adds HITLConfig and hitl support on flow functions. |
app/ai/voice/agents/breeze_buddy/template/transition.py |
Invokes HITL confirmation before hooks/transitions. |
app/ai/voice/agents/breeze_buddy/template/global_function.py |
Adds HITL-related comments/placeholders for global functions. |
app/ai/voice/agents/breeze_buddy/template/context.py |
Propagates HITL config through the context wrapper. |
app/ai/voice/agents/breeze_buddy/template/builder.py |
Serializes and passes HITL config into transition handlers. |
app/ai/voice/agents/breeze_buddy/features/hitl/manager.py |
Implements Breeze Buddy voice confirmation flow. |
app/ai/voice/agents/breeze_buddy/features/hitl/__init__.py |
Exports the HITL manager API. |
Comments suppressed due to low confidence (1)
app/ai/voice/agents/breeze_buddy/template/global_function.py:394
- The docstring now says built-in global functions check HITL, but the actual HITL block below is commented out and
BaseGlobalFunctiondoes not define ahitlfield. A configuredhitlkey on a built-in global function will therefore be ignored despite this wrapper claiming it is enforced.
1. Checks for HITL confirmation if enabled
2. Runs filler phrase TTS + enables background music (if configured)
3. Passes function_config to the dispatcher via kwargs
4. Disables background music after handler returns
5. Fires post-actions (fire-and-forget)
"""
# Step 1: Check for HITL confirmation
| logger.info("HITL: Interruptions disabled") | ||
|
|
||
| response_received = asyncio.Event() | ||
| result = {"approved": True, "reason": "no_response"} |
| # Fast substring match | ||
| for keyword in keywords: | ||
| if keyword.lower() in text: | ||
| return True |
| ) | ||
|
|
||
| timeout = float( | ||
| min(config.timeout_seconds or BREEZE_BUDDY_HITL_DEFAULT_TIMEOUT, 6) |
| original_config = None | ||
|
|
||
| if user_aggregator: | ||
| original_config = getattr(user_aggregator, "interruption_config", None) | ||
|
|
| tts_complete = asyncio.Event() | ||
|
|
||
| async def on_bot_stopped(task_obj, frame): | ||
| if isinstance(frame, BotStoppedSpeakingFrame): | ||
| tts_complete.set() | ||
|
|
||
| task.add_event_handler("on_frame_reached_downstream", on_bot_stopped) |
| 1. Checks for HITL confirmation if enabled | ||
| 2. Runs filler phrase TTS + enables background music (if configured) | ||
| 3. Calls the wrapped handler (with_context wrapper) passing function_config | ||
| 4. Disables background music after handler returns | ||
| 5. Fires post-actions (fire-and-forget) | ||
| """ | ||
| # Step 1: Check for HITL confirmation |
| task.add_event_handler("on_frame_reached_upstream", on_frame_received) | ||
|
|
||
| # ------------------------ | ||
| # TTS Phase | ||
| # ------------------------ | ||
|
|
||
| tts_complete = asyncio.Event() | ||
|
|
||
| async def on_bot_stopped(task_obj, frame): | ||
| if isinstance(frame, BotStoppedSpeakingFrame): | ||
| tts_complete.set() | ||
|
|
||
| task.add_event_handler("on_frame_reached_downstream", on_bot_stopped) |
kanakagrawal-crypto
force-pushed
the
HITL
branch
9 times, most recently
from
38c14d7 to
480e1f2
Compare
Summary by CodeRabbit
Release Notes