Skip to content

Carry over "deactivated" versions of functions from one node to the next (made while exploring Gemini Live <-> Pipecat Flows interop)#232

Open
kompfner wants to merge 13 commits intomainfrom
pk/gemini-live-support
Open

Carry over "deactivated" versions of functions from one node to the next (made while exploring Gemini Live <-> Pipecat Flows interop)#232
kompfner wants to merge 13 commits intomainfrom
pk/gemini-live-support

Conversation

@kompfner
Copy link
Contributor

@kompfner kompfner commented Feb 2, 2026

See CHANGELOG for a more detailed description.

Fixes: #230

Note that we're actually not (at least today) officially supporting Gemini Live in Pipecat Flows, but this feature is generally useful regardless.

Pipecat core companion PR, made during the exploration: pipecat-ai/pipecat#3620

Why not already officially support Gemini Live in Pipecat Flows?

  • The need to reconnect on context and tools updates makes Gemini Live not the most natural fit for Flows, performing notably slower than cascaded text-to-text pipelines during node transitions
  • Because of this, the remaining investment to fully support Gemini Live with Flows (rethinking Flows node action timings for realtime models, in addition to ongoing long-term maintenance costs) doesn't—at least today—clear the necessary priority bar

…e that Gemini Live doesn't work yet with PIpecat Flows.
… LLMs are still aware of functions called or simply referenced in conversation history.

With Gemini Live in particular, if historical context text messages even mention missing functions, the API may return errors when connecting and seeding conversation history.
…to the APPEND strategy when we're using an LLM that don't support summarization (like Gemini Live).

I opted to do the simple thing of just checking the LLM name rather than build a more general system where each LLM reports whether it's capable of summarization.
@kompfner kompfner force-pushed the pk/gemini-live-support branch from a812ff1 to a24c344 Compare February 4, 2026 17:33
- Conditionally add STT and TTS to the pipeline, based on whether a traditional LLM or a speech-to-speech service like Gemini Live is being used
- Tweak system prompt wording to:
  - prevent unnecessary responses before node transitions
  - nudge the model to convey the results of a previous function call to the user, even when it's not the most recent message in context
@kompfner kompfner force-pushed the pk/gemini-live-support branch from a24c344 to 499accc Compare February 4, 2026 18:45
…des wouldn't be properly reactivated when transitioning to a node where those functions are active again.
@kompfner kompfner force-pushed the pk/gemini-live-support branch from ce7f142 to 2008ec4 Compare February 5, 2026 16:49
@kompfner kompfner marked this pull request as ready for review February 5, 2026 18:32
- `LLMMessagesTransformFrame`
- Changes to `GeminiLiveLLMService` that allow its context (messages, tools) to be programmatically updated at runtime
]
dependencies = [
"pipecat-ai>=0.0.85",
"pipecat-ai>=0.0.102",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: update this if pipecat-ai/pipecat#3620 doesn't make it in that version.

TODO: uv sync to update uv.lock once the required version is published.

@kompfner kompfner changed the title Changes to support Gemini Live Changes made while exploring Gemini Live <-> Pipecat Flows interop Feb 6, 2026
@kompfner kompfner changed the title Changes made while exploring Gemini Live <-> Pipecat Flows interop Carry over "deactivated" versions of functions from one node to the next (made while exploring Gemini Live <-> Pipecat Flows interop) Feb 6, 2026
@kompfner kompfner force-pushed the pk/gemini-live-support branch from 913a4db to a237c1b Compare February 6, 2026 15:16

Args:
time (str): Requested reservation time in "HH:MM AM/PM" format. Must be between 5 PM and 10 PM.
time (str): Requested reservation time in "HH:MM AM/PM" format, with NO leading 0s (e.g. "6:00 PM"). Must be between 5 PM and 10 PM.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug I fixed during testing

@kompfner kompfner force-pushed the pk/gemini-live-support branch from fe30969 to 43756d8 Compare February 6, 2026 15:32
# Prepare hold music args
flow_manager.state["hold_music_args"] = {
"script_path": Path(__file__).parent.parent / "assets" / "hold_music" / "hold_music.py",
"script_path": Path(__file__).parent.parent
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug I fixed during testing

@kompfner kompfner force-pushed the pk/gemini-live-support branch from 43756d8 to fbb44a3 Compare February 6, 2026 15:49
{
"role": "system",
"content": "Confirm the reservation details and ask if they need anything else.",
"content": "Confirm the reservation details and ask if they need anything else. If they don't, go ahead and end the conversation by calling the appropriate function.",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug I fixed during testing

{
"role": "system",
"content": "Confirm the reservation details and ask if they need anything else.",
"content": "Confirm the reservation details and ask if they need anything else. If they don't, go ahead and end the conversation by calling the appropriate function.",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug I fixed during testing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flows doesn't unregister_function() functions that are not in the new node

4 participants