LLMRunFrame from the top causes inconsistent behavior depending on timing #209
Description
Different problem but related to pipecat-ai/pipecat#2787 ...
The cases that are timing-dependent are actually pipecat-flows related ...
For edge functions, instead of the LLM being triggered from the assistant context aggregator going upstream (which gets queued and delayed until speech finishes), an LLMRunFrame comes DOWNSTREAM from the top, and it can arrive before, during, or after the speech.
- When LLMRunFrame wins the race, the LLM doesn't know about the text that's about to be said, so it often REPEATS itself.
- When the speech wins the race, the tool call is buried behind the speech in the context, so it often responds with NOTHING (different problem - same as general case of normal function calls in aforementioned issue).
I think sending context upstream the same way node functions do it will fix the inconsistency issue and only have one problem to fix.
I'm throwing together a suite of integration tests with mock LLM but real TTS with different timing cases to illustrate all this.