langgraph/how-tos/streaming-events-from-within-tools/

[giscus[bot]]

langgraph/how-tos/streaming-events-from-within-tools/

Build language agents as graphs

https://langchain-ai.github.io/langgraph/how-tos/streaming-events-from-within-tools/

[duob-ai]

Followed your tutorial and token streaming from within tools works fine in a Google Colab environment.

But somehow it is extremely slow when doing the exact same thing with a Remote Runnable.
Using the tool-chains directly via the Remote Runnable works super fast.
As soon as you have the ReAct agent in the pipeline the tokens streamed from the tool-chain is super slow.
Tracking with LangSmith shows that the high latency seems to come from the llm, but it makes no sense that gpt-4o seems to need 50 seconds to process 3k tokens:

I don't know why and have no idea how to debug this. In my understanding, using a Remote Runnable should deliver the exact same experience as using the agent directly