How can I display an "AI is thinking…" message in a slow ReAct workflow without interrupting the tool call?

[luciotudisco]

Hi everyone!

I'm working with a simple ReAct graph in LangGraph that uses a single tool, but that tool occasionally runs very slowly. Here’s a simple setup:

tools = [slow_tool]
llm = ChatOpenAI(model="gpt-4o-mini")
llm_with_tools = llm.bind_tools(tools)

def call_model(state: State) -> State:
    return {"messages": llm_with_tools.invoke(state["messages"])}

graph_builder = StateGraph(State)
graph_builder.add_node("llm", call_model)
graph_builder.add_node("tools", ToolNode(tools=tools))
graph_builder.add_conditional_edges("llm", tools_condition, "tools")
graph_builder.add_edge("tools", "llm")
graph_builder.set_entry_point("llm")

I’d like to add an AI message to the state that says something like “The tool is taking longer than usual, please wait…”, just to provide user feedback when the tool is slow. However, I don’t want to actually interrupt or cancel the ongoing tool call.

Does anyone know how to achieve this in LangGraph? Any tips or pointers on adding that extra AI message during a potentially long-running tool call would be greatly appreciated!

Thanks in advance for your help.

[hinthornw]

Streaming! https://langchain-ai.github.io/langgraph/concepts/streaming/?h=streaming

[luciotudisco]

Thanks William.

I am running the above agent in the LangGraph platform and already streaming the content it produces using the LangGraph SDK API.

thread = await langgraph_client.threads.create()
..
async for chunk in client.runs.stream(
        thread_id,
        assistant_id,
        input=input_data,
        stream_mode="messages-tuple",
        config=config,
    ):

I am still confused on how to achieve the above.

Is the solution to intercept the streamed tool call request and run a stopwatch in the client until the tool result event happens?
Is there a specific event I need to react to?
Something else?