Proposal: Separate the Step Between Tool Result and Assistant Message

[nallwhy]

I’d like to propose separating the step between the tool result and the assistant message in this library.

Currently, even when running in the default mode, after execute_tool_calls/1 is called, the next step do_run/1 is executed immediately, making it impossible for me to intervene or customize anything in between.

My use case is that when the tool_result returns a result for routing, I want to directly use this data (specifically the processed_content) to render a custom UI for the user at that point. This approach would allow me to provide a much faster and richer user experience by returning a tailored UI immediately after receiving the tool_result.

In my view, the entire flow—message -> tool_call -> execute (tool result) -> message—should be treated as a sequence of equal steps. If we make these steps equally accessible, users of this library will gain much more control over each stage, enabling a variety of advanced use cases.

Looking forward, I also hope to add support for the while_needs_response mode, where a function could be provided to determine at each step whether to proceed or pause, giving library users even more flexibility.

I believe that to achieve this, the library needs to treat each step in the chain as an equal unit, as described above. If the proposal is accepted, I am willing to work on this myself and contribute the changes to the library.

Thank you for considering this enhancement!

[nallwhy]

I think 459a392 might address the issue I was facing. I’ll test it out and report back.

[nallwhy]

After checking the changes, I realized that the new behavior always retries until the specific tool is used. It doesn’t work for my use case, as I also need to support generic tool usage.

[brainlid]

Hi @nallwhy!

I have a couple thoughts.

If you use the run without a mode, it stops at each received message. At this point you could make any adjustments or determinations then optionally continue the chain.

As for your rich UI idea, I do this by including a process to notify in the context that a function/tool receives. It could be a LiveView, PubSub, whatever. But it allows me to async reflect what's going on.

[nallwhy]

Thanks for the reply!

As you suggested, I’m currently using run without a mode and handling the control flow myself. It did take a bit of work to implement the logic for when to continue and when to intercept, but it’s working well for my use case.