Show tool calls more clearly in the chat log

[mfisher87]

Problem

I'm consistently seeing "✔️ Other..." for tool use in the chat log, with some exceptions. When I open the .chat file I can that these correspond to tool calls. Often (always?) the raw_output of the tool call is written as stringified JSON.

Proposed Solution

That tool call metadata that's hidden in the .chat file's JSON would be nice to display in a friendly way. Especially for tool calls that correspond to Jupyter commands, I'd love to see rich metadata: What command was called always visible as the "title" of the tool call, and the rest in a collapsible section: what args were passed in, what was the input/output? For example, jupyter-geoagents (cc @cboettig) can submit SQL queries against a geospatial dataset and inspecting those SQL queries is a critical element of a reasonable user experience for the LLM user.

Even if we don't have a way to nicely display info about the tool call, I would rather be able to click a little 🔺 to expand it in the chat log and see the raw data.

Additional context

❤️ you

[brichet]

Related PR jupyter-ai-contrib/jupyter-ai-acp-client#95

We recently moved the jupyter-chat-components repo to jupyter-ai-contrib org: https://github.com/jupyter-ai-contrib/jupyter-chat-components

You can see some examples of components (including tool calls) at https://jupyter-ai-contrib.github.io/jupyter-chat-components/

[mfisher87]

Thanks Nicolas! Playing around with the notebook. What I'm seeing there looks very different to (and much better than 😍 ) what I'm seeing when I use jupyter-ai. Is this work in progress that will be integrated in the future? Or is this just a difference between singular and grouped tool calls? They look quite different and grouped tool calls look a lot like what I was seeing.

I'm sure there's a good reason, no judgement, just wondering about why there's stringified JSON in much of the tool call metadata?

[brichet]

Is this work in progress that will be integrated in the future? Or is this just a difference between singular and grouped tool calls? They look quite different and grouped tool calls look a lot like what I was seeing.

Sorry, I should have give some context.

The idea behind this repo was to avoid duplicating components that may be useful for several projects.

Indeed, there are 2 tool call components, the singular one initially comes from jupyterlite-ai, while the grouped one initially comes from jupyter-ai.
The grouped tool call component is now used in jupyterlite-ai, with some modifications to display a summary in the title (including the command that has been called for example) and the input value in the details. The simple tool call is suppose to be removed soon if not used anymore.

I'm sure there's a good reason, no judgement, just wondering about why there's stringified JSON in much of the tool call metadata?

The only reason was probably to handle both string and JSON data, but this is not the case anymore with the grouped tool call.

[mfisher87]

I think the singular tool call UI looks great. It'd be amazing to change the grouped tool calls to use those same design ideas for a consistent experience across jupyterlite-ai and jupyter-ai!

[dlqqq]

"I'm consistently seeing "✔️ Other..." for tool use in the chat log, with some exceptions"

Thanks for reporting this. I think I've seen the same behavior when OpenCode calls MCP tools - there is some issue in our jupyter-ai-acp-client with how we render the MCP tool call results returned by OpenCode. It likely uses a slightly different format we do not expect.

We're still discussing how best to unify the UI between Jupyter AI and Jupyterlite AI.