UI display for N8N AI Agent tool usage in n8n.py (N8P Pipeline) #75
Description
Contribution Terms
- I have reviewed the project’s Code of Conduct and contribution guidelines.
- I plan to implement this feature myself and submit a pull request.
Feature title
UI display for N8N AI Agent tool usage in n8n.py (N8P Pipeline)
Feature overview
Add a UI element to display which tools the N8N AI Agent has called during a pipeline run. This feature aims to make tool invocation transparent for users by showing the tool usage history for both streaming and non-streaming responses. Example response payloads are provided for both modes, including detailed intermediateSteps with tool/action, toolInput, log, and messageLog. Target users are developers or analysts who want more insight into the AI agent's workflow execution steps.
Implementation details
- Extend the n8n.py pipeline to extract and forward tool usage information from the 'intermediateSteps' array in both streaming and non-streaming N8N responses.
- Design a UI component (e.g., expandable details or sidebar) that visually lists tool calls, their parameters, and results per step.
- Display tool name, input, and optionally log/messageLog details for each action, supporting both streaming and non-streaming JSON structures as shown in the user's payload examples.
- Key code locations for integration:
pipelines/n8n/n8n.py(see pipe() method and handling of streaming/non-streaming responses)- Ensure tool usage info from 'intermediateSteps' is available to the frontend/UI.
- Example code locations:
- Optionally, provide a mapping or filtering for known tool types.
- Maintain robust error handling and backward compatibility for users who do not require this feature.
Tasks and milestones
- Parse 'intermediateSteps' in both streaming and non-streaming modes of the N8N response
- Define and implement a UI element to display tool usage
- Connect backend extraction to frontend display logic
- Write tests for tool usage extraction and display
- Document the feature with usage examples
Questions or areas for feedback
- Should the tool usage UI be always visible, or collapsible/hidden by default?
- Should full logs/messageLog per tool call be shown, or only tool name and input?
- Should this support all future tool types, or only a curated set?
Additional context or references
Example N8N responses:
- Streaming:
[
{
"output": "Current time in Europe/London: 2025-10-10 09:46:45 BST (UTC+1). \n\nWhat Wikipedia says (summary):\n- The Wikipedia \"Time zone\" article explains that time zones are regions that observe a uniform standard time, defined by offsets from Coordinated Universal Time (UTC). Offsets range from UTC−12:00 to UTC+14:00; some zones use 30- or 45-minute offsets.\n- Many places use daylight saving time (DST), moving clocks ahead (typically by one hour) in spring/summer. Wikipedia gives the example that during DST the United Kingdom observes UTC+01:00 (British Summer Time).\n- The article also covers history: the Royal Observatory, Greenwich (founded 1675) established Greenwich Mean Time (GMT) as a reference; GMT became legal time in Britain in 1880; the piece describes the move from local solar time to standard time (including \"railway time\").\n\nWould you like a direct link to the full Wikipedia article or the exact article text about Europe/London (IANA time zone) or London itself?",
"intermediateSteps": [
{
"action": {
"tool": "Date_Time",
"toolInput": {
"Include_Current_Time": true,
"Timezone": "Europe/London"
},
"log": "Calling Date_Time with input: {\"Include_Current_Time\":true,\"Timezone\":\"Europe/London\"}",
"messageLog": [
{
"lc": 1,
"type": "constructor",
"id": [
"langchain_core",
"messages",
"AIMessageChunk"
],
"kwargs": {
"content": "",
"additional_kwargs": {
"tool_calls": [
{
"function": {
"arguments": "{\"Include_Current_Time\": true, \"Timezone\": \"Europe/London\"}",
"name": "Date_Time"
},
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"index": 0,
"type": "function"
},
{
"function": {
"arguments": "{\"input\": \"Europe/London time zone Wikipedia\"}",
"name": "Wikipedia"
},
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"index": 1,
"type": "function"
}
]
},
"response_metadata": {
"usage": {
"completion_tokens": 259,
"completion_tokens_details": {
"accepted_prediction_tokens": 0,
"audio_tokens": 0,
"reasoning_tokens": 192,
"rejected_prediction_tokens": 0
},
"prompt_tokens": 278,
"prompt_tokens_details": {
"audio_tokens": 0,
"cached_tokens": 0
},
"total_tokens": 537
}
},
"tool_call_chunks": [
{
"name": "Date_Time",
"args": "{\"Include_Current_Time\": true, \"Timezone\": \"Europe/London\"}",
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"index": 0,
"type": "tool_call_chunk"
},
{
"name": "Wikipedia",
"args": "{\"input\": \"Europe/London time zone Wikipedia\"}",
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"index": 1,
"type": "tool_call_chunk"
}
],
"id": "chatcmpl-CP33qmnqlTpj89lAGpEhbLZr9jqTV",
"usage_metadata": {
"input_tokens": 278,
"output_tokens": 259,
"total_tokens": 537,
"input_token_details": {
"audio": 0,
"cache_read": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 192
}
},
"tool_calls": [
{
"name": "Date_Time",
"args": {
"Include_Current_Time": true,
"Timezone": "Europe/London"
},
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"type": "tool_call"
},
{
"name": "Wikipedia",
"args": {
"input": "Europe/London time zone Wikipedia"
},
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"type": "tool_call"
}
],
"invalid_tool_calls": []
}
}
],
"toolCallId": "call_FB0sIgrwuIGJkOaROor7raU2",
"type": "tool_call"
},
"observation": "[{\"currentDate\":\"2025-10-10T09:46:45.754+01:00\"}]"
},
{
"action": {
"tool": "Wikipedia",
"toolInput": {
"input": "Europe/London time zone Wikipedia"
},
"log": "Calling Wikipedia with input: {\"input\":\"Europe/London time zone Wikipedia\"}",
"messageLog": [
{
"lc": 1,
"type": "constructor",
"id": [
"langchain_core",
"messages",
"AIMessageChunk"
],
"kwargs": {
"content": "",
"additional_kwargs": {
"tool_calls": [
{
"function": {
"arguments": "{\"Include_Current_Time\": true, \"Timezone\": \"Europe/London\"}",
"name": "Date_Time"
},
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"index": 0,
"type": "function"
},
{
"function": {
"arguments": "{\"input\": \"Europe/London time zone Wikipedia\"}",
"name": "Wikipedia"
},
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"index": 1,
"type": "function"
}
]
},
"response_metadata": {
"usage": {
"completion_tokens": 259,
"completion_tokens_details": {
"accepted_prediction_tokens": 0,
"audio_tokens": 0,
"reasoning_tokens": 192,
"rejected_prediction_tokens": 0
},
"prompt_tokens": 278,
"prompt_tokens_details": {
"audio_tokens": 0,
"cached_tokens": 0
},
"total_tokens": 537
}
},
"tool_call_chunks": [
{
"name": "Date_Time",
"args": "{\"Include_Current_Time\": true, \"Timezone\": \"Europe/London\"}",
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"index": 0,
"type": "tool_call_chunk"
},
{
"name": "Wikipedia",
"args": "{\"input\": \"Europe/London time zone Wikipedia\"}",
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"index": 1,
"type": "tool_call_chunk"
}
],
"id": "chatcmpl-CP33qmnqlTpj89lAGpEhbLZr9jqTV",
"usage_metadata": {
"input_tokens": 278,
"output_tokens": 259,
"total_tokens": 537,
"input_token_details": {
"audio": 0,
"cache_read": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 192
}
},
"tool_calls": [
{
"name": "Date_Time",
"args": {
"Include_Current_Time": true,
"Timezone": "Europe/London"
},
"id": "call_FB0sIgrwuIGJkOaROor7raU2",
"type": "tool_call"
},
{
"name": "Wikipedia",
"args": {
"input": "Europe/London time zone Wikipedia"
},
"id": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"type": "tool_call"
}
],
"invalid_tool_calls": []
}
}
],
"toolCallId": "call_QFUtaSdUI2PtgjhkDTmbRknt",
"type": "tool_call"
}
},
{
"action": {
"tool": "Wikipedia",
"toolInput": {
"input": "Europe/London time zone site:en.wikipedia.org Europe/London"
},
"log": "Calling Wikipedia with input: {\"input\":\"Europe/London time zone site:en.wikipedia.org Europe/London\"}",
"messageLog": [
{
"lc": 1,
"type": "constructor",
"id": [
"langchain_core",
"messages",
"AIMessageChunk"
],
"kwargs": {
"content": "",
"additional_kwargs": {
"tool_calls": [
{
"function": {
"arguments": "{\"input\":\"Europe/London time zone site:en.wikipedia.org Europe/London\"}",
"name": "Wikipedia"
},
"id": "call_nUopQ8uhZOzhQYKLUPUzHNUG",
"index": 0,
"type": "function"
}
]
},
"response_metadata": {
"usage": {
"completion_tokens": 228,
"completion_tokens_details": {
"accepted_prediction_tokens": 0,
"audio_tokens": 0,
"reasoning_tokens": 192,
"rejected_prediction_tokens": 0
},
"prompt_tokens": 1252,
"prompt_tokens_details": {
"audio_tokens": 0,
"cached_tokens": 0
},
"total_tokens": 1480
}
},
"tool_call_chunks": [
{
"name": "Wikipedia",
"args": "{\"input\":\"Europe/London time zone site:en.wikipedia.org Europe/London\"}",
"id": "call_nUopQ8uhZOzhQYKLUPUzHNUG",
"index": 0,
"type": "tool_call_chunk"
}
],
"id": "chatcmpl-CP33vRWGen7W4tVo6QGxlNSDW7k9j",
"usage_metadata": {
"input_tokens": 1252,
"output_tokens": 228,
"total_tokens": 1480,
"input_token_details": {
"audio": 0,
"cache_read": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 192
}
},
"tool_calls": [
{
"name": "Wikipedia",
"args": {
"input": "Europe/London time zone site:en.wikipedia.org Europe/London"
},
"id": "call_nUopQ8uhZOzhQYKLUPUzHNUG",
"type": "tool_call"
}
],
"invalid_tool_calls": []
}
}
],
"toolCallId": "call_nUopQ8uhZOzhQYKLUPUzHNUG",
"type": "tool_call"
}
}
]
}
]- Non-streaming:
[
{
"output": "Current time in Europe/London: 09:47:11 on 10 October 2025 (BST — UTC+1).\n\nWhat Wikipedia says (brief summary from the London article):\n- London is the capital and largest city of England and the United Kingdom.\n- Population: roughly 9.8 million in Greater London (2025) with a wider metropolitan area around 15 million.\n- Sits on the River Thames; its historic core (the City of London) was founded by the Romans as Londinium.\n- The City of Westminster hosts national government and Parliament.\n- A major global city with strong influence in art, finance, education, media, science and technology; Europe’s largest city economy and a leading financial centre.\n- Home to many higher-education institutions (Imperial, UCL, KCL, LSE, etc.) and over 300 languages spoken.\n- Notable features: London Underground (world’s oldest rapid transit), four UNESCO World Heritage Sites (including the Tower of London and Greenwich/Prime Meridian), numerous museums, theatres and landmarks.\n\nIf you’d like, I can fetch a specific section from the Wikipedia page or provide a direct link/longer excerpt.",
"intermediateSteps": [
{
"action": {
"tool": "Date_Time",
"toolInput": {
"Include_Current_Time": true,
"Timezone": "Europe/London"
},
"toolCallId": "call_PvgeiPQ2AoDl6QTEOM4YQQHk",
"log": "Invoking \"Date_Time\" with {\"Include_Current_Time\":true,\"Timezone\":\"Europe/London\"}\n",
"messageLog": [
{
"lc": 1,
"type": "constructor",
"id": [
"langchain_core",
"messages",
"AIMessage"
],
"kwargs": {
"content": "",
"additional_kwargs": {
"tool_calls": [
{
"function": {
"arguments": "{\"Include_Current_Time\": true, \"Timezone\": \"Europe/London\"}",
"name": "Date_Time"
},
"id": "call_PvgeiPQ2AoDl6QTEOM4YQQHk",
"type": "function"
},
{
"function": {
"arguments": "{\"input\": \"Europe/London\"}",
"name": "Wikipedia"
},
"id": "call_7A6HBB9lpLoNqJRWXcejCUER",
"type": "function"
}
]
},
"response_metadata": {
"tokenUsage": {
"promptTokens": 278,
"completionTokens": 256,
"totalTokens": 534
},
"finish_reason": "tool_calls",
"model_name": "gpt-5-mini-2025-08-07"
},
"id": "chatcmpl-CP34Fu1Ui3J4tYKccGGyN2mFgvE60",
"tool_calls": [
{
"name": "Date_Time",
"args": {
"Include_Current_Time": true,
"Timezone": "Europe/London"
},
"type": "tool_call",
"id": "call_PvgeiPQ2AoDl6QTEOM4YQQHk"
},
{
"name": "Wikipedia",
"args": {
"input": "Europe/London"
},
"type": "tool_call",
"id": "call_7A6HBB9lpLoNqJRWXcejCUER"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"output_tokens": 256,
"input_tokens": 278,
"total_tokens": 534,
"input_token_details": {
"audio": 0,
"cache_read": 0
},
"output_token_details": {
"audio": 0,
"reasoning": 192
}
}
}
}
]
},
"observation": "[{\"currentDate\":\"2025-10-10T09:47:11.011+01:00\"}]"
},
{
"action": {
"tool": "Wikipedia",
"toolInput": {
"input": "Europe/London"
},
"toolCallId": "call_7A6HBB9lpLoNqJRWXcejCUER",
"log": "Invoking \"Wikipedia\" with {\"input\":\"Europe/London\"}\n",
"messageLog": []
},
"observation": "Page: London\nSummary: London is the capital and largest city of both England and the United Kingdom, with a population of 9.1 million people in 2024. Its wider metropolitan area is the largest in Western Europe, with a population of 15.1 million. London stands on the River Thames in southeast England, at the head of a 50-mile (80 km) tidal estuary down to the North Sea, and has been a major settlement for nearly 2,000 years. Its ancient core and financial centre, the City of London, was founded by the Romans as Londinium and has retained its medieval boundaries. The City of Westminster, to the west of the City of London, has been the centuries-long host of the national government and parliament. London grew rapidly in the 19th century, becoming the world's largest city at the time. Since the 19th century the name \"London\" has referred to the metropolis around the City of London, historically split between the counties of Middlesex, Essex, Surrey, Kent and Hertfordshire, which since 1965 has largely comprised the administrative area of Greater London, governed by 33 local authorities and the Greater London Authority.\nAs one of the world's major global cities, London exerts a strong influence on world art, entertainment, fashion, commerce, finance, education, healthcare, media, science, technology, tourism, transport and communications. London is Europe's largest city economy, and is one of the world's major financial centres. London hosts Europe's largest concentration of higher education institutions, comprising over 50 universities and colleges and enrolling more than 500,000 students as at 2023. It is home to several of the world's leading academic institutions: Imperial College London, internationally recognised for its excellence in natural and applied sciences, and University College London (UCL), a comprehensive research-intensive university, consistently rank among the top ten globally. Other notable institutions include King's College London (KCL), highly regarded in law, humanities and health sciences; the London School of Economics (LSE), globally prominent in social sciences and economics; and specialised institutions such as the Royal College of Art (RCA), Royal Academy of Music (RAM), the Royal Academy of Dramatic Art (RADA), the School of Oriental and African Studies (SOAS) and London Business School (LBS). It is the most-visited city in Europe and has the world's busiest city airport system. The London Underground is the world's oldest rapid transit system.\nLondon's diverse cultures encompass over 300 languages. The 2025 population of Greater London of just over 9.8 million made it Europe's third-most populous city, accounting for 13.1 per cent of the United Kingdom's population and 15.5 per cent of England's population. The Greater London Built-up Area is the fourth-most populous in Europe, with about 9.8 million inhabitants as of 2011. The London metropolitan area is the third-most-populous in Europe, with about 15 million inhabitants as of 2025, making London a megacity.\nFour World Heritage Sites are located in London: Kew Gardens; the Tower of London; the site featuring the Palace of Westminster, the Church of St Margaret, and Westminster Abbey; and the historic settlement in Greenwich where the Royal Observatory defines the prime meridian (0° longitude) and Greenwich Mean Time. Other landmarks include Buckingham Palace, the London Eye, Piccadilly Circus, St Paul's Cathedral, Tower Bridge and Trafalgar Square. The city has the most museums, art galleries, libraries and cultural venues in the UK, including the British Museum, the National Gallery, the Natural History Museum, Tate Modern, the British Library and numerous West End theatres. Important sporting events held in London include the FA Cup Final, the Wimbledon Tennis Championships and the London Marathon. It became the first city to host three Summer Olympic Games upon hosting the 2012 Summer Olympics.\n\n\n== Etymology ==\n\nLondon is an ancient name, att"
}
]
}
]Relevant code:
- pipelines/n8n/n8n.py (see
pipe()method for response processing)
Related docs: