Problem
Tool results can contain multimodal content blocks, including base64-encoded images. The current UI renders these results as raw JSON/text, exposing the base64 string instead of displaying the image.
LangChain’s Python and JavaScript implementations also use different image field names:
- Python:
base64, mime_type
- JavaScript:
data, mimeType
Expected behavior
When a tool result’s content is an array containing a block with type: "image":
- Render the image inline using a
data:<mime-type>;base64,<data> URL.
- Support both Python and JavaScript field conventions.
- Render accompanying
type: "text" blocks above the image.
- Do not display the raw base64 data.
- Make the image responsive, width-constrained, rounded, and accessible with descriptive alt text.
- Preserve the existing renderer for JSON and ordinary text results.
Example payloads
Python LangChain
[
{
"type": "text",
"text": "Generated chart"
},
{
"type": "image",
"base64": "<base64-data>",
"mime_type": "image/png"
}
]
Problem
Tool results can contain multimodal content blocks, including base64-encoded images. The current UI renders these results as raw JSON/text, exposing the base64 string instead of displaying the image.
LangChain’s Python and JavaScript implementations also use different image field names:
base64,mime_typedata,mimeTypeExpected behavior
When a tool result’s
contentis an array containing a block withtype: "image":data:<mime-type>;base64,<data>URL.type: "text"blocks above the image.Example payloads
Python LangChain
[ { "type": "text", "text": "Generated chart" }, { "type": "image", "base64": "<base64-data>", "mime_type": "image/png" } ]