Skip to content

Support inline image rendering for LangChain tool results #307

Description

Problem

Tool results can contain multimodal content blocks, including base64-encoded images. The current UI renders these results as raw JSON/text, exposing the base64 string instead of displaying the image.

LangChain’s Python and JavaScript implementations also use different image field names:

  • Python: base64, mime_type
  • JavaScript: data, mimeType

Expected behavior

When a tool result’s content is an array containing a block with type: "image":

  • Render the image inline using a data:<mime-type>;base64,<data> URL.
  • Support both Python and JavaScript field conventions.
  • Render accompanying type: "text" blocks above the image.
  • Do not display the raw base64 data.
  • Make the image responsive, width-constrained, rounded, and accessible with descriptive alt text.
  • Preserve the existing renderer for JSON and ordinary text results.

Example payloads

Python LangChain

[
  {
    "type": "text",
    "text": "Generated chart"
  },
  {
    "type": "image",
    "base64": "<base64-data>",
    "mime_type": "image/png"
  }
]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions