Skip to content

fix: properly render DALL-E/gpt-image responses in request drawer#5542

Merged
chitalian merged 1 commit intomainfrom
fix/dalle-image-rendering
Jan 21, 2026
Merged

fix: properly render DALL-E/gpt-image responses in request drawer#5542
chitalian merged 1 commit intomainfrom
fix/dalle-image-rendering

Conversation

@chitalian
Copy link
Contributor

Summary

  • Fixes gpt-image-1 and gpt-image-1.5 responses not rendering correctly in the request drawer
  • Adds proper role field to response messages so they appear in ChatOnlyView
  • Adds request.messages array so user prompts are displayed
  • Adds data:image/png;base64, prefix for base64 images so ImageContent can render them

Problem

After PR #5536 added cost support for gpt-image models, the generated images weren't rendering in the request drawer. The UI showed "No user or assistant messages found in this request."

Root Cause

The DALL-E mapper was not properly structuring messages for the UI:

  1. Response messages lacked a role field, causing ChatOnlyView to filter them out
  2. Request didn't include a messages array, so the Chat component didn't show the user prompt
  3. Raw base64 strings from OpenAI's b64_json field weren't being recognized as images by ImageContent (which checks for base64, or https:// in the URL)

Test plan

  • Run llm-mapper tests: npx jest llm-mapper
  • Make a test request to gpt-image-1.5 locally
  • Verify the request drawer shows the user prompt
  • Verify the generated image renders in the assistant response

Screenshot

gpt-image-1.5 rendering correctly

🤖 Generated with Claude Code

- Add `role: "assistant"` to response messages so ChatOnlyView displays them
- Add `request.messages` array with user prompt for proper Chat component rendering
- Add `data:image/png;base64,` prefix for base64 images so ImageContent can process them

The DALL-E mapper was not properly structuring messages for the UI:
1. Response messages lacked a `role` field, causing them to be filtered out
2. Request didn't include a `messages` array, so the user prompt wasn't shown
3. Raw base64 strings from OpenAI's `b64_json` weren't being recognized as images

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Jan 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
helicone Building Building Preview, Comment Jan 21, 2026 10:02pm
helicone-bifrost Building Building Preview, Comment Jan 21, 2026 10:02pm
helicone-eu Building Building Preview, Comment Jan 21, 2026 10:02pm

Request Review

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@supabase
Copy link

supabase bot commented Jan 21, 2026

This pull request has been ignored for the connected project bolqqmqbrciybnypvklh because there are no changes detected in supbase/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@supabase
Copy link

supabase bot commented Jan 21, 2026

This pull request has been ignored for the connected project lmahfbbnchpworytrrqk because there are no changes detected in .supabase/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@claude
Copy link
Contributor

claude bot commented Jan 21, 2026

Claude finished @chitalian's task —— View job


PR Review: DALL-E Image Rendering Fix

Todo List:

  • Read and analyze the modified DALL-E mapper file
  • Review the test file changes
  • Check for security implications
  • Evaluate code quality and best practices
  • Provide detailed feedback with recommendations
  • Assign confidence score

Score: 8/10 - Good to merge with minor suggestions

This is a solid fix that properly addresses the rendering issues for DALL-E generated images. The implementation is correct and well-tested.

Detailed Review

Strengths

  1. Correctly Fixes the Root Issues:

    • ✅ Adds missing role: "assistant" field (line 70 in dalle.ts)
    • ✅ Adds messages array to request schema for proper UI rendering (lines 57-65)
    • ✅ Properly formats base64 images with data:image/png;base64, prefix (lines 38-42)
  2. Comprehensive Test Coverage:

    • ✅ Tests successful image generation scenario with all key fields
    • ✅ Tests error handling with proper error message extraction
    • ✅ Tests edge cases with missing data
  3. Good Code Quality:

    • ✅ Clear, readable logic with appropriate comments
    • ✅ Proper null/undefined handling
    • ✅ Consistent with existing LlmSchema structure

🔍 Minor Suggestions for Improvement

1. Image Type Detection Enhancement (dalle.ts:41)

// Current: Always assumes PNG
: `data:image/png;base64,${rawImageData}`

// Suggestion: Detect image type or make it configurable
: `data:image/${request.response_format === 'b64_json' ? 'png' : 'png'};base64,${rawImageData}`

2. Type Safety Enhancement (dalle.ts:36-37)

The code accesses nested properties without type guards:

// Consider adding optional chaining validation
const rawImageData = response?.data?.[0]?.b64_json || response?.data?.[0]?.url || "";

This is actually well-handled, but could benefit from stronger typing.

3. Test Robustness (dalle.test.ts:18)

The test uses a very short base64 string "iVB=". Consider using a more realistic test value or mocking:

// More realistic test data
b64_json: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="

🔐 Security Assessment

  • No security concerns: Base64 image data is properly handled
  • Input validation: Proper checks for malformed responses
  • No injection risks: Direct assignment without dynamic evaluation

🎯 Performance Assessment

  • Minimal overhead: Simple string operations
  • No unnecessary computations: Efficient conditional logic
  • Good data flow: Clear separation between schema and preview

Suggestions Summary:

  • Optional: Consider dynamic image type detection instead of hardcoded PNG
  • Optional: Use more realistic test data for base64 strings
  • Optional: Add stronger TypeScript typing for response structure

Ready to Merge

This PR effectively solves the stated problem with a clean, well-tested implementation. The minor suggestions above are optional improvements that don't block merging.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant