Skip to content

Map EventMsg::ImageGenerationEnd to ACP session/update (minimal) #251

@Yuyz0112

Description

@Yuyz0112

Summary

Codex generates images successfully (file lands in ~/.codex/generated_images/<session>/), but no ACP session/update is emitted, so editor clients see nothing. Tracking #242; this issue proposes a small, self-contained patch that doesn't depend on the larger #206.

Evidence

A real session rollout (rollout-…-<session>.jsonl) contains:

event_msg counts:
  agent_message ............... 7
  image_generation_end ........ 2
  task_started/task_complete .. 6/6
  token_count ................. 12
  user_message ................ 6

response_item counts:
  message ................. 19
  reasoning ............... 5
  image_generation_call ... 2

image_generation_end payload (truncated):

{
  "type": "image_generation_end",
  "call_id": "ig_00c8…2050",
  "status": "generating",
  "revised_prompt": "A retro sci-fi poster …",
  "result": "<base64 PNG>"
}

On the ACP wire we observed only agent_message_chunk, usage_update, and available_commands_update for that turn — no tool_call, no tool_call_update, no non-text content block. So image generation is not surfaced through the bridge at all today.

ACP already supports this

agent-client-protocol-schema (v0.12.0, used by codex-acp main) already defines ContentBlock::Image { data, mimeType, uri?, … }, and ContentBlock is allowed inside both agent_message_chunk.content and tool_call_update.content[].content. No protocol/SDK change is needed.

Proposed minimal change

In src/thread.rs, in PromptState::handle_event, add arms for the image generation events and emit them as a tool-call lifecycle, mirroring how exec commands are surfaced:

  • EventMsg::ImageGenerationCall { call_id, … } (or the begin event, depending on what Codex emits first) → SessionUpdate::ToolCall with kind: "other", title: "Image generation", status: "in_progress".
  • EventMsg::ImageGenerationEnd { call_id, result (base64), revised_prompt, status }SessionUpdate::ToolCallUpdate with status: "completed" and content: [{ type: "content", content: { type: "image", data: result, mimeType: "image/png" } }]. Optionally include revised_prompt as a text content block alongside the image so clients without image support still see something.

Open question: should the image be inlined as base64, or written to disk and referenced by uri? Codex already writes the file to generated_images/; emitting both data and uri would let clients pick. Happy to follow whichever convention the maintainers prefer.

Why a separate issue from #206

#206 is a great larger effort but bundles many independent changes (forked sessions, plan mode, slash command rewrites, auto-compaction, hook surfacing, …) and has been a draft since March. The image-gen mapping by itself is ~30 lines and unlocks #242 today. Happy to open a focused PR if maintainers are open to it.

Repro

  1. Run codex via codex-acp with a model that supports image gen, prompt for an image.
  2. Observe file at ~/.codex/generated_images/<session>/ig_*.png.
  3. Inspect ACP traffic: no tool_call* for the image; only agent_message_chunk/usage_update.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions