Inline media artifact events (follow-up to #956)

[heavygee]

Context

Follow-up to #956 / PR #958 (cross-flavor inline image + video). v1 ships a unified generated-image wire path with optional source provenance (ingress, flavor, toolCallId, toolName). This issue tracks the full artifact-event model proposed by @Necmttn on #956.

Problem

Inline media enters HAPI through multiple ingress paths (display_image / display_video MCP, ACP image blocks, Codex/tool-result images, future #508 tool-result base64). Operators and overseer tooling need one comparable record per inline artifact, not just bytes + a chat card.

Proposed v2 artifact event (seed spec — please confirm)

Each inline media display should be traceable as an artifact linked to its producer:

Field	Purpose
sessionId	HAPI session row
ingress	mcp | acp | tool_result (v1 source.ingress)
flavor	Agent flavor when known (cursor, codex, gemini, …)
toolCallId	Producer tool call when applicable
toolName	e.g. display_image, display_video, or native tool name
artifactId	Stable id (= current imageId / generated-media store key in v1)
mimeType	e.g. image/png, video/mp4
byteSize	Stored payload size
contentHash	Optional digest for dedup / audit
hubRoute	e.g. GET /api/sessions/:id/generated-images/:artifactId
renderStatus	e.g. registered | fetched | failed
visibilityOutcome	e.g. inline chat card shown | empty card | fetch error

v1 already emits a subset on the chat message (source + imageId + mime). v2 would persist/query artifact rows (likely overseer/events substrate or hub-side registry) without changing the chat card UX.

Ask for @Necmttn

Please review this field list and comment:

1. Confirm this is the right minimal artifact shape, or
2. Revise (add/remove/rename fields — especially ingress vs path, and whether renderStatus / visibilityOutcome should be hub-owned vs web-reported)

Non-goals (this issue)

• Renaming generated-image message type or hub route (can be a separate cleanup)
• Cursor IDE composer inline media (HAPI web only)

References

• feat(cli+web): inline image display for all agent flavors #956 enhancement
• PR feat(cli): cross-flavor inline image and video display via MCP and ACP #958
• v1 types: cli/src/modules/common/inlineMediaSource.ts, wire field source on generated-image messages

[heavygee]

@Necmttn — you raised the artifact-event model on #956. This issue seeds the v2 field list we would persist/query beyond v1 source on chat messages. Please confirm, revise, or add fields (especially render/visibility telemetry). Thanks!

[github-actions]

Verified current baseline in this checkout. The proposed v2 model is not implemented yet: current inline media is still a generated-image chat message plus a CLI in-memory blob lookup. I did not find a persisted artifact row/event substrate, artifactId, contentHash, renderStatus, visibilityOutcome, display_video, or generated-video implementation in cli/src, hub/src, shared/src, or web/src.

For the v2 field list, the fields directly derivable from current code are sessionId at fetch time, current imageId as the artifact key, mimeType, fileName, byte size from stored content, and hubRoute. Producer provenance is only partially derivable today: MCP display_image and Codex generated_image are separate ingress paths, but I did not find the source provenance object mentioned in the issue in this checkout.

Recommendation from code evidence: keep producer-owned fields (ingress, flavor, toolCallId, toolName, artifactId, mimeType, byteSize, optional contentHash, hubRoute) separate from presentation telemetry. renderStatus can be hub-owned for fetch/registry state, while visibilityOutcome looks web-owned because the web card is where object URL creation and fetch errors are observed.

Relevant code:

• cli/src/modules/common/generatedImages.ts:3 - current stored generated-image metadata is id, fileName, content, mimeType, createdAt.
• cli/src/modules/common/generatedImages.ts:15 - generated images are held in a process-local Map, not a persisted table.
• cli/src/modules/common/generatedImages.ts:88 - old images are evicted by count/total-byte limits.
• cli/src/claude/utils/startHappyServer.ts:84 - MCP display_image tool registration.
• cli/src/claude/utils/startHappyServer.ts:108 - display_image registers bytes and emits a generated-image message with imageId, fileName, and mimeType.
• cli/src/codex/codexRemoteLauncher.ts:2213 - Codex generated_image event path registers an image from saved_path.
• cli/src/modules/common/handlers/files.ts:56 - CLI RPC handler reads a generated image by id and returns base64 content, MIME type, and filename.
• hub/src/web/routes/git.ts:152 - hub route is GET /api/sessions/:id/generated-images/:imageId.
• hub/src/web/routes/git.ts:179 - hub fetches generated image bytes through session RPC.
• web/src/api/client.ts:320 - web client fetches generated-image blobs from the hub route.
• web/src/components/AssistantChat/messages/ToolMessage.tsx:57 - web card tracks local object URL/error state while loading the image.

Need more info:

• Should v2 first cover only the currently implemented generated-image path, or also define video/base64 rows before those paths exist in this checkout?
• Should visibilityOutcome be sent back from web as telemetry, or should v2 stop at hub fetch/registry status for now?

---

HAPI Bot