Skip to content

addon-image: Fix Kitty placement lifecycle - #6098

Open
mitchdenny wants to merge 1 commit into
xtermjs:masterfrom
mitchdenny:mitchdenny-fix-kitty-placements
Open

addon-image: Fix Kitty placement lifecycle#6098
mitchdenny wants to merge 1 commit into
xtermjs:masterfrom
mitchdenny:mitchdenny-fix-kitty-placements

Conversation

@mitchdenny

Copy link
Copy Markdown

Follow-up to #5707 and #5722. Partially addresses the a/A viewport scope and
lower/uppercase lifetime requirements in #5710; the remaining delete selectors
from that issue are out of scope.

Why

Kitty image data and placements have different lifetimes. One transmitted image
can have multiple displayed placements, and a non-zero placement ID identifies
one placement by (imageId, placementId).

The current addon maps a Kitty image ID to one ImageStorage entry. Multiple
placements therefore overwrite shared bookkeeping, same-ID replacement can
leave old cells visible, and targeted deletion can remove shared image data.
It also decodes the transmitted source again for each placement.

Behavior

Command Result
Repeated a=p,i=N,p=P Replaces only named placement (N,P)
a=p,i=N or p=0 Adds an independent anonymous placement
d=i,i=N,p=P Deletes only (N,P) and preserves image data
d=I,i=N,p=P Deletes only (N,P) and frees data only if unreferenced
d=i/I,i=N Deletes all placements for image N; uppercase frees unreferenced data
d=a/A Deletes only active-buffer placements visible in the viewport; uppercase frees only data left unreferenced
Retransmit i=N Removes all old placements and replaces the transmitted data

Explicit replacement and deletion clear their cell metadata so stale
placeholders cannot remain.

Implementation

  • Separates image-scoped payload/decoded-source state from named and anonymous
    placement records.
  • Reuses one lazily decoded source bitmap for placement crop/scale operations.
  • Indexes placement cells so targeted cleanup is proportional to the affected
    placement rather than terminal scrollback size.
  • Rebuilds cell indexes after resize/reflow and handles shifted cells.
  • Scans both normal and alternate buffers before removing unreferenced index
    entries.
  • Preserves placement tombstones when raster quota eviction needs placeholder
    and later deletion semantics.
  • Accounts decoded source pixels in storage limits and retains the existing hard
    cap of 256 transmitted images.
  • Invalidates async placement work on reset, retransmit, delete, and dispose;
    stale bitmaps are closed before they can reach storage.
  • Treats HAS_EXTENDED as authoritative in new visibility/index scans, matching
    the guidance in image addon: overwriting with text does not remove tiles anymore #5860. This PR does not change image addon: overwriting with text does not remove tiles anymore #5860's existing renderer/text
    overwrite behavior.
  • Extends the image demo with a single 10-second placement-rain smoke demo. It
    transmits one transparent teardrop sprite, animates a dense randomized field
    of named placements at independent pixel velocities using sub-cell offsets,
    targeted-deletes and reuses them, adds anonymous p=0 placements, clears
    visible placements with d=a, proves payload reuse once more, and ends with
    uppercase cleanup.
  • Adds a 15-second bouncing wireframe-cube demo that pre-renders and transmits 60
    transparent 200x200 PNG frames once, then moves/replaces the active placement
    and switches frames through lowercase hide/show operations before freeing all
    stored frame data.

Performance

  • Image bytes are transmitted once and source pixels are decoded once while the
    decoded source remains cached.
  • Named move/replace and targeted delete are O(cells in that placement).
  • d=a/A scans only the active viewport.
  • Resize/reflow is the only path that rebuilds indexes across buffers.
  • Decoded source pixels are quota-accounted and evicted before visible placement
    rasters.

Scope and compatibility

Validation

  • npm run build
  • npm run esbuild
  • npm run esbuild-demo-client
  • npm run esbuild-demo-server
  • npm run lint-changes
  • npm run test-unit -- addons/addon-image/out-esbuild/kitty/KittyGraphicsTypes.test.js
    • 23 passing
  • npm run test-integration -- --suite=addon-image --project=Chromium
    • 236 passing, 1 skipped
  • npm run test-integration -- --suite=addon-image
    • 707 passing, 4 skipped across Chromium, Firefox, and WebKit
  • git diff --check

The demo client/server bundles were built, and the Playwright suite exercised
the addon through its automatically started demo server. This was automated
integration validation, not a manual walkthrough of the interactive xterm.js
demo UI. The new placement-rain button was also exercised headlessly through its
full animation with no page errors. The wireframe-cube button completed its
preload, 15-second animation, and cleanup without page errors or viewport
scrolling.

The integration suite adds 31 regression cases covering named siblings,
same-placement replacement, anonymous placements, targeted and viewport-scoped
deletion, scrollback and alternate-buffer references, shared decode lifetime,
crop fragments, retransmission, quota pressure, stale metadata, ICH/DCH shifts,
resize/reflow, alternate-only index rebuilds, and deterministic async
reset/retransmit/delete/dispose races.

Downstream validation in Hex1b used the packed CommonJS addon with placeholders
enabled:

  • one transmission produced two independently rendered named placements;
  • moving one placement cleared its old pixels without affecting its sibling;
  • targeted lowercase deletion removed only one placement;
  • the deleted placement was recreated without retransmission;
  • a 24-step browser drag showed no stale pixels or checkerboard artifacts.

Hex1b was used as an independent black-box conformance oracle; the implementation
and regression tests are written directly for xterm.js.

Track Kitty image data separately from named and anonymous placements, implement protocol-correct placement deletion and replacement, and reuse decoded sources across placements.

Add regression coverage for placement identity, cleanup, buffer scope, quotas, and async races, plus visual placement demos.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: b97ee6e0-c828-45cd-a388-a4bdd396b4d6
@mitchdenny

Copy link
Copy Markdown
Author

I spiced up the demo app to show how useful the ability retain and re-place images can be:

Screen.Recording.2026-08-11.at.10.51.46.pm.mov

The reason I started working on this is I was trying to hook up my .NET terminal emulator/UI stack project to xterm.js' KGP support and found a few glitches in my rendering which turned out to be incomplete implementation on the xterm.js side. Here is an example of xterm.js in action in a demo web-UX that hosts my TUI framework.

Screen.Recording.2026-08-11.at.8.29.22.pm.mov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant