Retry frames refused by the Metal renderer

[batonogov]

Metal renderer stops presenting after a refused frame

Summary

MetalTerminalRenderer.draw(in:) can permanently stop presenting frames. The view keeps accepting input and terminal output, the Terminal and PTY stay healthy, but nothing is ever drawn again — the pane is simply black until the client rebuilds the renderer.

Mechanism

pendingRedraw is consumed in exactly one place: the completion handler of a submitted command buffer.

commandBuffer.addCompletedHandler { [weak self, weak view] _ in
    frameSemaphore.signal()
    ...
    if self.consumePendingRedraw() { ...setNeedsDisplay... }
}

But every path that refuses a frame returns before a command buffer exists:

1. frameSemaphore.wait(timeout: .now()) != .success → markPendingRedraw(), return.
2. guard let drawable, let passDescriptor else → markPendingRedraw(), signal, return.
3. makeCommandBuffer() / makeRenderCommandEncoder(descriptor:) returning nil → signal, return — this one does not even mark a pending redraw, so the requested frame is lost outright.

In all three cases the pending request has no consumer. Because the view is on-demand (isPaused = true, enableSetNeedsDisplay = true, autoResizeDrawable = false), nothing re-drives it on its own.

Recovery then depends entirely on an external invalidation — but for a terminal that invalidation re-enters the same refusal. Further output calls queueMetalDisplay() → setNeedsDisplay → draw(in:) → refused again. drawMetalFrameNow() is no better: it calls metalView.draw(), which lands on the same early return. So from the embedding app's side the renderer is unrecoverable, and there is no public signal ("was a frame presented?") to detect the state and react.

Case 1 is the most durable: if a completion handler is ever missed or delayed, the semaphore stays held, and from then on every draw takes the first early return.

Proposed fix

Re-request the frame from the refusal paths themselves instead of relying on a completion handler that, by construction, does not exist there.

The attached patch:

• schedules a retry from all three refusal paths, and marks a pending redraw in case 3, which currently drops it silently;
• coalesces retries (one in flight at a time) and backs off from one 60 Hz frame up to 500 ms, so a view that genuinely cannot present (zero-sized, off-screen, waiting on a lost completion) settles into a cheap poll rather than spinning at frame rate;
• resets the backoff whenever a frame is actually submitted;
• peeks at pendingRedraw in the retry rather than consuming it, so the flag stays owned by the submitted-frame completion handler and the existing invariant is unchanged.

Builds clean against current main (swift build); the diagnosis was originally made against v1.18.0, where the code is identical.

How this was found

Pine, a macOS editor embedding SwiftTerm, saw terminal panes go black while their shells kept running — every child shell alive on its own tty, typing accepted, nothing rendered, and no repaint request from the app able to recover it. That pointed at presentation rather than the PTY, and the refusal paths above are the only way the renderer can go quiet without any error surfacing.

As a stopgap Pine added a user-invoked command that rebuilds the renderer via setUseMetal(false/true), which does recover the pane — further evidence that the terminal state itself is intact and only presentation is stuck.

I could not construct a deterministic reproducer for the semaphore case from outside the library — the window between a dropped completion and the next draw is not observable through the public API — so this is a code-level diagnosis rather than a test-backed one. Happy to adjust the shape of the fix (e.g. a bounded number of retries, or surfacing a "frame presented" signal so clients can react themselves) if you would rather solve it differently.

[migueldeicaza]

Thank you for your contribution!

Will add some comments to the review.

[migueldeicaza]

Let me write a small synthetic error injection, so we can debug this more easily.

[batonogov]

Thanks — that's exactly what this needs; I couldn't drive these paths deterministically through the public API.

To cover the bug, the injection needs to force all three refusal paths in draw(in:) independently: semaphore timeout, missing drawable / render pass descriptor, and failed command buffer or encoder creation. The third one matters most — it's the only path that never marks a pending redraw, so the request is lost outright, and a fix that only reschedules from the first two would still leave a permanently black view there.

The assertion I'd write on top of it: after N injected refusals, with injection turned off, a frame is presented without any further external invalidation.

If a frame-presented signal or a test-visible counter is easier to expose than a full harness, that's enough for me to write the regression test.

[migueldeicaza]

Ok, I am working on a more substantial change - it turns out that there are various conditions that can render metal unusable - and we should gracefully fallback to CoreGraphics in that case. And I have noticed that when trying Ghostty, when my machine is overloaded, I lose that terminal entirely, because it is GPU backend - so I can not recover the machine from the command line.

While Terminal.app continues to work, so I am going to make it so that we can recover from a transient metal failure.

[migueldeicaza]

Ok, I landed a version that deals better with this, can you try it?

[migueldeicaza]

Ok, the 2.0 release does not include the fix yet, because this introduced a performance regression there that I can not understand yet.