Skip to content

Commit 638ee8d

Browse files
tui: dashboard color, cursor, auto-publish; async-first docs (#331)
Draft. Ships the visible tui-dashboard improvements on the existing vt100 backend, decoupled from the ghostty engine swap (see the companion `ix-vt` PR). Opened from in-flight agent work so progress is visible; not yet final. ## What this does - **Color**: `collect_frames` encodes the viewport's vt100 styled cells as minimal ANSI SGR into the frame `screen` string (new `packages/tui/src/frame/sgr.rs`); the dashboard parses SGR into styled spans (16 / 256-palette / truecolor, plus bold/italic/underline/inverse). - **Cursor**: position from vt100; shape (bar/block/underline) by sniffing DECSCUSR (`ESC [ Ps SP q`) in the PTY byte stream (new `packages/tui/src/actor/decscusr.rs`), since vt100 does not expose cursor shape. - **Auto-publish**: spawning a `Tui` auto-exposes it to `nix run .#tui-dashboard` (sync `ensure_published` in `tui-py`, opt out with `IX_TUI_AUTOPUBLISH=0`). No manual `tui.publish()` needed. - **Dead/empty cards**: placeholder + stable card height so empty or exited terminals still render; dead terminals stay listed with their exit code. - **Hover motion removed** from the dashboard; **MCP docs** (`server_instructions.txt`) and the `tui` Python docstrings rewritten to be async-first and explain auto-publish. ## Status - nix builds of the affected packages succeeded during development. - Carrying the agent's work as a single WIP commit; left to do: split into logical commits, final `nix run .#lint`, an end-to-end dashboard check, then mark ready for review. ## Approach note The DECSCUSR byte-sniff is intentionally throwaway. The companion `ix-vt` PR swaps the backend to ghostty's libghostty-vt, which exposes cursor shape natively. The wire here is engine-agnostic (SGR string + cursor fields), so that swap only changes the data source, not the renderer. Made with Claude (claude-opus-4-8). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 306cbe8 commit 638ee8d

18 files changed

Lines changed: 1081 additions & 26 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,15 @@
11
Persistent Python sessions on a pinned interpreter. Every session can `import tui` with no install step: `tui` is the bundled PyO3 PTY driver (spawn and drive PTY-backed processes with full vt100 emulation, scrollback, and optional NumPy access to per-cell data; the API is async-only, so use it from python_exec/python_eval with top-level await). numpy is preinstalled too, and each session's writable venv keeps `pip install` working for anything else.
22

33
Execution model: each session is single-threaded with one persistent asyncio event loop, reused across calls. Write top-level `await` directly (e.g. `rows = await pool.fetch(sql)`); because the loop is shared, an async client or pool created in one call stays usable in later ones. Do not call `asyncio.run()`: inside an awaited block it raises `RuntimeError: asyncio.run() cannot be called from a running event loop`, and outside one it spins up a throwaway loop your resources cannot outlive. `asyncio.get_running_loop()` resolves only inside awaited code. A blocking synchronous call (heavy CPU, `time.sleep`, sync network I/O) freezes the whole session, so reach for the async equivalent.
4+
5+
Driving a terminal with `tui`: spawn and control a process in one block, awaiting every I/O step.
6+
7+
from tui import Tui, Key
8+
async with Tui("python", "-q") as t:
9+
await t.enter("print(2 + 2)")
10+
snap = await t.wait_for("4", timeout=2.0)
11+
print(snap.text)
12+
13+
Every method that touches the terminal is a coroutine you must `await`: `send`, `enter`, `read`, `viewport`, `text`, `snapshot`, `wait_for`, `resize`, `kill`, `close`. Only the cached accessors are synchronous: `id`, `command`, `args`, `size`, `is_alive`, `exit_code`. Send keystrokes with the `Key` enum (`Key.ENTER`, `Key.ctrl("c")`, arrows). The `async with` block calls `close()` on exit, so an editor or REPL that ignores Ctrl+C still goes away.
14+
15+
Terminals auto-show in the dashboard. The first `Tui(...)` publishes this process, so a running `nix run .#tui-dashboard` (it watches `tui.socket_dir()`) renders every spawned terminal with no `tui.publish()` call. Set `IX_TUI_AUTOPUBLISH=0` to opt out.

packages/tui-py/python/tui/__init__.py

Lines changed: 42 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,25 @@
1-
"""High-level Python API for the `tui` PTY-backed terminal manager.
1+
"""High-level async API for the `tui` PTY-backed terminal manager.
22
33
Spawn child processes attached to real pseudo-terminals, drive them with
4-
keystrokes, and observe their VT100-rendered viewport. The public surface is:
4+
keystrokes, and observe their VT100-rendered viewport. The whole I/O surface is
5+
async: every method that touches the terminal is a coroutine you must `await`.
6+
7+
async with Tui("python", "-q") as t:
8+
await t.enter("print('hi')")
9+
snap = await t.wait_for("hi", timeout=2.0)
10+
11+
The awaitables are native asyncio coroutines bridged from Rust via
12+
pyo3-async-runtimes, with no thread-pool hop. The only synchronous surface is
13+
construction and the cached accessors: `id`, `command`, `args`, `size`,
14+
`is_alive`, `exit_code`. Everything else (`send`, `enter`, `read`, `viewport`,
15+
`text`, `snapshot`, `wait_for`, `resize`, `kill`, `close`) is a coroutine.
16+
17+
Every spawned terminal auto-shows in the web dashboard. The first `Tui(...)`
18+
binds a process-global producer, so running `nix run .#tui-dashboard` (it
19+
watches `socket_dir()`) renders this process's terminals with no explicit
20+
`tui.publish()`. Opt out by setting `IX_TUI_AUTOPUBLISH=0`.
21+
22+
The public surface:
523
624
Tui a single spawned process; the workhorse handle
725
Snapshot an immutable read-time view of one process
@@ -10,11 +28,6 @@
1028
StyledCell one viewport cell: character + VT100 attributes
1129
Color a cell color: None (default), int (palette), or (r, g, b)
1230
WaitTimeout raised by `Tui.wait_for(...)` when nothing matched in time
13-
14-
The interface is async-only. Every I/O method is a coroutine that returns a
15-
native asyncio-awaitable from the Rust side (via pyo3-async-runtimes); no
16-
thread-pool hop is involved. Construction and the shape accessors (`id`,
17-
`command`, `size`, `is_alive`, `exit_code`) are the only synchronous surface.
1831
"""
1932

2033
from __future__ import annotations
@@ -37,6 +50,7 @@
3750
StyledCell as StyledCell,
3851
TuiInstance as _RawTuiInstance,
3952
__version__,
53+
ensure_published as _raw_ensure_published,
4054
publish as _raw_publish,
4155
serve as _raw_serve,
4256
socket_dir as socket_dir,
@@ -194,6 +208,21 @@ def _build_predicate(pattern: Pattern) -> Callable[[Snapshot], bool]:
194208
return pattern
195209

196210

211+
# --------------------------------------------------------------------------- #
212+
# Auto-publish
213+
# --------------------------------------------------------------------------- #
214+
215+
def _ensure_autopublish() -> None:
216+
"""Bind the process-global dashboard producer once, on first `Tui(...)`.
217+
218+
Spawned terminals then appear in `nix run .#tui-dashboard` with no explicit
219+
`tui.publish()`. Idempotency (bind at most once per process) and the
220+
`IX_TUI_AUTOPUBLISH=0` opt-out both live in the Rust `ensure_published`, so
221+
this stays a thin call into it rather than re-implementing either guard here.
222+
"""
223+
_raw_ensure_published()
224+
225+
197226
# --------------------------------------------------------------------------- #
198227
# Tui
199228
# --------------------------------------------------------------------------- #
@@ -214,7 +243,11 @@ class Tui:
214243
spawned PTY; each I/O method returns a native asyncio coroutine bridged
215244
through pyo3-async-runtimes, with no thread-pool hop. Construction and the
216245
shape accessors (`id`, `command`, `args`, `size`, `is_alive`, `exit_code`)
217-
are the only synchronous surface.
246+
are the only synchronous surface; everything else is a coroutine to await.
247+
248+
The first `Tui(...)` auto-publishes this process to the web dashboard, so
249+
`nix run .#tui-dashboard` shows the terminal without an explicit
250+
`tui.publish()`. Set `IX_TUI_AUTOPUBLISH=0` to opt out.
218251
219252
`kill()` sends SIGKILL; `interrupt()` sends a cooperative Ctrl+C; `close()`
220253
force-kills and drops the terminal from `list_all()`. `async with` blocks
@@ -232,6 +265,7 @@ def __init__(
232265
cols: int | None = None,
233266
scrollback_lines: int | None = None,
234267
) -> None:
268+
_ensure_autopublish()
235269
self._raw = _RawTuiInstance(command, list(args), rows, cols, scrollback_lines)
236270

237271
@classmethod

packages/tui-py/python/tui/_tui.pyi

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,4 +107,5 @@ class Publisher:
107107
def __repr__(self) -> str: ...
108108

109109
def publish(path: str | None = ..., poll_ms: int = ...) -> Awaitable[Publisher]: ...
110+
def ensure_published(poll_ms: int = ...) -> None: ...
110111
def socket_dir() -> str: ...

packages/tui-py/src/lib.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ fn _tui(module: &Bound<'_, PyModule>) -> PyResult<()> {
2525
module.add_class::<publish::Publisher>()?;
2626
module.add_function(wrap_pyfunction!(dashboard::serve, module)?)?;
2727
module.add_function(wrap_pyfunction!(publish::publish, module)?)?;
28+
module.add_function(wrap_pyfunction!(publish::ensure_published, module)?)?;
2829
module.add_function(wrap_pyfunction!(publish::socket_dir, module)?)?;
2930
module.add("__version__", env!("CARGO_PKG_VERSION"))?;
3031
Ok(())

packages/tui-py/src/publish.rs

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,14 @@
66
//! process-wide handle to Python. The returned [`Publisher`] keeps the socket
77
//! alive until stopped or dropped. Both `publish` and `stop` are awaitable so
88
//! the binding surface is uniformly async.
9+
//!
10+
//! [`ensure_published`] is the synchronous twin that the high-level Python API
11+
//! calls on first `Tui(...)`: it binds one process-global producer so terminals
12+
//! show up in `tui-dashboard` with no explicit `tui.publish()`.
913
1014
use std::path::PathBuf;
15+
use std::sync::OnceLock;
16+
use std::sync::atomic::{AtomicBool, Ordering};
1117
use std::time::Duration;
1218

1319
use parking_lot::Mutex;
@@ -16,6 +22,25 @@ use pyo3_async_runtimes::tokio::future_into_py;
1622

1723
use crate::manager::global_manager;
1824

25+
/// The process-global producer bound by [`ensure_published`].
26+
///
27+
/// It outlives every Python handle so the dashboard keeps seeing this process's
28+
/// terminals for the life of the interpreter. The `OnceLock` makes the first
29+
/// call the binder; the inner `Mutex<Option<_>>` holds the live producer (or
30+
/// `None` once a bind was attempted and skipped or torn down).
31+
static AUTOPUBLISHER: OnceLock<Mutex<Option<tui::Publisher>>> = OnceLock::new();
32+
33+
/// Set once any producer is bound for this process, by either [`publish`] or
34+
/// [`ensure_published`]. Auto-publish checks it so an explicit
35+
/// `tui.publish(...)` (chosen for a custom socket path or poll interval) is not
36+
/// shadowed by a second process-global producer on the next `Tui(...)`, which
37+
/// would make the aggregator list every terminal twice.
38+
static PROCESS_PUBLISHED: AtomicBool = AtomicBool::new(false);
39+
40+
/// The env var that opts a process out of auto-publishing. The literal `"0"`
41+
/// disables it; any other value (or unset) leaves auto-publish on.
42+
const AUTOPUBLISH_OPT_OUT: &str = "IX_TUI_AUTOPUBLISH";
43+
1944
/// Handle to a running producer. Dropping it or awaiting `stop` stops the
2045
/// streaming loops and unlinks the socket.
2146
#[pyclass(module = "tui._tui")]
@@ -70,7 +95,18 @@ pub fn publish(py: Python<'_>, path: Option<String>, poll_ms: u64) -> PyResult<B
7095
let path = path.map_or_else(tui::socket_path, PathBuf::from);
7196
let manager = global_manager();
7297
future_into_py(py, async move {
98+
// An explicit publish supersedes auto-publish: stop the process-global
99+
// producer if one was already bound, so the process exposes exactly one
100+
// producer rather than a duplicate under a second id. `take()` releases
101+
// the lock before the await.
102+
let previous = AUTOPUBLISHER.get().and_then(|slot| slot.lock().take());
103+
if let Some(mut previous) = previous {
104+
previous.stop().await;
105+
}
73106
let publisher = tui::publish(&manager, path, Duration::from_millis(poll_ms)).await?;
107+
// Mark the process published so a later `Tui(...)` does not auto-bind a
108+
// second producer on top of this explicit one.
109+
PROCESS_PUBLISHED.store(true, Ordering::Release);
74110
Ok(Publisher {
75111
path: publisher.path().display().to_string(),
76112
producer: publisher.producer_id().to_owned(),
@@ -79,6 +115,53 @@ pub fn publish(py: Python<'_>, path: Option<String>, poll_ms: u64) -> PyResult<B
79115
})
80116
}
81117

118+
/// Bind one process-global producer if none is running yet.
119+
///
120+
/// Synchronous and idempotent: the first call binds the producer on the global
121+
/// manager's tokio runtime and stashes the handle in [`AUTOPUBLISHER`]; later
122+
/// calls are a no-op. A no-op too when `IX_TUI_AUTOPUBLISH=0`. The high-level
123+
/// `tui.Tui` calls this once on construction so spawned terminals appear in
124+
/// `tui-dashboard` without an explicit `tui.publish()`.
125+
///
126+
/// The bind future is driven to completion on the pyo3-async-runtimes tokio
127+
/// runtime under `py.detach` (releasing the GIL); `tui::publish` then runs its
128+
/// own poll and accept loops on the manager's runtime, so the producer survives
129+
/// past this call. A bind failure is swallowed: auto-publish is a convenience,
130+
/// not a hard dependency, and an explicit `await tui.publish()` still surfaces
131+
/// errors.
132+
#[pyfunction]
133+
#[pyo3(signature = (poll_ms=100))]
134+
pub fn ensure_published(py: Python<'_>, poll_ms: u64) {
135+
if std::env::var(AUTOPUBLISH_OPT_OUT).as_deref() == Ok("0") {
136+
return;
137+
}
138+
// An explicit `tui.publish(...)` already exposed this process; do not bind a
139+
// second producer on top of it.
140+
if PROCESS_PUBLISHED.load(Ordering::Acquire) {
141+
return;
142+
}
143+
144+
let slot = AUTOPUBLISHER.get_or_init(|| Mutex::new(None));
145+
let mut guard = slot.lock();
146+
if guard.is_some() {
147+
return;
148+
}
149+
150+
let manager = global_manager();
151+
let runtime = pyo3_async_runtimes::tokio::get_runtime();
152+
let publisher = py.detach(|| {
153+
runtime.block_on(tui::publish(
154+
&manager,
155+
tui::socket_path(),
156+
Duration::from_millis(poll_ms),
157+
))
158+
});
159+
if let Ok(publisher) = publisher {
160+
*guard = Some(publisher);
161+
PROCESS_PUBLISHED.store(true, Ordering::Release);
162+
}
163+
}
164+
82165
/// The discovery directory where producers expose sockets and the aggregator
83166
/// looks for them.
84167
#[pyfunction]

packages/tui/src/actor/decscusr.rs

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
//! Incremental sniffer for the `DECSCUSR` cursor-shape escape.
2+
//!
3+
//! vt100 parses and applies cursor *position* but does not model cursor
4+
//! *shape*, so the actor watches the same byte stream it feeds the parser and
5+
//! folds out the latest `DECSCUSR` sequence: `CSI Ps SP q`, i.e. `ESC [`, an
6+
//! optional decimal parameter, a space (`0x20`), then `q`. The scanner is fed
7+
//! the raw PTY reads in order, and a sequence split across two reads resumes
8+
//! from the carried state, so a child that emits the escape in pieces is still
9+
//! recognized.
10+
//!
11+
//! Only this one final byte (`q`) is matched. Any other intermediate or final
12+
//! byte abandons the in-progress sequence, so unrelated CSI sequences (colors,
13+
//! cursor moves) never produce a false shape change.
14+
15+
use crate::types::CursorShape;
16+
17+
/// Where the scanner is inside a candidate `CSI Ps SP q` sequence.
18+
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
19+
enum State {
20+
/// No escape in progress.
21+
#[default]
22+
Ground,
23+
/// Saw `ESC`, waiting for `[`.
24+
Escape,
25+
/// Inside `CSI`, accumulating the decimal parameter (and looking for the
26+
/// space that ends it).
27+
Params,
28+
/// Saw the space after the parameter, waiting for the final `q`.
29+
Intermediate,
30+
}
31+
32+
/// A byte-stream scanner that surfaces the most recent `DECSCUSR` shape.
33+
///
34+
/// One scanner lives per terminal actor. Feed it every PTY read with
35+
/// [`Scanner::feed`]; it returns `Some(shape)` for each `DECSCUSR` it completes
36+
/// so the caller can publish the latest one.
37+
#[derive(Debug, Default)]
38+
pub struct Scanner {
39+
state: State,
40+
/// The decimal parameter accumulated in [`State::Params`], capped so a
41+
/// hostile child cannot overflow it; real parameters are one or two digits.
42+
param: u16,
43+
}
44+
45+
const ESC: u8 = 0x1b;
46+
const CSI_OPEN: u8 = b'[';
47+
const SPACE: u8 = b' ';
48+
const FINAL: u8 = b'q';
49+
50+
impl Scanner {
51+
/// Scan `bytes`, returning the last completed `DECSCUSR` shape if any.
52+
///
53+
/// Returns the final shape in the buffer rather than every intermediate one
54+
/// because the actor only tracks the current shape; an earlier escape in
55+
/// the same read is immediately superseded.
56+
pub fn feed(&mut self, bytes: &[u8]) -> Option<CursorShape> {
57+
let mut latest = None;
58+
for &byte in bytes {
59+
if let Some(shape) = self.step(byte) {
60+
latest = Some(shape);
61+
}
62+
}
63+
latest
64+
}
65+
66+
fn step(&mut self, byte: u8) -> Option<CursorShape> {
67+
match self.state {
68+
State::Ground => {
69+
if byte == ESC {
70+
self.state = State::Escape;
71+
}
72+
None
73+
}
74+
State::Escape => {
75+
self.state = if byte == CSI_OPEN {
76+
self.param = 0;
77+
State::Params
78+
} else if byte == ESC {
79+
State::Escape
80+
} else {
81+
State::Ground
82+
};
83+
None
84+
}
85+
State::Params => {
86+
if byte.is_ascii_digit() {
87+
self.param = self
88+
.param
89+
.saturating_mul(10)
90+
.saturating_add(u16::from(byte - b'0'));
91+
} else if byte == SPACE {
92+
self.state = State::Intermediate;
93+
} else {
94+
// Any other byte (another intermediate, a different final,
95+
// or a stray ESC) ends this candidate.
96+
self.restart_on(byte);
97+
}
98+
None
99+
}
100+
State::Intermediate => {
101+
let shape = (byte == FINAL).then(|| CursorShape::from_decscusr(self.param));
102+
self.restart_on(byte);
103+
shape
104+
}
105+
}
106+
}
107+
108+
/// Reset to ground, but honor a fresh `ESC` so back-to-back sequences with
109+
/// no separator are not dropped.
110+
const fn restart_on(&mut self, byte: u8) {
111+
self.state = if byte == ESC { State::Escape } else { State::Ground };
112+
}
113+
}
114+
115+
#[cfg(test)]
116+
mod tests {
117+
use super::*;
118+
119+
#[test]
120+
fn maps_each_decscusr_parameter_to_a_shape() {
121+
// The boundary between the three shape buckets, including the unknown
122+
// fallback to block.
123+
let cases = [
124+
(b"\x1b[0 q".as_slice(), CursorShape::Block),
125+
(b"\x1b[2 q", CursorShape::Block),
126+
(b"\x1b[3 q", CursorShape::Underline),
127+
(b"\x1b[4 q", CursorShape::Underline),
128+
(b"\x1b[5 q", CursorShape::Bar),
129+
(b"\x1b[6 q", CursorShape::Bar),
130+
(b"\x1b[ q", CursorShape::Block),
131+
(b"\x1b[99 q", CursorShape::Block),
132+
];
133+
for (bytes, want) in cases {
134+
let mut scanner = Scanner::default();
135+
assert_eq!(scanner.feed(bytes), Some(want), "for {bytes:?}");
136+
}
137+
}
138+
139+
#[test]
140+
fn ignores_unrelated_csi_sequences() {
141+
// A color SGR and a cursor move must not be read as a shape change.
142+
let mut scanner = Scanner::default();
143+
assert_eq!(scanner.feed(b"\x1b[1;31mhi\x1b[2;5H"), None);
144+
}
145+
146+
#[test]
147+
fn resumes_a_sequence_split_across_feeds() {
148+
let mut scanner = Scanner::default();
149+
assert_eq!(scanner.feed(b"\x1b[5"), None);
150+
assert_eq!(scanner.feed(b" q"), Some(CursorShape::Bar));
151+
}
152+
153+
#[test]
154+
fn returns_the_last_shape_in_one_feed() {
155+
let mut scanner = Scanner::default();
156+
assert_eq!(
157+
scanner.feed(b"\x1b[3 q text \x1b[6 q"),
158+
Some(CursorShape::Bar)
159+
);
160+
}
161+
}

0 commit comments

Comments
 (0)