Skip to content

Commit eb11c07

Browse files
fix(python): echo what is typed at the >>> prompt
The REPL's move onto the blocking stdin syscall took it out from under the only thing that had been showing a person their own keystrokes: the shell, which echoes the characters it reads and hands a foreground child raw bytes on purpose. Nothing below the guest cooks a terminal — process.stdin only records setRawMode — so the reader of a line is what has to show it, and at a >>> prompt that reader is repl(). makeLineReader takes an echo sink and becomes the line discipline while it has one: each character as it arrives, DEL rubbing one out of the line as well as off the screen, and a control byte shown as ^X so an arrow key's ESC [ A cannot steer the cursor into output printed earlier. Not in installStdin, one layer down, which would echo getpass() too with no way for Python to stop it: Emscripten's tty reports ECHO already clear and accepts a tcsetattr it ignores, so getpass prints no warning and the password would be on the screen. input() therefore still shows nothing, which is recorded as a known gap. Echo goes to stderr rather than stdout (an echo is a write to the terminal, not part of what the process produces) and only under VV_TTY=1, so captured and piped runs are unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 4ac0dc9 commit eb11c07

7 files changed

Lines changed: 421 additions & 14 deletions

File tree

AGENTS.md

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2561,6 +2561,17 @@ Gotchas:
25612561
`sys.exit(pytest.main(...))` on every run. Note Pyodide's WebLoop *also* re-raises
25622562
SystemExit as a second, unhandled rejection — harmless in a browser, but Node aborts on
25632563
it, so headless harnesses must swallow it.
2564+
- **The interactive REPL is a second SystemExit path, and it echoes.** `exit()` typed at a
2565+
`>>>` comes back OUT of `code.InteractiveConsole.push` — CPython's `runcode` re-raises
2566+
`SystemExit` for the loop above it to act on — so `repl()` ends the session there, with
2567+
the code `terminationFromError` reads (it now names the ending: `exit`/`interrupt`/
2568+
`error`, so there is still one parser). A `KeyboardInterrupt` must NOT end it: name
2569+
printed, fresh top-level prompt. And the loop echoes what it reads, to **stderr** and only
2570+
under `VV_TTY=1`, because nothing below the guest cooks a terminal (see the shell/stdin
2571+
section). Do not move that echo into `installStdin`: it would cover `getpass()`, which
2572+
cannot turn it off — Emscripten's tty reports ECHO already clear and ignores `tcsetattr`,
2573+
so no warning is printed and the password is on the screen. `input()` echo is therefore
2574+
still missing, on purpose.
25642575
- **Django works, but only on WSGI**, and only with `DJANGO_ALLOW_ASYNC_UNSAFE=1` and
25652576
`tzdata`. Its ASGI path goes through `asgiref`, which starts a `ThreadPoolExecutor` per
25662577
request even for `async def` views — the existing `anyio` patch does not help, that is a
@@ -3492,7 +3503,13 @@ runtime's `drainStdin` normalizes strings vs bytes to a Buffer). The host termin
34923503
The interactive line editor (echo, backspace, Ctrl+C→SIGINT the whole foreground
34933504
job — every stage of a pipeline via `currentKill`, with keystrokes forwarded to
34943505
the pipeline's first stage) lives in the `sh` coreutil, not in a TTY line
3495-
discipline — there's nothing cooked below it. It also does **↑/↓ history recall**
3506+
discipline — there's nothing cooked below it. **The consequence for any other
3507+
program that reads lines: whoever reads a line has to show it.** `sh` echoes only
3508+
what *it* reads and forwards raw keystrokes to a foreground child on purpose, and
3509+
`process.stdin.setRawMode` merely records the mode, so a child that reads and does
3510+
not echo (which is what the Python REPL did) leaves the person typing blind. Echo
3511+
belongs in that program's own read loop — see `repl()` in `builtins/python.js`,
3512+
which echoes to stderr under `VV_TTY` for the reasons in the Python section. It also does **↑/↓ history recall**
34963513
(a module-scoped `commandHistory` array shared with the `history` builtin, which
34973514
lists it bash-style 1-indexed) and **Tab completion** (first token → builtins +
34983515
PATH programs; later tokens → the VFS, dirs suffixed `/`; unique match inserts +
@@ -4196,6 +4213,26 @@ Two smaller ones from the same batch, both worth knowing before you touch these
41964213
the change.** The blocking stdin syscall worked on the first try; what it broke was the REPL,
41974214
which had been reading the flowing stream perfectly happily until an `input()` at a `>>>` prompt
41984215
could take stdin away from it.
4216+
- **Moving a reader moves it out from under whatever was quietly serving it.** The same move cost
4217+
the REPL its echo, and it took a user session to notice: nothing the person typed appeared, so
4218+
the only thing on screen was the output of the `print()` calls they were typing blind. Nothing
4219+
in this system echoes on a program's behalf — xterm does not, `process.stdin.setRawMode` only
4220+
records the mode, and `sh` echoes what *it* reads and forwards raw bytes to a foreground child
4221+
deliberately. Ask what the layer you are leaving was doing for you besides the thing you came
4222+
for.
4223+
- **The layer that is easiest to echo from is the one that leaks the password.** Echoing inside
4224+
`installStdin` — the interpreter's own stdin callback — would have covered `input()` too, and
4225+
`getpass()` could not have switched it off: Emscripten's tty answers `tcgetattr` with ECHO
4226+
already clear and accepts a `tcsetattr` it ignores, so getpass prints none of its "cannot
4227+
control the terminal" warnings and reads the password onto the screen. Both facts were measured
4228+
against the real interpreter before the layer was chosen. Echo therefore sits in the REPL loop,
4229+
which knows its line is source meant to be seen, and `input()` echo stays an open gap.
4230+
- **A `SystemExit` that CPython re-raises is a message to the loop above it.**
4231+
`code.InteractiveInterpreter.runcode` re-raises it instead of reporting it *because* the driving
4232+
loop is supposed to end the session; a `catch` that treats every exception as printable text
4233+
turns `exit()` into a traceback plus another prompt. When you find yourself parsing one, check
4234+
whether the file already has a parser — `terminationFromError` did, and now reports which of
4235+
`exit`/`interrupt`/`error` it found so the REPL can act without a second copy of the rules.
41994236
- **A skip-list entry is a recorded reason, not a rule.** `python` was excluded from debug targets
42004237
because instrumenting a Node shim debugs the shim. Deleting the entry to add Python debugging
42014238
would have done exactly the thing the comment warned about — and done it silently, offering

ARCHITECTURE.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1792,6 +1792,31 @@ flowing stream meant the first `input()` at a `>>>` prompt took stdin away from
17921792
The cost is that the process's event loop does not turn while a prompt waits, which is what
17931793
CPython does at a `>>>` as well.
17941794

1795+
**The REPL also echoes, because in this system the reader of a line is what shows it.** There
1796+
is no cooked-mode line discipline below the guest — `process.stdin.setRawMode` records the
1797+
mode and nothing more, and `sh` echoes the keystrokes *it* reads while handing a foreground
1798+
child raw bytes so the child can drive its own display. So `sh` echoes at its prompt and
1799+
`repl()` echoes at a `>>>`, through the same `makeLineReader`, which with an echo sink also
1800+
applies what a canonical terminal would: DEL leaves the character neither in the line nor on
1801+
the screen, and a control byte is shown as `^X` (echo an arrow key's `ESC [ A` verbatim and it
1802+
steers the terminal's cursor into output printed earlier). It writes to **stderr** — an echo
1803+
goes to the terminal, not into this process's stdout, which may be a pipe — and only when
1804+
`VV_TTY=1` says a terminal is attached, so captured and scripted runs are unchanged.
1805+
Deliberately *not* in `installStdin`, one layer down: that is every read the interpreter makes,
1806+
`getpass()` included, and Python cannot opt out here because Emscripten's tty reports ECHO
1807+
already clear and accepts a `tcsetattr` it ignores — getpass would print no warning and the
1808+
password would be on screen. The cost of that choice is that `input()` typed at a prompt still
1809+
shows nothing.
1810+
1811+
**`exit()` ends the session with its exit code.** `exit`/`quit`/`sys.exit` raise `SystemExit`,
1812+
and `code.InteractiveInterpreter.runcode` re-raises it rather than reporting it, for the loop
1813+
above to act on; the loop reads it with the same `terminationFromError` a script's top-level
1814+
exception goes through (it names the ending — `exit`, `interrupt`, `error` — so the REPL needs
1815+
no SystemExit parser of its own), so `exit()`/`exit(0)` leave 0, `exit(3)` leaves 3 and
1816+
`exit("bye")` prints `bye` to stderr and leaves 1. Ctrl-D is a newline and 0. A
1817+
`KeyboardInterrupt` is the one that must *not* end it: it prints its name and returns to a
1818+
fresh top-level prompt, abandoning any half-typed block.
1819+
17951820
**The first interpreter of a session is snapshotted, and the rest resume from it.** Booting
17961821
CPython costs ~1.8s and this runtime pays it per command, which was the single biggest thing
17971822
wrong with Python here. Pyodide can serialise a just-booted interpreter's linear memory and

packages/runtime/builtins/python.js

Lines changed: 93 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -143,31 +143,37 @@ export function flushStreams(pyodide) {
143143
// integer, prints just that argument and exits 1. We used to dump the whole
144144
// WASM traceback for every sys.exit(), which meant a clean `sys.exit(0)` — what
145145
// `python -m pytest` does on every green run — looked like a crash.
146+
//
147+
// `kind` names WHICH of the three this was, because the interactive REPL has to
148+
// tell them apart and must not grow a second SystemExit parser to do it: an
149+
// `exit()` typed at a `>>>` ends the session with this code, a Ctrl-C returns to
150+
// a fresh prompt, and anything else is one statement's error in a session that
151+
// carries on. A script needs none of that distinction and ignores the field.
146152
export function terminationFromError(e) {
147153
const msg = (e && e.message) || String(e);
148154
const last = msg.trimEnd().split("\n").pop().trim();
149155
// Ctrl-C. CPython prints the traceback like any other exception and exits
150156
// 128+SIGINT, and a shell that reports 130 is how a script author tells an
151157
// interrupted run from a failed one.
152158
if ((e && e.type === "KeyboardInterrupt") || /^KeyboardInterrupt\b/.test(last)) {
153-
return { code: 130, report: msg };
159+
return { kind: "interrupt", code: 130, report: msg };
154160
}
155161
const isExit = (e && e.type === "SystemExit") || /^SystemExit\b/.test(last);
156-
if (!isExit) return { code: 1, report: msg };
162+
if (!isExit) return { kind: "error", code: 1, report: msg };
157163
const m = /^SystemExit:\s*([\s\S]*)$/.exec(last);
158164
const value = m ? m[1].trim() : "";
159-
if (!value || value === "None") return { code: 0, report: "" }; // bare sys.exit()
160-
if (/^-?\d+$/.test(value)) return { code: Number(value) | 0, report: "" };
165+
if (!value || value === "None") return { kind: "exit", code: 0, report: "" }; // bare sys.exit()
166+
if (/^-?\d+$/.test(value)) return { kind: "exit", code: Number(value) | 0, report: "" };
161167
// Bools ARE ints in Python: sys.exit(True) exits 1 and sys.exit(False) exits
162168
// 0, printing nothing either way. The traceback spells both as
163169
// "SystemExit: True"/"False" — indistinguishable from sys.exit("True"), so
164170
// the message is a lossy channel and this picks the far likelier reading.
165171
// `sys.exit(not ok)` is a common idiom; exiting with the literal string
166172
// "False" is not. It is also the reading that keeps a *successful* run
167173
// reporting success, which the string reading got backwards.
168-
if (value === "True") return { code: 1, report: "" };
169-
if (value === "False") return { code: 0, report: "" };
170-
return { code: 1, report: value }; // sys.exit("message")
174+
if (value === "True") return { kind: "exit", code: 1, report: "" };
175+
if (value === "False") return { kind: "exit", code: 0, report: "" };
176+
return { kind: "exit", code: 1, report: value }; // sys.exit("message")
171177
}
172178

173179
// Undo one consequence of our own Node masquerade: it switches Python's HTTP off.
@@ -911,8 +917,13 @@ export function dataPackagesFor(source) {
911917
*
912918
* Returns null at end of input — but a part-typed line without its newline is
913919
* still a line, as it is in CPython when you type something and press Ctrl-D.
920+
*
921+
* With an `echo` sink this also becomes the line discipline a canonical-mode
922+
* terminal would provide, because nothing under it does: it shows each character
923+
* as it arrives and lets DEL rub one out. Only the interactive REPL passes one —
924+
* see repl(), which also decides when echoing is right at all.
914925
*/
915-
export function makeLineReader(read) {
926+
export function makeLineReader(read, echo) {
916927
const readChunk = read || (() => (globalThis.__ocReadStdin ? globalThis.__ocReadStdin() : null));
917928
let buf = "";
918929
return () => {
@@ -932,7 +943,33 @@ export function makeLineReader(read) {
932943
}
933944
return null;
934945
}
935-
buf += chunk;
946+
if (!echo) {
947+
buf += chunk;
948+
continue;
949+
}
950+
for (const ch of chunk) {
951+
if (ch === "\x7f" || ch === "\b") {
952+
// Erase within the line being typed only: a DEL at a fresh prompt must
953+
// rub out neither the prompt that invited it nor a line already queued
954+
// behind this one (a paste arrives as one chunk of several lines).
955+
if (buf && !buf.endsWith("\n")) {
956+
buf = buf.slice(0, -1);
957+
echo("\b \b");
958+
}
959+
continue;
960+
}
961+
buf += ch;
962+
// Control characters go on the screen as ^X, which is what a terminal
963+
// with ECHOCTL does and is here for a sharper reason than fidelity: an
964+
// arrow key arrives as the three bytes ESC [ A, and echoing those
965+
// verbatim would move the terminal's cursor into output printed earlier
966+
// and type the rest of the line there. Newline and tab are excluded
967+
// because their effect on the screen is the point of them.
968+
const code = ch.charCodeAt(0);
969+
echo(code < 0x20 && ch !== "\n" && ch !== "\r" && ch !== "\t"
970+
? "^" + String.fromCharCode(code + 64)
971+
: ch);
972+
}
936973
}
937974
};
938975
}
@@ -2278,8 +2315,10 @@ json.dumps({
22782315
const console_ = pyodide.globals.get("_vv_console");
22792316

22802317
let more = false;
2318+
let ended = false;
22812319
const prompt = () => process.stdout.write(more ? "... " : ">>> ");
22822320
const finish = (codeVal) => {
2321+
ended = true;
22832322
try {
22842323
console_.destroy && console_.destroy();
22852324
} catch {
@@ -2292,6 +2331,25 @@ json.dumps({
22922331
// InteractiveConsole.push returns True when more input is needed.
22932332
more = !!withInterruptsSync(() => console_.push(line));
22942333
} catch (e) {
2334+
const t = terminationFromError(e);
2335+
// exit(), quit(), sys.exit(): CPython's InteractiveInterpreter.runcode
2336+
// re-raises SystemExit instead of reporting it (Lib/code.py) precisely
2337+
// so that the loop driving it can end the session — and this is that
2338+
// loop. Treated as a printable error, as it used to be, `exit()`
2339+
// answered with a traceback and a fresh prompt: a crash report for
2340+
// the one line every user of a REPL knows how to type.
2341+
//
2342+
// The code and the message are the same reading a script's top-level
2343+
// SystemExit gets, from the same parser: exit()/exit(0) leave 0,
2344+
// exit(3) leaves 3, exit("bye") prints bye and leaves 1.
2345+
if (t.kind === "exit") {
2346+
flushStreams(pyodide);
2347+
if (t.report) {
2348+
process.stderr.write(t.report.endsWith("\n") ? t.report : t.report + "\n");
2349+
}
2350+
finish(t.code);
2351+
return;
2352+
}
22952353
// Ctrl-C during a statement. CPython prints the name alone and gives
22962354
// a fresh top-level prompt, abandoning any half-typed block — the
22972355
// session survives, which is the whole point of interrupting it.
@@ -2322,7 +2380,29 @@ json.dumps({
23222380
//
23232381
// The cost is that this process's event loop does not turn while the
23242382
// prompt waits, which is what CPython does at a `>>>` too.
2325-
const readLine = makeLineReader();
2383+
//
2384+
// And it echoes, because in this system nobody else can. On a real
2385+
// terminal the keystroke a person types is shown by the line discipline
2386+
// in the kernel's tty layer, and there is none here: process.stdin
2387+
// records setRawMode and nothing more (packages/runtime/index.js), and
2388+
// the shell hands its foreground child raw keystrokes without echoing
2389+
// them (coreutils.js) exactly so the child can drive its own display.
2390+
// So the reader of a line is the thing that must show it, which is why
2391+
// the shell echoes at its own prompt and why this loop must at a `>>>`.
2392+
//
2393+
// Not one layer lower, in installStdin: that would echo every read the
2394+
// interpreter makes, including `getpass()`, and Python could not turn it
2395+
// off — Emscripten's tty answers tcgetattr with ECHO already clear and
2396+
// accepts a tcsetattr it then ignores, so getpass believes echo is off,
2397+
// prints no warning, and the password would go up on the screen.
2398+
//
2399+
// To stderr, not stdout, because echo is a write to the terminal and not
2400+
// part of what this process produces: `cat script.py | python > out` must
2401+
// not find the typed source in out. And only with a terminal attached
2402+
// (VV_TTY, the shell's own marker — the same one `ls` colours on), so a
2403+
// captured or scripted run stays byte-for-byte what it was.
2404+
const echo = process.env.VV_TTY === "1" ? (text) => process.stderr.write(text) : null;
2405+
const readLine = makeLineReader(null, echo);
23262406

23272407
// One statement per macrotask rather than a `while` loop, so timers and
23282408
// the process's own exit path get their turn between lines.
@@ -2334,6 +2414,9 @@ json.dumps({
23342414
return;
23352415
}
23362416
feed(line);
2417+
// An `exit()` on that line ended the session. Reading again would park
2418+
// this process on a stdin nobody is going to type into.
2419+
if (ended) return;
23372420
setTimeout(step, 0);
23382421
};
23392422
prompt();

0 commit comments

Comments
 (0)