Pasting in new REPL is slow

[natedogith1]

Bug report

Bug description:

Pasting large blocks of text in the new REPL is slow. Tested by pasting the following string (3035 characters):

test value

"""1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"""

When using the old repl (started via $env:PYTHON_BASIC_REPL="true"; python in powershell), pasting the string is near immediate. When using the new repl (started via python in powershell), pasting the string takes 15 seconds.

The closed issue #119517 also mentioned some pasting performance issues, though that was for Linux with significantly more characters.

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Linked PRs

• gh-130328: Faster Windows new REPL paste (DO NOT MERGE) #130677

[StanFromIreland]

Possibly a windows issue, something to do with powershell? On linux (main branch build) it pastes as expected, instantly.

cc windows expert @zooba

[zooba]

It's no good looking at me, I know nothing about the new repl (other than that I've suggested a couple of times that it should stop using ctypes and use native code instead, which could be the cause of any performance issue).

@pablogsal @lysnikolaou @ambv are code owners for the REPL.

[natedogith1]

I also see this when python is started from the win-r run dialog.

Tested some other environments, using the first 604 lines of this copy of Frankenstein (32,003 characters) (same as #119517 did). WSL old REPL takes ~2 seconds. WSL new REPL takes ~4 seconds, Powershell old REPL takes ~1 second. Powershell new REPL takes ~55 seconds for just the first 25 lines to appear; when pasting only the first 25 lines (659 characters) it takes ~2 seconds.

[devdanzin]

I think it's caused by the Windows console reading one character at a time from input:

cpython/Lib/_pyrepl/windows_console.py

Lines 386 to 389 in 65b339c

	rec = INPUT_RECORD()
	read = DWORD()
	if not ReadConsoleInput(InHandle, rec, 1, read):
	raise WinError(GetLastError())

But I don't know enough about ctypes or Windows API to know how to increase that value, naively using larger ints results in errors.

cc @DinoV, who implemented the Windows console.

[devdanzin]

I was able to make ReadConsoleInput read 1024 chars at a time and also reduced self.console.wait() in Reader.handle1 from 100 to 0.1. That reduced time to paste the content from first post from 10s to 6s, which isn't a significant improvement given the numbers from WSL and basic REPL.

[devdanzin]

#124119 seems to improve things, but nowhere near WSL numbers.

Edit: With that PR, time to past the sample from first post goes from 10s to 2.5s. Reading 32 chars of input instead of 1 and reducing Reader.console.wait from 100 to 10 further reduces time to 1.7s. So we can paste 5x faster than current version, which pales compared to the near instant paste in WSL.

[devdanzin]

Can you check whether devdanzin#1 speeds up pasting significantly for you?

It's a quick and dirty POC built on top of #124119 that:

• Batches event processing;
• Reads 32 chars from input instead of 1;
• Enables virtual terminal (from the mentioned PR);
• Reduces Reader.console.wait() from 100 to 10.

It fails some tests and the code is ugly, but if it makes things better we can try to improve on this.

[devdanzin]

Please test the #130677 PR to check whether it significantly improves pasting time for you.