Read child stdout pipe by chunks rather than by lines - #214
Conversation
| | _ -> Lwt.return_unit) | ||
|
|
||
| let read_exn ic = | ||
| match%lwt Lwt_io.read ~count:1024 ic with |
There was a problem hiding this comment.
impl is ok but choice of 1kb feels strange - why is it better than reading by lines?
can do Lwt_io.read_char and accumulate into buffer which is sent by timer,
There was a problem hiding this comment.
From the issue:
in a nutshell print_string("one\ntwo\nthree"); flush stdout will print only one and two. Reason is, the kernel reads stdout from REPL process line by line.
This PR replaces it reading by "chunks", arbitrary chosen at 1KB limit. If stdout has less than 1KB data, this will be read and sent to jupyter too. Tbh, 1KB only to protect from user mistakes, since I doubt somebody will actually output KBs of text to Jupyter out. We rely on Lwt read to perform heavy lifting which might not be guaranteed by specs, potentially would need to replace with reading byte by byte in the future in-house.
2 participants
This PR aims to address the long standing issue #162
The previous logic was to read the stdout from the REPL pipe line by line.
The new proposed solution aims to read by chunks, at most 1KB size.
Tested locally with various edge cases directly in the notebook.