[shell] Flushed data from spawned process is not sent to JS unless it ends in a newline

[atdyer]

I am communicating with a spawned process via stdin and stdout using something like the following:

const cmd = Command.create('example-program');
cmd.stdout.on('data', (data) => console.log(data));
cmd.stderr.on('data', (data) => console.log(data));

const proc = await cmd.spawn();
proc.write("This is a message to the spawned process");

The program being spawned listens for input from stdin and writes some output to stdout in response. The issue I'm running into is that while the the output is flushed correctly by the spawned program, it is not terminated with a newline character. As a result, the JS above receives all output except for the last line, which it will receive as the first line of text in responsed to the subsequent proc.write() call.

To demonstrate, the following Rust code is a simple echo program:

use std::io;
use std::io::Write;

fn main() {
    io::stdout().write_all(b"Type something: ").expect("Error writing to stdout");
    io::stdout().flush().expect("Error flushing stdout");
    loop {
        let mut buffer = String::new();
        io::stdin().read_line(&mut buffer).expect("Error reading from stdin");
        io::stdout().write_all(b"You wrote: ").expect("Error writing to stdout");
        io::stdout().write_all(buffer.as_bytes()).expect("Error writing to stdout");
        io::stdout().write_all(b"Type something: ").expect("Error writing to stdout");
        io::stdout().flush().expect("Error flushing stdout");
    }
}

When executed from the command line, it behaves as follows:

Screen.Recording.2024-08-07.at.12.58.05.PM.mov

When attached to Tauri as a sidecar using the above code, the output in the console is as follows:

Screen.Recording.2024-08-07.at.1.00.25.PM.mov

Note specifically that the string "Type something:" is not written to the console until a newline character has been written to stdout.

[atdyer]

Okay I did a little digging through the code to try to debug this and found a workaround.

Command.create and Command.sidecar both accept an optional SpawnOptions parameter that has an optional env string field. Passing the string 'raw' to this field will tell Tauri to use a pipe that reads raw bytes rather than newline separated strings (see spawn_pipe_reader). You can see here where the 'raw' string is used to determine which pipe to set up; as far as I can tell this specific string and the resulting behavior are not yet in the documentation.

So to get the example above working, we can modify as follows:

const cmd = Command.create('example-program', [], { encoding: 'raw' });
const decoder = new TextDecoder();

cmd.stdout.on('data', (data) => {
  const bytes = new Uint8Array(data);
  const string = decoder.decode(bytes);
  console.log(string);
});
cmd.stderr.on('data', (data) => {
  const bytes = new Uint8Array(data);
  const string = decoder.decode(bytes);
  console.error(string);
});

const proc = await cmd.spawn();
proc.write("This is a message to the spawned process");

It might be worth considering whether waiting for newline characters is the correct approach when piping strings, since it is inconsistent with the behavior seen when using the same program from the terminal (as demonstrated in the initial issue). Otherwise, an option to pipe string data without waiting for newline characters could be a useful alternative. I'd happily implement this myself but I unfortunately don't have the Rust expertise.

I'd be happy to help with updating the documentation to reflect this option and the resulting behavior if it's not already planned/underway.

[amrbashir]

I always though Node.js childProcess.stdout.on('data', console.log) was triggered on new lines only, but seems like that's not the case https://nodejs.org/api/stream.html#event-data. It seems like it also doesn't support encoding and just returns bytes or a valid utf8 string.

@lucasfernog what do you think about removing the new line reading (and probably the encoding stuff)?

[lucasfernog]

I think the current behavior is fine as long as it's documented.