Refactor reporter interface to support streamable output for immediate test feedback

[Lojhan]

The current implementation of the runTestFile function and its associated reporter interface only displays test feedback after the entire test file process has finished. All stdout and stderr are buffered into a variable and processed only on the close or error events.

This behavior limits the immediate feedback that could be displayed in a streamable output, which is crucial for a good developer experience, especially with long-running tests or verbose logging.

culprit Behavior (in runTestFile.ts)
The root of this behavior is in how the child process output is handled.

A single output string variable is initialized.

All data from stdout and stderr is appended to this variable via the stdOut function.

let output = '';

const stdOut = (data: Buffer): void => {
    output += String(data);
};

// ...

    child.stdout!.on('data', stdOut);
    child.stderr!.on('data', stdOut);

This entire output buffer is only parsed and sent to the reporter after the child process exits in the 'close' event handler.

    child.on('close', async (code) => {
      // ...
      if (showLogs) {
        const parsedOutputs = parserOutput({
          output, // <-- The entire buffer is used here
          result,
        })?.join('\n');

        // ...

        reporter.onFileResult({
            // ...
          output: parsedOutputs, // <-- And sent to the reporter all at once
        });
      }
    // ...
    });

It may be beneficial to refactor this to provide immediate feedback to the user by processing the output as it arrives.

This could be achieved in one of two ways:

Implement a Writable Stream: The reporter interface could expose a WritableStream. The child.stdout and child.stderr could then be piped directly to this stream, allowing the reporter to process and display data in real-time.

New Reporter Method: Introduce a new method to the reporter interface, such as reporter.onFileChunk(chunk) or reporter.onFileLog(line). The 'data' event listener would then call this method directly instead of buffering into the output variable.

The goal is to move from "report on completion" to "report as it happens," significantly improving the tool's perceived performance and usability.

[wellwelwel]

The tests don't have a strict order of execution and can vary depending on the user's environment, whether it's Windows or Linux, whether it's case sensitive, etc.

The current approach helps to deal more easily with outputs and indentations with custom reporters (regardless of the execution order), especially when execution is not sequential, where outputs would mix among themselves, forcing output groups (e.g., headers or footers) to identify which file a specific piece of output came from, increasing both the complexity of reading and the size of the output for the end user.