[Bug]: JSON reporter silently produces 0-byte file when merged report exceeds V8 max string length

[sheikhmaamir]

Version

1.56.1

Steps to reproduce

Run playwright merge-reports --reporter json against a large set of blob reports — specifically one where the merged dataset would serialize to more than ~512 MB of JSON.

In our case:

• ~2,900 tests across 6 shards
• ~130 failing tests, each carrying stack traces, attachments (screenshots, traces), and stdout/stderr captures
• Total blob report data: ~2 GB
• Resulting JSON (if it could be serialized) would be ~2 GB

npx playwright merge-reports --reporter json ./all-blob-reports > playwright-report/report.json


### Expected behavior

Either:

- The JSON reporter produces a valid JSON file (e.g. via streaming serialization), OR

- The merge command fails loudly with a clear error indicating the output couldn't be generated and a non-zero exit code, so callers can detect the failure.

### Actual behavior

Inside the JSON reporter, JSON.stringify(report, undefined, 2) throws RangeError: Invalid string length because the result would exceed V8's maximum string length (~512 MB / 2²⁹ - 24 characters on 64-bit Node).

Playwright catches this error inside the reporter's onEnd and logs it as a warning to stderr. The merge command then exits 0 (success). If the caller used a shell redirect (>), a 0-byte file is left behind. Downstream tooling that tries to parse the file fails with Unexpected end of JSON input, with no clear connection back to the original error.

The warning is easy to miss in CI environments where many other warnings stream past.

Stack trace observed in our CI:


Error in reporter RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at outputReport (node_modules/playwright/lib/reporters/json.js:229:29)
    at JSONReporter.onEnd (node_modules/playwright/lib/reporters/json.js:61:11)
    at Multiplexer.onEnd (node_modules/playwright/lib/reporters/multiplexer.js:57:31)
    at TeleReporterReceiver._onEnd (node_modules/playwright/lib/isomorphic/teleReceiver.js:186:33)
    at dispatchEvents (node_modules/playwright/lib/reporters/merge.js:69:22)
    at createMergedReport (node_modules/playwright/lib/reporters/merge.js:85:9)
JSON report generated successfully    ← misleading — merge-reports exited 0 despite the error

### Additional context

Additional context
This appears similar to issue #37719 (RangeError in junit reporter if test has too many logs), where a different JS engine limit (call stack size) silently broke a reporter on large data. That issue was fixed by exiting the process when the limit is hit. A similar approach in the JSON reporter would convert this silent failure into a visible one without requiring a streaming serialization rewrite.

Suggested minimum fix: when outputReport catches an exception from JSON.stringify, surface it rather than swallowing it as a warning, and have the merge command exit non-zero.

Workaround: we now use a custom Playwright reporter that streams the JSON output to disk incrementally and emits only the fields our downstream tooling needs. Happy to share if useful, but it intentionally drops data (attachment bodies, full stack traces) that the built-in reporter preserves.





### Environment

```shell
System:
    OS: macOS 26.3.1
    CPU: (12) arm64 Apple M4 Pro
    Memory: 262.23 MB / 24.00 GB
  Binaries:
    Node: 22.14.0 - /Users/xxx/.nvm/versions/node/v22.14.0/bin/node
    npm: 10.9.2 - /Users/xxx/.nvm/versions/node/v22.14.0/bin/npm
    pnpm: 10.17.1 - /Users/xxx/.nvm/versions/node/v22.14.0/bin/pnpm
  IDEs:
    Claude Code: 2.1.145 - /Users/xxx/.local/bin/claude
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.56.1 => 1.56.1 
    playwright-lighthouse: ^4.0.0 => 4.0.0 
    playwright-qase-reporter: ^2.1.0 => 2.1.0

[sheikhmaamir]

Thanks @Skn0tt for picking this up and @aydinomer00 for the PR!
Happy to test the fix against our actual data when it lands — we have reproducible blob reports that trigger the issue at ~2,900 tests / 130 failures.

One note for context: even with the "fail loudly" fix, anyone with our data scale will need to do something at the reporter level to actually produce a usable JSON. We worked around it by writing a custom slim streaming reporter that drops attachment bodies and stack traces, producing a ~1 MB output instead of the ~2 GB the default reporter would generate.

If a streaming JSON reporter is on the longer-term roadmap, happy to share what i built and contribute back if it'd help shape that work.

[Skn0tt]

Hi @sheikhmaamir! I think your approach of writing a custom reporter is the right one and i'm not interested in teaching our JSON reporter streaming. Users with JSON > 512mb probably have custom enough needs that they should want to write their own reporter, and we want to make that decision easy :) The thing I want to fix is that our merge reporter needs to fail loudly.

[Skn0tt]

There's two separate but related issues going on:

1. Errors thrown from reporters don't affect the exit code and are only visible from logs; both with and without merge-reports.
2. merge-reports ignores onEnd return value and does not mirror exit codes behaviour.

Both seem wrong, but touching it is a breaking change, @dgozman let's discuss this.

[7rulnik]

By the way there is https://github.com/discoveryjs/json-ext that could help with large JSONs