Dynamic script path passed to `new Worker()` causes Next.js build to crash

[tom-sherman]

I'm opening this issue as more of a PSA than anything else, if folks stumble here when encountering errors in Next.js it may be useful. It also may be useul to the pino maintainers if they ever want to better support Next.js apps.

Reproduction: https://github.com/tom-sherman/next-turbopack-thread-stream-compile-bug

Running pnpm build produces the error:

Parsing ecmascript source code failed
> 1 | MIT License
    |     ^^^^^^^
  2 |
  3 | Copyright (c) 2021 Matteo Collina
  4 |

Expected ';', '}' or <eof>

Import trace:
  Server Component:
    ./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/LICENSE
    ./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/index.js
    ./app/page.tsx


./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/test/syntax-error.mjs:2:7
Parsing ecmascript source code failed
  1 | // this is a syntax error
> 2 | import
    |       ^
  3 |

Expected 'from', got '<eof>'

Import trace:
  Server Component:
    ./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/test/syntax-error.mjs
    ./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/index.js
    ./app/page.tsx


./node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/test
Module not found: Can't resolve './ROOT/node_modules/.pnpm/thread-stream@4.0.0/node_modules/thread-stream/test/close-on-gc.js'
server relative imports are not implemented yet. Please try an import relative to the file you are importing from.


https://nextjs.org/docs/messages/module-not-found

<< many more similar errors >>

The problem code in thread-stream is here:

thread-stream/index.js

Lines 52 to 55 in 541a162

	const bundlerOverrides = '__bundlerPathsOverrides' in globalThis ? globalThis.__bundlerPathsOverrides : {}
	const toExecute = bundlerOverrides['thread-stream-worker'] || join(__dirname, 'lib', 'worker.js')
	const worker = new Worker(toExecute, {

Next.js performs a process of "tracing" to figure out which files to include in the server bundle. It recently added support for tracing new Worker() in vercel/next.js#85734, alongside it's support for require(). Next.js (Turbopack and Webpack mode) is only able to accurately trace require() and new Worker() calls when the module ID is static (or close to static), which the code inside of thread-stream definitely isn't. In other words, the module ID passed to new Worker() is too dynamic for Next.js to be able to trace.

Workaround for Next.js users

A workaround for Next.js users is to add thread-stream to serverExternalPackages in the Next.js config.

Note that as of Next.js 16.0, in the event that thread-stream is a transitive dependency, users will also need to make sure they install the exact version of it as a direct dependency for Next.js to properly bundle. This will be fixed in Next.js 16.1 which lands better support for transitive serverExternalPackages (see vercel/next.js#86375).

There is an example fix in one of my repos here: frontpagefyi/frontpage#294

Suggested fix

I believe all of this complexity would be avoided if thread-stream made the new Worker() call static but I'm not sure how feasible that is exactly! Again, I'm writing this issue more of a PSA than anything so feel free to close if it's unactionable for you folks.

[mcollina]

I believe all of this complexity would be avoided if thread-stream made the new Worker() call static but I'm not sure how feasible that is exactly! Again, I'm writing this issue more of a PSA than anything so feel free to close if it's unactionable for you folks.

So you mean to remove the webpack bundling support logic? That might actually not be a bad idea, but it would require a new version of pino.

Having a fixed path would help?

[tom-sherman]

Yes I think a fixed path would resolve this as next.js would then be able to trace worker.js 👍

[mcollina]

Would you like to send a Pull Request to address this issue? Remember to add unit tests.