Question: share a single log thread between multiple worker threads

[hexchain]

It seems that if a pino logger with asynchronous transport is initialized in each threads created, each transport will create a separate thread. My understanding is that this is a bit wasteful, and probably going to cause issues if they are all writing to the same file. Is it possible to let the whole process share a single log thread instead?

[JulianVallee]

You can probably use (multistream)[https://getpino.io/#/docs/api?id=pino-multistream], afaik this allows multiple logging streams in the same main thread

[hexchain]

Thanks for the suggestion, but I don't think that is what I wanted. An example might make it clear:

import process from 'node:process';
import { Worker, isMainThread, threadId } from 'node:worker_threads';
import { setTimeout } from 'node:timers/promises';
import pino from 'pino';

process.on('worker', (w) => {
  console.log('new thread created from thread', threadId);
});

if (isMainThread) {
  for (let i = 0; i < 5; i++) {
    new Worker(import.meta.filename);
  }
} else {
  const logger = pino({
    transport: {
      target: 'pino-pretty',
      options: {
        sync: false
      }
    }
  });
  logger.info('hello world from thread %d', threadId);
}

Running this script produces the following output:

new thread created from thread 0
new thread created from thread 0
new thread created from thread 0
new thread created from thread 0
new thread created from thread 0
new thread created from thread 3
new thread created from thread 4
new thread created from thread 5
new thread created from thread 2
new thread created from thread 6
[00:08:51.156] INFO (708322): hello world from thread 3
[00:08:51.168] INFO (708322): hello world from thread 4
[00:08:51.204] INFO (708322): hello world from thread 2
[00:08:51.222] INFO (708322): hello world from thread 6
[00:08:51.194] INFO (708322): hello world from thread 5

The main thread (id=0) created 5 worker threads, and in each worker thread, the pino instance created one extra thread for the logger. In the end, there are 5 logger transport threads. This is a bit wasteful, and if all of them writes to the same file, logs from different threads might interleave and corrupt the file.