feat(cli): support compile and extract with multi thread

[timofei-iatsenko]

Description

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Examples update

Fixes # (issue)

Checklist

• I have read the CONTRIBUTING and CODE_OF_CONDUCT docs
• I have added tests that prove my fix is effective or that my feature works
• I have added the necessary documentation (if appropriate)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Updated (UTC)
js-lingui	Ready	Preview	Aug 19, 2025 8:41am

[timofei-iatsenko]

@Zxilly changes from your PR:

1. I reverted multithreading changes for experemental extractor. The splitting work for worker pool was done in in the incorrect place. So instead of parallelezing work between workers, it actually create many parallel pools, with one worker only. This need more effort to make it work correctly, i would like to focus more on the stable extractor / compile first and then optimize new extractor.
2. I added support for custom extractors as well, instead of passing a config as a parameter to a worker, i'm re-creating the config in every of them. I also updated jiti (a lib used to read esm/ts/js configs on the fly in nodejs) version to make sure that it using a cache for config transpilation, also i'm passing a resolved config path to a worker, to avoid extra config crawling.
3. I simplified the code, and made it more aligned to the non-multithreading version. Instead of passing file content to the worker, worker reading fs by itself.

I saw you added some rate limit to readFile util, i'm not sure why it's needed, could you elaborate?

I'm also going to review the compile approach, probably some changes would be done there as well. I'll keep you posted.

[Zxilly]

The rate limit was because I tried to apply my modified CLI in our production environment, and the process opened too many files, exceeding the operating system's file descriptor limit, as we have over 4,000 PO files.

[codecov]

Codecov Report

❌ Patch coverage is 74.04580% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.36%. Comparing base (6bb8983) to head (6e73172).
⚠️ Report is 217 commits behind head on main.

Files with missing lines	Patch %	Lines
packages/cli/src/workers/extractWorker.ts	0.00%	7 Missing ⚠️
packages/cli/src/api/catalog/extractFromFiles.ts	83.33%	2 Missing and 4 partials ⚠️
packages/cli/src/api/utils.ts	68.42%	3 Missing and 3 partials ⚠️
packages/cli/src/workers/compileWorker.ts	0.00%	6 Missing ⚠️
packages/cli/src/api/stats.ts	80.00%	0 Missing and 2 partials ⚠️
packages/cli/src/lingui-compile.ts	84.61%	1 Missing and 1 partial ⚠️
packages/cli/src/lingui-extract-experimental.ts	0.00%	1 Missing and 1 partial ⚠️
packages/cli/src/api/compile.ts	96.29%	0 Missing and 1 partial ⚠️
packages/conf/src/getConfig.ts	75.00%	0 Missing and 1 partial ⚠️
packages/vite-plugin/src/index.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2317      +/-   ##
==========================================
- Coverage   77.05%   76.36%   -0.69%     
==========================================
  Files          84       92       +8     
  Lines        2157     2589     +432     
  Branches      555      667     +112     
==========================================
+ Hits         1662     1977     +315     
- Misses        382      486     +104     
- Partials      113      126      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Zxilly]

Can you continue working on #2318? I would like to continue contributing to this patch.

[timofei-iatsenko]

@Zxilly please contact me in DM in the discord

[Zxilly]

Sorry, I didn't see your Discord contact information in your profile. Could you please provide it?

[timofei-iatsenko]

@Zxilly you can join a lingui server by this link https://discord.gg/gFWwAYnMtA and then you can find me in the contributing channel as "Tim Iatsenko"