Skip to content

feat(FR-3214): exclude pnpm store from the dev type-check watcher to cut per-process watch FDs#8357

Open
nowgnuesLee wants to merge 1 commit into
06-26-fix_fr-3214_exclude_pnpm_store_from_type-check_watcher_so_multi-repo_dev_doesn_t_exhaust_file_descriptorsfrom
07-23-feat_fr-3214_exclude_pnpm_store_from_the_dev_type-check_watcher_to_cut_per-process_watch_fds
Open

feat(FR-3214): exclude pnpm store from the dev type-check watcher to cut per-process watch FDs#8357
nowgnuesLee wants to merge 1 commit into
06-26-fix_fr-3214_exclude_pnpm_store_from_type-check_watcher_so_multi-repo_dev_doesn_t_exhaust_file_descriptorsfrom
07-23-feat_fr-3214_exclude_pnpm_store_from_the_dev_type-check_watcher_to_cut_per-process_watch_fds

Conversation

@nowgnuesLee

@nowgnuesLee nowgnuesLee commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Related to #8047 (FR-3214) — the issue itself is resolved by #8156 (process-group kill); this PR is the complementary footprint reduction, split out per review discussion.

Stacked on #8156

Problem

Even with orphan leaks fixed in #8156, each live dev server still holds far more file descriptors than it needs. Measured on a live dev stack: a single vite dev process holds ~5,894 open file descriptors, of which ~4,481 are regular files inside the pnpm store (immutable dependency .d.ts type declarations, ~3,840 of them).

The holder is vite-plugin-checker's TypeScript watch program (ts.createWatchProgram), which installs a per-file watcher on every file in the program — including every dependency .d.ts in the store. On macOS each per-file watch costs 1 open FD, so:

  • GUI/IDE-launched dev inherits launchd's soft maxfiles = 256 → instant EMFILE.
  • Terminal-launched dev survives one repo but running N repos multiplies ~5,900 × N toward the kernel limit.

Fix

Tell the checker's TS watch program not to watch the pnpm store via watchOptions.excludeDirectories / excludeFiles.

Two constraints shape the implementation:

  1. The exclude spec must be an absolute store path. TS anchors **/-globs under the project basePath (combinePaths()), so they can never match the store realpath which lives outside the repo (~/Library/pnpm/store/v11/...). Measured: **/node_modules/**, **/store/** etc. all had zero effect. Since the absolute path is machine-specific, it cannot live in the committed tsconfig.json.
  2. vite-plugin-checker (0.9.3) has no inline watchOptions API — it calls ts.createWatchCompilerHost without a watchOptionsToExtend arg, so the only injection point is the tsconfig file it reads.

Therefore react/vite.config.ts now generates a gitignored react/tsconfig.checker.json at dev startup — { extends: './tsconfig.json', watchOptions: { excludeDirectories/excludeFiles: ['<absolute store root>/**'] } } — and points checker({ typescript: { tsconfigPath } }) at it. The store root reuses the existing pnpmStorePath (from pnpm store path).

Why excluding the store from the watch is safe

  • Type-checking stays complete. watchOptions excludes only drop the file watcher, not the read. TS still reads every dependency .d.ts to build the program — verified: error count identical before/after (real-stack boot: "Found 0 errors" both ways; in an error-injection harness, 296 → 296).
  • CI / pre-commit / scripts/verify.sh are untouched. Those run one-shot tsc --noEmit (no --watch), which ignores watchOptions entirely — and this PR does not modify the committed react/tsconfig.json at all. The exclude lives only in the gitignored derived file that the dev-mode checker reads. Every gate that actually enforces type safety is unaffected.
  • vite build is untouched. pnpmStorePath is only resolved when command === 'serve'; on build the helper returns the plain base tsconfig without writing anything (a one-shot build installs no file watchers anyway).
  • HMR and workspace packages keep full watch. HMR is Vite's own separate watcher, unrelated to the checker's watchOptions. backend.ai-ui / backend.ai-client resolve via tsconfig paths to ../packages/*/src — project-internal source outside the store — so they keep normal per-file watch and instant re-check.
  • Graceful fallback. If the store path can't be resolved (npm/yarn flat installs) or the derived file can't be written, the checker falls back to the plain tsconfig.json — original behavior, no exclude.

The only theoretical gap: a dependency .d.ts changing while dev is running, with zero source edits won't auto-trigger a re-check. In practice pnpm install/add/update triggers Vite's dep re-optimization / dev restart (which re-reads the store), and using a new package requires a source edit — which is watched and re-triggers the checker. The store is content-addressed and immutable during normal dev, and CI re-checks everything regardless.

Measurements (real stack, lsof on the live vite pid)

config vite open REG files files held open under pnpm store type errors live edit detection
before (no watchOptions) 5,884 3,825 baseline YES (~2s)
after (this PR) 1,404 (−76%) 2 identical YES (~2s)

Also verified this is not a Vite regression by actually booting the last pure-craco commit (00e8431a1) in a worktree and measuring the same tsc --watch program: craco-era = 6,870 FDs (6,055 in-store) vs current Vite = 5,894 — the Vite stack was already slightly lighter; the store-watch cost is inherent to running an in-process TS type-checker, and this PR removes it.

Verification

  • Re-measured on this stacked branch (same method both runs: boot pnpm run dev, wait for the checker's "Found 0 errors", then lsof the vite pid): parent branch without the exclude = 6,056 open REG files (4,566 under the pnpm store) vs this branch = 87 open REG files (2 under the store). tsconfig.checker.json generated with the correct absolute store glob and confirmed gitignored; checker reports "Found 0 errors" in both runs.
  • bash scripts/verify.sh=== ALL PASS === (Relay / Lint / Format / TypeScript), re-run on this stacked branch.
  • Real-stack boot test (from the original round of this change): tsconfig.checker.json generated correctly, confirmed gitignored, checker reports "Found 0 errors", FD table above.
  • Injected a TS2322 into react/src/index.tsx via atomic save while dev was running → checker reported it within ~2s (source watch intact); reverting cleared it.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the size:M 30~100 LoC label Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • flow:merge-queue - adds this PR to the back of the merge queue
  • flow:hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces dev-server file descriptor usage by excluding the immutable pnpm store from TypeScript’s watch program.

Changes:

  • Generates a machine-specific checker tsconfig during development.
  • Falls back safely to the standard tsconfig.
  • Ignores the generated config in Git.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File Description
react/vite.config.ts Generates and applies the pnpm-store watch exclusions.
.gitignore Ignores the generated checker tsconfig.

…cut per-process watch FDs

Stacked on the process-group kill fix. The group kill stops orphaned
watch trees from accumulating; this change additionally shrinks each
live dev server's footprint by ~4,500 FDs: vite-plugin-checker's TS
watch program installs a per-file kqueue watcher on every dependency
.d.ts in the immutable pnpm store. dev.mjs-side vite config now derives
a gitignored react/tsconfig.checker.json with
watchOptions.excludeDirectories/excludeFiles pointing at the absolute
store root and points the checker at it. Type-checking completeness,
CI, build, and HMR are unaffected (see PR description for measurements).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ar6Rr2uEGLPBXcWNpASXDD
@nowgnuesLee
nowgnuesLee force-pushed the 07-23-feat_fr-3214_exclude_pnpm_store_from_the_dev_type-check_watcher_to_cut_per-process_watch_fds branch from 35067ae to f4ce9bf Compare July 23, 2026 08:53

@agatha197 agatha197 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mechanism looks correct and I verified the injection locally (generated an equivalent tsconfig.checker.json; tsc --showConfig carries watchOptions with the absolute store glob, inherits the base paths workaround, and the program file set is unchanged — so type-checking stays complete and only the watchers are dropped). The FD reduction itself is macOS/kqueue-specific and I can't reproduce it on Linux, so that rests on your live lsof measurements, which look thorough.

Requesting one change (the stale-file cleanup) plus two optional nits inline. None are blocking correctness — happy to see it merge once the cleanup is addressed or consciously waived.

Comment thread react/vite.config.ts
// store-resolution / write failure → no-op, original behavior preserved).
function resolveCheckerTsconfigPath(storeRoot: string | undefined): string {
const baseTsconfig = resolve(__dirname, 'tsconfig.json');
if (!storeRoot) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested change — clean up a stale derived config on the degrade path. If an earlier serve wrote tsconfig.checker.json and a later run resolves no store root (pnpm store lookup fails, or a flat npm/yarn install), we return the base tsconfig but leave the previously-generated file on disk. It's gitignored and unreferenced so it's harmless, but the checker silently keeps reading a stale exclude config across runs. Suggest unlinking it here before returning:

if (!storeRoot) {
  try { rmSync(resolve(__dirname, 'tsconfig.checker.json'), { force: true }); } catch {}
  return baseTsconfig;
}

Comment thread react/vite.config.ts
// TS watchOptions globs use forward slashes, even on Windows.
const storeGlob = `${normalizePath(storeRoot)}/**`;
try {
writeFileSync(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (optional) — side-effecting file write during config evaluation. writeFileSync runs as a side effect of evaluating the defineConfig factory (the plugins array), so it fires on every config load/reload rather than as part of the dev-server lifecycle. It works, but writing a file while resolving config is a little surprising. Consider moving the generation into a small plugin configResolved/buildStart hook so the write is tied to an explicit lifecycle step. Non-blocking.

Comment thread react/vite.config.ts
extends: './tsconfig.json',
watchOptions: {
excludeDirectories: [storeGlob],
excludeFiles: [storeGlob],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (optional). excludeFiles is matched against files, whereas <storeRoot>/** is a directory-subtree glob already covered by excludeDirectories above — so this line is redundant belt-and-suspenders. It's harmless (and your measurements confirm the exclude works), just noting it in case you'd rather keep excludeFiles for an actual file pattern or drop it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30~100 LoC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants