Skip to content

fix(pm): serialize cross-process cache writes - #3257

Merged
elrrrrrrr merged 1 commit into
nextfrom
codex/fix-cross-process-cache-lock
Jul 22, 2026
Merged

fix(pm): serialize cross-process cache writes#3257
elrrrrrrr merged 1 commit into
nextfrom
codex/fix-cross-process-cache-lock

Conversation

@elrrrrrrr

@elrrrrrrr elrrrrrrr commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a persistent zero-byte sibling lock per registry cache slot and clone target, backed by std::fs::File locks (flock on Unix, LockFileEx on Windows)
  • acquire the lock around the existing extract and clone flows, with their existing resolved/target checks performed under the lock
  • make copy fallback unlink and exclusively recreate its destination so it cannot O_TRUNC a cache inode through a hardlink
  • ignore persistent lock files when collecting entries for utoo clean

Scope

This intentionally keeps the existing extraction commit protocol and direct clone behavior unchanged. It does not add a new staging model, cache-manifest validation, or automatic repair of already-corrupted slots. Existing broken cache entries still need to be cleared once.

Root cause

The install scheduler deduplicates work inside one process, but independent utoo processes can still race.

Two paths need serialization:

  1. Concurrent clones can hardlink the same cache file into one target. A losing clone then falls back to copy; opening a destination that is the same inode as the source truncates the cache file.
  2. Registry extraction checks _resolved and removes an unresolved destination before extracting. Without a process lock, another process can publish the slot between those operations.

Fixes #3256.
Related reproduction: #3255.

Validation

  • cargo test -p utoo-pm: 331 passed, 3 ignored
  • cargo clippy -p utoo-pm --all-targets -- -D warnings
  • child-process test verifies cross-process exclusion and a persistent empty lock file
  • concurrent extraction test verifies one complete shared cache slot
  • Linux hardlink regression test verifies copy fallback does not truncate the cache source

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@elrrrrrrr
elrrrrrrr force-pushed the codex/fix-cross-process-cache-lock branch from 064f478 to eefb9e6 Compare July 22, 2026 04:21
@elrrrrrrr
elrrrrrrr force-pushed the codex/fix-cross-process-cache-lock branch from eefb9e6 to e6d0de0 Compare July 22, 2026 05:40
@elrrrrrrr
elrrrrrrr marked this pull request as ready for review July 22, 2026 05:48
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@elrrrrrrr elrrrrrrr added the benchmark Run pm-bench on PR label Jul 22, 2026 — with ChatGPT Codex Connector
@github-actions

Copy link
Copy Markdown

📊 pm-bench-phases · 2dc246e · linux (ubuntu-latest)

Workflow run — ant-design

registry.npmjs.org

utoo (PR) vs utoo-next (baseline): ✅ p0 +0.5% · ✅ p1 +2.2% · ✅ p3 +3.5% · ✅ p4 -3.1%

✅ within noise · 🚀 faster · ⚠️ slower — Δ is the median of per-round paired deltas (interleaved rounds share weather windows); flagged when |Δ| > 5% and ≥80% of rounds agree on sign

p0 · full cold install

PM wall (mean ± σ) min user sys RSS Δ vs baseline
utoo (PR) 8.72s ± 0.27s 8.48s 11.0s 13.8s 941M +0.5% ✅
utoo-next (baseline) 8.56s ± 0.19s 8.35s 10.6s 13.5s 935M
bun 10.7s ± 0.51s 10.1s 11.4s 11.3s 707M +24.7% ⚠️

p1 · resolve

PM wall (mean ± σ) min user sys RSS Δ vs baseline
utoo (PR) 2.35s ± 0.02s 2.32s 4.97s 1.73s 580M +2.2% ✅
utoo-next (baseline) 2.38s ± 0.18s 2.26s 5.01s 1.69s 570M
bun 2.45s ± 0.12s 2.35s 4.40s 1.23s 542M +4.0% ✅

p3 · cold install

PM wall (mean ± σ) min user sys RSS Δ vs baseline
utoo (PR) 7.73s ± 1.06s 7.11s 5.85s 12.4s 438M +3.5% ✅
utoo-next (baseline) 6.95s ± 0.21s 6.78s 5.52s 12.0s 424M
bun 8.22s ± 0.97s 7.21s 6.86s 11.0s 640M +20.5% ⚠️

p4 · warm link

PM wall (mean ± σ) min user sys RSS Δ vs baseline
utoo (PR) 2.48s ± 0.08s 2.40s 0.49s 4.11s 46M -3.1% ✅
utoo-next (baseline) 2.54s ± 0.07s 2.46s 0.46s 3.93s 46M
bun 3.86s ± 0.26s 3.72s 0.19s 2.56s 136M +48.2% ⚠️
Resources & footprint

p0 · full cold install

PM vCtx iCtx netRX netTX cache node_modules lock
utoo 84309 64137 1.28G 8M 1.88G 1.86G 2M
utoo-next 73909 55097 1.28G 7M 1.88G 1.86G 2M
bun 18368 18737 1.30G 7M 2.03G 1.90G 1M

p1 · resolve

PM vCtx iCtx netRX netTX cache node_modules lock
utoo 23532 46523 215M 3M 7M 0B 2M
utoo-next 23158 43222 215M 3M 7M 0B 2M
bun 10854 4403 216M 3M 118M 0B 1M

p3 · cold install

PM vCtx iCtx netRX netTX cache node_modules lock
utoo 76377 48489 1.07G 5M 1.86G 1.86G 2M
utoo-next 72226 43875 1.07G 5M 1.86G 1.86G 2M
bun 5539 8184 1.09G 4M 1.91G 1.91G 1M

p4 · warm link

PM vCtx iCtx netRX netTX cache node_modules lock
utoo 20499 11033 3K 9K 1.86G 1.86G 2M
utoo-next 20198 10905 960B 3K 1.86G 1.86G 2M
bun 303 29 3K 5K 1.91G 1.91G 1M

@elrrrrrrr
elrrrrrrr merged commit 36a1ad2 into next Jul 22, 2026
53 of 85 checks passed
@elrrrrrrr
elrrrrrrr deleted the codex/fix-cross-process-cache-lock branch July 22, 2026 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark Run pm-bench on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 package cache can become unstable across concurrent utoo installs

2 participants