Skip to content

perf(pm): make resolver registry stateless#3069

Draft
elrrrrrrr wants to merge 5 commits into
perf/pm-review-resolver-mainloop-cleanupfrom
perf/pm-review-resolver-registry-cleanup
Draft

perf(pm): make resolver registry stateless#3069
elrrrrrrr wants to merge 5 commits into
perf/pm-review-resolver-mainloop-cleanupfrom
perf/pm-review-resolver-registry-cleanup

Conversation

@elrrrrrrr
Copy link
Copy Markdown
Contributor

Summary

  • Remove registry cache hooks and memory cache layer.
  • Make UnifiedRegistry stateless now that the mainloop owns cache/inflight state.
  • Update cache ownership docs and resolver diagnostics.

Review Focus

  • No remaining resolver-side OnceMap/MemoryCache ownership in registry.

@elrrrrrrr elrrrrrrr added A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR labels May 25, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the manifest caching and resolution logic by removing the global MemoryCache and the OnceMap-based single-flight mechanism from UnifiedRegistry. Instead, the BFS resolver now owns the in-memory manifest maps for the duration of a resolution run, making the UnifiedRegistry service more stateless. A potential performance regression was identified regarding persistent storage I/O: the removal of the stored_version deduplication set means that multiple dependency specifications resolving to the same version will now trigger redundant writes to the ManifestStore.

let manifest = Arc::new(
manifest::parse_json_vec_off_runtime::<CoreVersionManifest>(bytes).await?,
);
self.store_version_manifest(&name, Arc::clone(&manifest));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The removal of the stored_version deduplication set in UnifiedRegistry introduces a performance regression for persistent storage I/O.

In the previous implementation, this set ensured that multiple dependency specifications resolving to the same version (e.g., ^1.0.0 and ^1.1.0 both resolving to 1.2.3) would only trigger a single write to the ManifestStore. With its removal, every ManifestJob::Version or ManifestJob::ExtractVersion that completes will now issue a redundant, fire-and-forget write to the store for the same version manifest.

While the BFS loop's FetchQueues deduplicates jobs with the same FetchKey, different ranges have different keys, so they will still result in multiple provider jobs that resolve to the same exact version. Consider re-introducing a deduplication mechanism, perhaps within the ManifestState of the BFS loop if the goal is to keep the UnifiedRegistry service itself stateless.

@elrrrrrrr elrrrrrrr force-pushed the perf/pm-review-resolver-mainloop-cleanup branch from a4df082 to 275de3c Compare May 25, 2026 09:49
@elrrrrrrr elrrrrrrr force-pushed the perf/pm-review-resolver-registry-cleanup branch from c458077 to ebdd9da Compare May 25, 2026 09:50
@github-actions
Copy link
Copy Markdown

📊 pm-bench-phases · 018a43a · linux (ubuntu-latest)

Workflow run — ant-design

PMs: utoo (this branch) · utoo-npm (latest published) · bun (latest)

npmjs.org

p0_full_cold

PM wall ±σ user sys RSS pgMinor
bun 9.30s 0.20s 10.53s 10.36s 762M 337.1K
utoo-next 8.56s 0.91s 10.61s 12.34s 998M 137.0K
utoo-npm 8.22s 0.31s 10.75s 12.29s 962M 117.9K
utoo 9.17s 1.00s 11.55s 12.43s 986M 147.8K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 14.9K 19.7K 1.22G 6M 1.94G 1.82G 1M
utoo-next 124.6K 86.1K 1.19G 5M 1.77G 1.77G 2M
utoo-npm 122.8K 83.1K 1.19G 5M 1.77G 1.77G 2M
utoo 125.2K 65.7K 1.19G 6M 1.77G 1.76G 2M

p1_resolve

PM wall ±σ user sys RSS pgMinor
bun 1.92s 0.16s 4.04s 1.01s 512M 161.6K
utoo-next 2.80s 0.03s 5.26s 1.66s 619M 87.1K
utoo-npm 2.98s 0.05s 5.51s 2.04s 626M 80.3K
utoo 2.44s 0.07s 6.07s 1.67s 647M 124.2K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 8.0K 4.7K 205M 3M 109M - 1M
utoo-next 47.3K 70.0K 202M 2M 7M 3M 2M
utoo-npm 71.9K 88.7K 202M 2M 7M 3M 2M
utoo 15.9K 20.6K 205M 3M 7M 3M 2M

p3_cold_install

PM wall ±σ user sys RSS pgMinor
bun 6.85s 0.12s 6.44s 10.01s 654M 211.3K
utoo-next 7.00s 1.77s 5.00s 11.07s 466M 62.4K
utoo-npm 7.63s 2.45s 5.18s 11.28s 532M 61.0K
utoo 6.67s 1.62s 5.04s 10.94s 511M 62.2K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 5.0K 7.3K 1.02G 3M 1.82G 1.82G 1M
utoo-next 107.0K 49.3K 1018M 3M 1.76G 1.76G 2M
utoo-npm 118.1K 49.9K 1018M 3M 1.76G 1.76G 2M
utoo 100.8K 52.9K 1018M 3M 1.76G 1.76G 2M

p4_warm_link

PM wall ±σ user sys RSS pgMinor
bun 3.36s 0.05s 0.20s 2.36s 136M 33.3K
utoo-next 2.33s 0.12s 0.50s 3.78s 81M 18.5K
utoo-npm 2.31s 0.04s 0.49s 3.74s 80M 18.3K
utoo 2.32s 0.04s 0.50s 3.82s 80M 18.6K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 472 24 5M 17K 1.98G 1.81G 1M
utoo-next 43.9K 19.2K 6K 6K 1.76G 1.76G 2M
utoo-npm 41.3K 18.9K 6K 22K 1.76G 1.76G 2M
utoo 45.0K 20.8K 7K 7K 1.77G 1.76G 2M

npmmirror.com: no output captured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant