Skip to content

perf(pm): add resolver provider foundation#3065

Draft
elrrrrrrr wants to merge 9 commits into
nextfrom
perf/pm-review-resolver-provider-foundation
Draft

perf(pm): add resolver provider foundation#3065
elrrrrrrr wants to merge 9 commits into
nextfrom
perf/pm-review-resolver-provider-foundation

Conversation

@elrrrrrrr
Copy link
Copy Markdown
Contributor

Summary

  • Add the ManifestProvider boundary and mock provider support.
  • Move version manifest parsing to Vec buffers and add speculative full-manifest core extraction.
  • Add registry HTTP client fan-out and initial PM wiring.

Review Focus

  • Provider trait shape and ownership boundaries.
  • No scheduler/mainloop behavior change yet.

@elrrrrrrr elrrrrrrr added A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR labels May 25, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a ManifestProvider trait to abstract manifest fetching and parsing, alongside a new HTTP client pooling mechanism to improve fan-out for registry resolution. It also optimizes manifest processing by reducing memory copies and enabling speculative version extraction during full manifest parsing. Review feedback highlighted concerns regarding the switch to unbounded channels in the DiskManifestStore, recommending the use of bounded channels and non-blocking try_send operations to prevent potential memory exhaustion and performance stalls during disk I/O.

Comment on lines +109 to +115
tx: Sender<ManifestWriteJob>,
handle: JoinHandle<()>,
}

impl ManifestWriter {
fn spawn() -> Self {
let (tx, rx) = mpsc::sync_channel(MANIFEST_WRITE_QUEUE_CAPACITY);
let (tx, rx) = mpsc::channel();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The change from a bounded channel to an unbounded one removes the safety mechanism that prevented memory growth when disk I/O stalled. Since the manifest store is opportunistic, it is safer to use a bounded SyncSender and drop writes when the queue is full to ensure memory usage remains bounded, especially given that the previous implementation explicitly handled this case.

Suggested change
tx: Sender<ManifestWriteJob>,
handle: JoinHandle<()>,
}
impl ManifestWriter {
fn spawn() -> Self {
let (tx, rx) = mpsc::sync_channel(MANIFEST_WRITE_QUEUE_CAPACITY);
let (tx, rx) = mpsc::channel();
tx: mpsc::SyncSender<ManifestWriteJob>,
handle: JoinHandle<()>,
}
impl ManifestWriter {
fn spawn() -> Self {
let (tx, rx) = mpsc::sync_channel(1024);

Comment on lines +135 to 137
if self.tx.send(job).is_err() {
tracing::debug!("Manifest store writer stopped before accepting write");
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To maintain the non-blocking, "fire-and-forget" nature of the manifest store, use try_send instead of a blocking send. This ensures the resolver is never stalled by disk latency, which is critical for performance and consistent with the opportunistic nature of the cache.

        match self.tx.try_send(job) {
            Ok(()) => {}
            Err(mpsc::TrySendError::Full(_)) => {
                tracing::debug!("Manifest store writer queue full; dropping cache write");
            }
            Err(mpsc::TrySendError::Disconnected(_)) => {
                tracing::debug!("Manifest store writer stopped before accepting write");
            }
        }

@elrrrrrrr elrrrrrrr force-pushed the perf/pm-review-ci-stability branch from 9c8f2a5 to 342a456 Compare May 25, 2026 09:13
@elrrrrrrr elrrrrrrr force-pushed the perf/pm-review-resolver-provider-foundation branch from ffee403 to 9a331cb Compare May 25, 2026 09:44
Base automatically changed from perf/pm-review-ci-stability to next May 25, 2026 11:15
@github-actions
Copy link
Copy Markdown

📊 pm-bench-phases · fc0b3a4 · linux (ubuntu-latest)

Workflow run — ant-design

PMs: utoo (this branch) · utoo-npm (latest published) · bun (latest)

npmjs.org

p0_full_cold

PM wall ±σ user sys RSS pgMinor
bun 9.46s 0.10s 10.71s 10.75s 731M 324.6K
utoo-next 8.29s 0.23s 10.63s 12.49s 1007M 125.3K
utoo-npm 8.40s 0.22s 10.93s 12.69s 951M 122.3K
utoo 8.78s 0.17s 11.32s 12.98s 1.05G 142.1K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 14.8K 19.1K 1.22G 6M 1.93G 1.81G 1M
utoo-next 115.1K 80.8K 1.19G 5M 1.77G 1.77G 2M
utoo-npm 120.2K 84.4K 1.19G 5M 1.77G 1.77G 2M
utoo 132.9K 75.4K 1.19G 6M 1.77G 1.76G 2M

p1_resolve

PM wall ±σ user sys RSS pgMinor
bun 1.92s 0.04s 4.11s 1.07s 509M 159.4K
utoo-next 2.86s 0.03s 5.43s 1.73s 622M 82.8K
utoo-npm 3.12s 0.04s 5.67s 2.13s 617M 76.1K
utoo 3.20s 0.03s 5.95s 1.95s 629M 97.5K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 8.1K 4.8K 205M 3M 111M - 1M
utoo-next 47.6K 71.7K 203M 2M 7M 3M 2M
utoo-npm 71.5K 92.0K 203M 2M 7M 3M 2M
utoo 54.0K 74.4K 205M 3M 7M 3M 2M

p3_cold_install

PM wall ±σ user sys RSS pgMinor
bun 7.04s 0.16s 6.48s 10.24s 653M 212.4K
utoo-next 6.40s 0.93s 5.17s 11.24s 448M 66.2K
utoo-npm 7.21s 1.29s 5.21s 11.37s 462M 61.3K
utoo 6.60s 1.07s 5.24s 11.19s 550M 67.1K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 5.3K 7.3K 1.02G 4M 1.82G 1.82G 1M
utoo-next 104.0K 49.8K 1018M 3M 1.76G 1.76G 2M
utoo-npm 112.1K 54.6K 1018M 3M 1.76G 1.76G 2M
utoo 96.3K 52.4K 1018M 2M 1.76G 1.76G 2M

p4_warm_link

PM wall ±σ user sys RSS pgMinor
bun 3.31s 0.01s 0.22s 2.40s 135M 33.0K
utoo-next 2.30s 0.13s 0.48s 3.87s 80M 18.7K
utoo-npm 2.19s 0.03s 0.49s 3.83s 79M 18.5K
utoo 2.24s 0.09s 0.48s 3.87s 80M 18.6K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 345 30 5M 21K 1.98G 1.81G 1M
utoo-next 41.5K 18.9K 41K 11K 1.76G 1.76G 2M
utoo-npm 40.4K 19.1K 41K 24K 1.76G 1.76G 2M
utoo 42.1K 19.7K 41K 7K 1.77G 1.76G 2M

npmmirror.com: no output captured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant