Skip to content

perf(uv): load wheel metadata lazily#1348

Open
zbarsky-openai wants to merge 1 commit into
aspect-build:mainfrom
zbarsky-openai:zbarsky/lazy-wheel-metadata
Open

perf(uv): load wheel metadata lazily#1348
zbarsky-openai wants to merge 1 commit into
aspect-build:mainfrom
zbarsky-openai:zbarsky/lazy-wheel-metadata

Conversation

@zbarsky-openai

Copy link
Copy Markdown
Contributor

Summary

  • Create one whl_install repository per concrete fetched wheel variant.
  • Keep the public install repository as a selector, so inactive platform wheels do not have their metadata extracted.

Validation

  • bazel run //:buildifier -- /tmp/rules-py-lazy/uv/private/extension/defs.bzl /tmp/rules-py-lazy/uv/private/whl_install/repository.bzl
  • bazel build //uv/private/extension:defs //uv/private/whl_install:repository

@CLAassistant

CLAassistant commented Jul 19, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@tamird tamird left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lazy-metadata split matches the OpenAI patch, but the current CI failures are causal and the universal-wheel path now pays the extra repository/target cost without a demonstrated win. Please refresh the affected snapshots/legacy target assumptions and cover the selector behavior before landing.

if install_cfg.post_install_patches:
variant_kwargs["post_install_patches"] = json.encode(install_cfg.post_install_patches)
variant_kwargs["post_install_patch_strip"] = install_cfg.post_install_patch_strip
exclude_glob = getattr(install_cfg, "exclude_glob", [])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates a selector repository even when the lock has one universal wheel. The accompanying benchmark grows from 451 to 556 loaded packages and 8,677 to 9,097 configured targets without a measured speedup. Can we keep the single-variant path direct and split only genuinely platform-varying installs? Please also port the shared-alias, both-platform, and sdist-fallback regressions from the OpenAI patch.

patches = repr(post_install_patches),
strip = post_install_patch_strip,
)
exclude_glob = getattr(repository_ctx.attr, "exclude_glob", [])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this exists yet?

@jbedard
jbedard force-pushed the zbarsky/lazy-wheel-metadata branch from d0b8092 to 8d628b0 Compare July 20, 2026 06:47
@jbedard

jbedard commented Jul 20, 2026

Copy link
Copy Markdown
Member

Alternative: #1356

tamird commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

The Bazel 8/9 failures are causal: the extra variant repositories change the observable install roots and invalidate the venv/image snapshots and legacy wheel-chain fixture, while increasing the configured graph. #1356 keeps the stable install tree and avoids that growth, so it is the better metadata boundary. I reviewed its remaining provider/downloader gaps in #1356 (review); once those are addressed, this approach should be retired rather than updating snapshots around the path change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants