Integrate bazel resource_sets into the build actions#1457
Integrate bazel resource_sets into the build actions#1457novas0x2a wants to merge 42 commits intobazel-contrib:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR integrates Bazel resource_sets into build actions to enable safer parallelization. By specifying resource consumption, Bazel can properly schedule builds instead of over-scheduling when build systems use parallel compilation (e.g., CMAKE_BUILD_PARALLEL_LEVEL).
Changes:
- Adds new resource_sets module that defines size classes (tiny/small/medium/large/enormous) with CPU and memory allocations
- Updates framework.bzl and built_tools_framework.bzl to set resource_set and environment variables for parallel builds
- Adds bazel_lib dependency (with temporary archive_override) for resource_set_for function
- Updates dependency versions for platforms, bazel_features, bazel_skylib, rules_cc, rules_shell, and aspect_rules_lint
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| foreign_cc/private/resource_sets.bzl | New module defining size classes, settings creation, and resource set computation logic |
| foreign_cc/settings/BUILD.bazel | New file that creates build settings for resource size configuration |
| foreign_cc/private/framework.bzl | Integrates SIZE_ATTRIBUTES and sets resource_set/env vars for cc_external_rule actions |
| foreign_cc/built_tools/private/built_tools_framework.bzl | Integrates SIZE_ATTRIBUTES and sets resource_set/env vars for built tool actions |
| toolchains/private/BUILD.bazel | Example usage: sets cmake_tool to "medium" resource size |
| foreign_cc/repositories.bzl | Adds bazel_lib dependency and updates versions for platforms, bazel_features, bazel_skylib, rules_cc, rules_shell |
| MODULE.bazel | Updates dependency versions and adds temporary archive_override for bazel_lib pending upstream PR |
| WORKSPACE.bazel | Updates aspect_rules_lint and switches from aspect_bazel_lib to bazel_lib |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
24c447e to
b04bc68
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 12 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 19 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This bumps bazel_lib to 3.1.0, and then syncs the prod dependencies between bzlmod and workspaces.
a5961cb to
5f9f345
Compare
https://bazel.build/release: - bazel 5 went eol jan 2025 - bazel 6 went eol dec 2025
64a7126 to
4d2e855
Compare
Every single release, zlib upstream breaks the previous urls. To work around this antisocial behavior, use the github url instead.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 43 changed files in this pull request and generated 11 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
replaced by #1465 |
This is an attempt to make parallelization of builds safer by telling bazel what resources an action will consume. By default, Bazel allocates 1 CPU and 250M of RAM, and this makes the current methods of parallelization (i.e.
--action_env=CMAKE_BUILD_PARALLEL_LEVEL=16) pretty unpleasant, since bazel won't know and will happily schedule 160 cores worth of actions on a 10-core machine.This is currently up to get early feedback; some things that are missing are:
Related to #329
A description of the intended workflow here is:
resource_size=tiny|small|medium|large|enormousas they see fit.CMAKE_BUILD_PARALLEL_LEVEL)@rules_foreign_cc//foreign_cc/settings:size_$size_(cpu|mem)@rules_foreign_cc//foreign_cc/settings:size_defaultCurrently, changing the values of these settings does invalidate the cache.