Skip to content

Integrate bazel resource_sets into the build actions#1457

Closed
novas0x2a wants to merge 42 commits intobazel-contrib:mainfrom
novas0x2a:resource-sets
Closed

Integrate bazel resource_sets into the build actions#1457
novas0x2a wants to merge 42 commits intobazel-contrib:mainfrom
novas0x2a:resource-sets

Conversation

@novas0x2a
Copy link
Collaborator

This is an attempt to make parallelization of builds safer by telling bazel what resources an action will consume. By default, Bazel allocates 1 CPU and 250M of RAM, and this makes the current methods of parallelization (i.e. --action_env=CMAKE_BUILD_PARALLEL_LEVEL=16) pretty unpleasant, since bazel won't know and will happily schedule 160 cores worth of actions on a 10-core machine.

This is currently up to get early feedback; some things that are missing are:

Related to #329

A description of the intended workflow here is:

  • target owners add resource_size=tiny|small|medium|large|enormous as they see fit.
    • these sizes map to default cpu/ram allocations (see code) which affect both the resource_set and the underlying build system's parallelization controls (e.g. CMAKE_BUILD_PARALLEL_LEVEL)
  • Someone running a build can:
    • override the defaults for what (e.g.) "large" means to either fit within a smaller machine or allow taking advantage of a larger one, by setting new values for @rules_foreign_cc//foreign_cc/settings:size_$size_(cpu|mem)
    • override the global default for targets without resource_size keys with @rules_foreign_cc//foreign_cc/settings:size_default

Currently, changing the values of these settings does invalidate the cache.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR integrates Bazel resource_sets into build actions to enable safer parallelization. By specifying resource consumption, Bazel can properly schedule builds instead of over-scheduling when build systems use parallel compilation (e.g., CMAKE_BUILD_PARALLEL_LEVEL).

Changes:

  • Adds new resource_sets module that defines size classes (tiny/small/medium/large/enormous) with CPU and memory allocations
  • Updates framework.bzl and built_tools_framework.bzl to set resource_set and environment variables for parallel builds
  • Adds bazel_lib dependency (with temporary archive_override) for resource_set_for function
  • Updates dependency versions for platforms, bazel_features, bazel_skylib, rules_cc, rules_shell, and aspect_rules_lint

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
foreign_cc/private/resource_sets.bzl New module defining size classes, settings creation, and resource set computation logic
foreign_cc/settings/BUILD.bazel New file that creates build settings for resource size configuration
foreign_cc/private/framework.bzl Integrates SIZE_ATTRIBUTES and sets resource_set/env vars for cc_external_rule actions
foreign_cc/built_tools/private/built_tools_framework.bzl Integrates SIZE_ATTRIBUTES and sets resource_set/env vars for built tool actions
toolchains/private/BUILD.bazel Example usage: sets cmake_tool to "medium" resource size
foreign_cc/repositories.bzl Adds bazel_lib dependency and updates versions for platforms, bazel_features, bazel_skylib, rules_cc, rules_shell
MODULE.bazel Updates dependency versions and adds temporary archive_override for bazel_lib pending upstream PR
WORKSPACE.bazel Updates aspect_rules_lint and switches from aspect_bazel_lib to bazel_lib

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@novas0x2a novas0x2a force-pushed the resource-sets branch 3 times, most recently from 24c447e to b04bc68 Compare January 23, 2026 02:30
@novas0x2a novas0x2a requested a review from Copilot January 23, 2026 02:41
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 19 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Every single release, zlib upstream breaks the previous urls. To work
around this antisocial behavior, use the github url instead.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 40 out of 43 changed files in this pull request and generated 11 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

novas0x2a and others added 2 commits February 18, 2026 23:35
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@novas0x2a
Copy link
Collaborator Author

replaced by #1465

@novas0x2a novas0x2a closed this Feb 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants