-
Notifications
You must be signed in to change notification settings - Fork 15
Add bzlmod support for arch_alias with module extension #3131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mmorel-35
wants to merge
1
commit into
envoyproxy:main
Choose a base branch
from
mmorel-35:clang_platform
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| # Repository Utils | ||
|
|
||
| This directory contains utility functions and rules for Bazel repository management. | ||
|
|
||
| ## arch_alias | ||
|
|
||
| `arch_alias` is a repository rule that creates architecture-specific platform aliases. This is useful for setting platform-specific build configurations based on the host CPU architecture. | ||
|
|
||
| ### Problem Statement | ||
|
|
||
| When building software that needs to target different platforms based on the host architecture, you need a way to create aliases that resolve to different platform targets. Using `native.alias` with `select()` doesn't work well when the host architecture is the selector. | ||
|
|
||
| ### Solution | ||
|
|
||
| The `arch_alias` repository rule detects the host CPU architecture at repository loading time and creates an alias that points to the appropriate platform target. | ||
|
|
||
| ### Usage | ||
|
|
||
| #### WORKSPACE Mode (Legacy) | ||
|
|
||
| ```starlark | ||
| load("@envoy_toolshed//bazel/repository:utils.bzl", "arch_alias") | ||
|
|
||
| arch_alias( | ||
| name = "clang_platform", | ||
| aliases = { | ||
| "amd64": "@envoy//bazel/platforms/rbe:rbe_linux_x64_clang_platform", | ||
| "aarch64": "@envoy//bazel/platforms/rbe:rbe_linux_arm64_clang_platform", | ||
| }, | ||
| ) | ||
| ``` | ||
|
|
||
| #### MODULE.bazel Mode (Bzlmod - Recommended) | ||
|
|
||
| ```starlark | ||
| # In MODULE.bazel | ||
| arch_alias_ext = use_extension("@envoy_toolshed//bazel/repository:utils.bzl", "arch_alias_ext") | ||
| arch_alias_ext.alias( | ||
| name = "clang_platform", | ||
| aliases = { | ||
| "amd64": "@envoy//bazel/platforms/rbe:rbe_linux_x64_clang_platform", | ||
| "aarch64": "@envoy//bazel/platforms/rbe:rbe_linux_arm64_clang_platform", | ||
| }, | ||
| ) | ||
| use_repo(arch_alias_ext, "clang_platform") | ||
| ``` | ||
|
|
||
| #### Using in .bazelrc | ||
|
|
||
| After defining the alias, you can reference it in your `.bazelrc`: | ||
|
|
||
| ``` | ||
| common:clang-common --host_platform=@clang_platform | ||
| ``` | ||
|
|
||
| ### How It Works | ||
|
|
||
| 1. The repository rule executes during the loading phase | ||
| 2. It detects the host CPU architecture using `ctx.os.arch` | ||
| 3. It looks up the corresponding platform target in the `aliases` dictionary | ||
| 4. It creates a BUILD file with an `alias()` target pointing to the selected platform | ||
| 5. The alias can then be referenced as `@<name>` in build configurations | ||
|
|
||
| ### Supported Architectures | ||
|
|
||
| The implementation uses `ctx.os.arch` which returns architecture strings like: | ||
| - `amd64` - x86-64 / x64 | ||
| - `aarch64` - ARM64 | ||
| - Other architectures as reported by the host OS | ||
|
|
||
| ### Bzlmod Best Practices | ||
|
|
||
| This implementation follows bzlmod best practices: | ||
|
|
||
| 1. **Module Extension**: Uses `module_extension()` for bzlmod compatibility | ||
| 2. **Tag Classes**: Defines structured configuration with `tag_class()` | ||
| 3. **Backward Compatibility**: The repository rule still works in WORKSPACE mode | ||
| 4. **Repository Declaration**: Repositories are created through the extension, not directly | ||
|
|
||
| ### Migration from WORKSPACE to Bzlmod | ||
|
|
||
| If you're migrating from WORKSPACE to bzlmod: | ||
|
|
||
| 1. Remove the `arch_alias()` call from your WORKSPACE file | ||
| 2. Add the equivalent configuration to your MODULE.bazel using the extension | ||
| 3. Ensure you call `use_repo()` to make the repository visible | ||
| 4. Update your `.bazelrc` if the repository name changed (though it shouldn't need to) | ||
|
|
||
| ### Example: Envoy Platform Configuration | ||
|
|
||
| In Envoy, this is used to select between different RBE (Remote Build Execution) platforms: | ||
|
|
||
| ```starlark | ||
| arch_alias_ext.alias( | ||
| name = "clang_platform", | ||
| aliases = { | ||
| "amd64": "@envoy//bazel/platforms/rbe:rbe_linux_x64_clang_platform", | ||
| "aarch64": "@envoy//bazel/platforms/rbe:rbe_linux_arm64_clang_platform", | ||
| }, | ||
| ) | ||
| ``` | ||
|
|
||
| Then in `.bazelrc`: | ||
| ``` | ||
| common:clang-common --host_platform=@clang_platform | ||
| ``` | ||
|
|
||
| This allows developers on different architectures to use the same bazel configuration while automatically selecting the appropriate platform. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # arch_alias Tests | ||
|
|
||
| This directory contains comprehensive tests for the `arch_alias` functionality in both WORKSPACE and bzlmod modes. | ||
|
|
||
| ## Directory Structure | ||
|
|
||
| ``` | ||
| test/ | ||
| └── arch_alias/ # Tests for arch_alias functionality | ||
| ├── bzlmod/ # Tests for bzlmod mode (MODULE.bazel) | ||
| │ ├── MODULE.bazel # Bzlmod test module configuration | ||
| │ ├── BUILD.bazel # Test targets | ||
| │ └── .bazelrc # Test configuration | ||
| ├── workspace/ # Tests for WORKSPACE mode (legacy) | ||
| │ ├── WORKSPACE # WORKSPACE test configuration | ||
| │ ├── BUILD.bazel # Test targets | ||
| │ └── .bazelrc # Test configuration | ||
| └── README.md # This file | ||
| ``` | ||
|
|
||
| ## Purpose | ||
|
|
||
| These tests validate: | ||
| 1. **Bzlmod Mode** (`arch_alias/bzlmod/`): | ||
| - The `arch_alias_ext` module extension works correctly | ||
| - Multiple arch aliases can be created in the same module | ||
| - The aliases resolve to correct platform targets based on host architecture | ||
| - Integration with BCR presubmit checks | ||
|
|
||
| 2. **WORKSPACE Mode** (`arch_alias/workspace/`): | ||
| - The `arch_alias` repository rule works correctly | ||
| - Backward compatibility with existing WORKSPACE-based configurations | ||
| - Multiple aliases can coexist in the same workspace | ||
|
|
||
| ## Running Tests | ||
|
|
||
| ### Test Bzlmod Mode | ||
|
|
||
| ```bash | ||
| cd bazel/test/arch_alias/bzlmod | ||
| bazel build //... | ||
| bazel test //... | ||
| ``` | ||
|
|
||
| ### Test WORKSPACE Mode | ||
|
|
||
| ```bash | ||
| cd bazel/test/arch_alias/workspace | ||
| bazel build //... | ||
| bazel test //... | ||
| ``` | ||
|
|
||
| ### Verify Platform Aliases | ||
|
|
||
| You can inspect the created aliases: | ||
|
|
||
| ```bash | ||
| # In bzlmod_test or workspace_test directory | ||
| bazel query @test_platform//... | ||
| bazel query @test_clang_platform//... | ||
| ``` | ||
|
|
||
| ## BCR Integration | ||
|
|
||
| The `bzlmod_test/` follows BCR best practices and is suitable for BCR presubmit testing: | ||
| - Located in the standard `test/` directory structure | ||
| - Uses `local_path_override` to test the parent module | ||
| - Can be run independently or as part of CI/CD | ||
| - Tests both single and multiple alias scenarios | ||
|
|
||
| ## Architecture Coverage | ||
|
|
||
| Both tests cover common architecture identifiers: | ||
| - `amd64` / `x86_64` - Intel/AMD 64-bit | ||
| - `aarch64` / `arm64` - ARM 64-bit | ||
|
|
||
| The aliases map to platform targets from `@platforms` for validation. | ||
|
|
||
| ## What Gets Tested | ||
|
|
||
| 1. **Repository Creation**: Aliases create proper external repositories | ||
| 2. **Platform Resolution**: Aliases resolve to correct platform targets based on host arch | ||
| 3. **Build Configuration**: Aliases work correctly in `.bazelrc` with `--host_platform` | ||
| 4. **Multiple Aliases**: Multiple aliases can coexist and work independently | ||
| 5. **API Compatibility**: Both WORKSPACE and bzlmod APIs function correctly |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # arch_alias Tests | ||
|
|
||
| This directory contains tests for the `arch_alias` repository rule and module extension. | ||
|
|
||
| ## Test Modes | ||
|
|
||
| ### bzlmod/ | ||
| Tests the `arch_alias_ext` module extension for bzlmod compatibility. This is the recommended approach for new projects using Bazel's module system. | ||
|
|
||
| ### workspace/ | ||
| Tests the `arch_alias` repository rule in WORKSPACE mode for backward compatibility with existing configurations. | ||
|
|
||
| ## What is arch_alias? | ||
|
|
||
| `arch_alias` enables architecture-specific platform selection by detecting the host CPU architecture and creating aliases that resolve to different platform targets. This is particularly useful for: | ||
|
|
||
| - Remote Build Execution (RBE) with architecture-specific platforms | ||
| - Cross-compilation scenarios | ||
| - CI/CD pipelines that need to adapt to different host architectures | ||
|
|
||
| ## Running All Tests | ||
|
|
||
| From this directory: | ||
| ```bash | ||
| # Test bzlmod mode | ||
| cd bzlmod && bazel build //... && bazel test //... | ||
|
|
||
| # Test workspace mode | ||
| cd workspace && bazel build //... && bazel test //... | ||
| ``` | ||
|
|
||
| See the individual subdirectories for detailed documentation on each test mode. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Bazel configuration for arch_alias extension test module | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the test files here seem misplaced - altho i understand these tests need a separate workspace to use toolshed externally, which i think is not a bad idea - but then it should be testing non-bzlmod also |
||
|
|
||
| # Enable bzlmod for this test | ||
| common --enable_bzlmod | ||
|
|
||
| # Use the test platform alias to verify it resolves correctly | ||
| build --host_platform=@test_platform | ||
|
|
||
| # Standard build configurations | ||
| build --color=yes | ||
| test --test_output=errors | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please dont keep submitting voluminous bot docs - it overwhelms reviewers and detracts from the actual changes