Skip to content

Commit 959ae48

Browse files
committed
Make the Scala toolchain optional, improve protoc
Enables users defining custom Scala toolchains to avoid instantiating any builtin compiler JAR repos or validating the Scala version, but still use other builtin toolchains. Adds the `scala` parameter to `scala_toolchains()` to control whether it instantiates default Scala toolchains, and is `True` by default. The Scala version check will now happen only when both `scala` and `validate_scala_version` are `True`, which is essentially how the previous API worked. Removes the `if len(toolchains) == 0` check from `_scala_toolchains_repo_impl`. Now that it's responsible for always generating a (possibly empty) `@rules_scala_toolchains//protoc` package, it's OK if there are no other toolchains. Moves protoc toolchain targets from `@rules_scala_toolchains//protoc` to `//protoc`. Experimenting with moving that toolchain to its own repo showed that this makes `register_toolchains("@rules_scala//protoc:all")` more robust by itself. Prompted by: bazel-contrib#1710 (comment) Includes `README.md` updates: - Bumps the `abseil-cpp` version in `README.md` - Indicates that compiler flags are required when using Bazel 7.5.0 with newer versions of `protobuf` and `abseil-cpp`. Adds `update_protoc_integrity.py` to `scripts/README.md`.
1 parent 09989c0 commit 959ae48

File tree

8 files changed

+124
-87
lines changed

8 files changed

+124
-87
lines changed

README.md

+78-48
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ load("@platforms//host:extension.bzl", "host_platform_repo")
7575
host_platform_repo(name = "host_platform")
7676

7777
# This is optional, but still safe to include even when not using
78-
# `--incompatible_enable_proto_toolchain_resolution`.
78+
# `--incompatible_enable_proto_toolchain_resolution`. Requires calling
79+
# `scala_toolchains()`.
7980
register_toolchains("@rules_scala//protoc:all")
8081

8182
load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies")
@@ -225,6 +226,10 @@ other toolchain registrations. It's safe to include even when not using
225226
register_toolchains("@rules_scala//protoc:all")
226227
```
227228

229+
Note that you _must_ call `scala_toolchains()` in `WORKSPACE`, even if all other
230+
toolchains are disabled (i.e., if using `scala_toolchains(scala = False)`). This
231+
isn't necessary under Bzlmod.
232+
228233
#### Specifying additional `protoc` platforms
229234

230235
Use the `protoc_platforms` parameter to specify additional [platforms][] if the
@@ -480,10 +485,41 @@ maximum available at the time of writing.
480485
| Bazel versions using Bzlmod<br/>(Coming soon! See bazelbuild/rules_scala#1482.) | 7.5.0, 8.x,<br/>`rolling`, `last_green` |
481486
| Bazel versions using `WORKSPACE` | 6.5.0, 7.5.0, 8.x<br/>(see the [notes on 6.5.0 compatibility](#6.5.0)) |
482487
| `protobuf` | v30.1 |
483-
| `abseil-cpp` | 20250127.0 |
488+
| `abseil-cpp` | 20250127.1 |
484489
| `rules_java` | 8.11.0 |
485490
| `ScalaPB` | 1.0.0-alpha.1 |
486491

492+
### Using a prebuilt `@com_google_protobuf//:protoc` or C++ compiler flags
493+
494+
Newer versions of `abseil-cpp`, required by newer versions of
495+
`@com_google_protobuf//:protoc`, fail to compile under Bazel 6.5.0 and 7.5.0 by
496+
default. [protoc will also fail to build on Windows when using
497+
MSVC](#protoc-msvc). You will have to choose one of the following approaches to
498+
resolve this problem.
499+
500+
You may use protocol compiler toolchainization with `protobuf` v29 or later to
501+
avoid recompiling `protoc`. You may want to enable this even if your build
502+
doesn't break, as it saves time by avoiding frequent `protoc` recompilation. See
503+
the [Using a precompiled protocol compiler](#protoc) section for details.
504+
505+
Otherwise, if migrating to Bazel 8 isn't an immediate option, you will need to
506+
set the following compiler flags in `.bazelrc` per bazelbuild/rules_scala#1647:
507+
508+
```txt
509+
common --enable_platform_specific_config
510+
511+
common:linux --cxxopt=-std=c++17
512+
common:linux --host_cxxopt=-std=c++17
513+
common:macos --cxxopt=-std=c++17
514+
common:macos --host_cxxopt=-std=c++17
515+
common:windows --cxxopt=/std=c++17
516+
common:windows --host_cxxopt=/std=c++17
517+
```
518+
519+
Note that this example uses `common:` config settings instead of `build:`. This
520+
seems to prevent invalidating the action cache between `bazel` runs, which
521+
improves performance.
522+
487523
## Usage with [bazel-deps](https://github.com/johnynek/bazel-deps)
488524

489525
Bazel-deps allows you to generate bazel dependencies transitively for maven artifacts. Generally we don't want bazel-deps to fetch
@@ -711,23 +747,25 @@ In `WORKSPACE`, this `register_toolchains()` call must come before calling
711747
`scala_register_toolchains()` to ensure this toolchain takes precedence. The
712748
same exact call will also work in `MODULE.bazel`.
713749

714-
### Disabling Scala version validation when defining a custom Scala toolchain
750+
### Disabling builtin Scala toolchains when defining custom Scala toolchains
715751

716752
When [defining a 'scala_toolchain()' using custom compiler JARs](
717753
docs/scala_toolchain.md#b-defining-your-own-scala_toolchain), while using
718-
`scala_toolchains()` to instantiate other builtin toolchains, set
719-
`validate_scala_version = False`:
754+
`scala_toolchains()` to instantiate builtin toolchains (like the [protoc
755+
toolchain](#protoc)), set `scala = False`:
720756

721757
```py
722758
# WORKSPACE
723759
scala_toolchains(
724-
validate_scala_version = False,
760+
scala = False,
725761
# ...other toolchain parameters...
726762
)
727763
```
728764

729-
This differs from the previous API in two ways that avoided instantiating the
730-
default Scala compiler JAR repositories, and thus Scala version validation:
765+
This avoids instantiating the default Scala toolchain and compiler JAR
766+
repositories and a corresponding Scala version check, which may fail when
767+
defining a custom toolchain. It's equivalent to two ways in which the previous
768+
API avoided the same default behavior:
731769

732770
- Calling `scala_repositories(load_jar_deps = False)` would instantiate only
733771
other `rules_scala` dependency repos (`rules_java`, `protobuf`, etc.) and
@@ -896,6 +934,29 @@ supporting Bazel + MSVC builds per:
896934
Enable [protocol compiler toolchainization](#protoc) to fix broken Windows
897935
builds by avoiding `@com_google_protobuf//:protoc` recompilation.
898936

937+
### Minimum of `protobuf` v28
938+
939+
`rules_scala` requires at least `protobuf` v28, which contains the
940+
`bazel/common/proto_common.bzl` required for [protocol compiler
941+
toolchain](#protoc) support. This file appeared in v27, and no `ScalaPB` release
942+
supports `protobuf` v25.6, v26, or v27.
943+
944+
#### Using earlier `protobuf` versions
945+
946+
If you can't update to `protobuf` v28 or later right now, you will need to patch:
947+
948+
- `scala/toolchains.bzl`: remove `setup_protoc_toolchains()` references
949+
- `protoc/BUILD`: remove entirely
950+
951+
Then build using Bazel 7 and the following maximum versions of key dependencies:
952+
953+
| Dependency | Max compatible version | Reason |
954+
| :-: | :-: | :- |
955+
| `protobuf` | v25.5 | Maximum version supported by `ScalaPB` 0.11.17. |
956+
| `rules_java` | 7.12.4 | 8.x requires `protobuf` v27 and later. |
957+
| `rules_cc` | 0.0.9 | 0.0.10 requires Bazel 7 to define `CcSharedLibraryHintInfo`.<br/>0.0.13 requires at least `protobuf` v27.0. |
958+
| `ScalaPB` | 0.11.17<br/>(0.9.8 for Scala 2.11) | Later versions only support `protobuf` >= v28. |
959+
899960
### Embedded resource paths no longer begin with `external/<repo_name>`
900961

901962
[Any program compiled with an external repo asset in its 'resources' attribute
@@ -1022,9 +1083,8 @@ not out of the box, and may break at any time.
10221083

10231084
#### Maximum of `protobuf` v29
10241085

1025-
First of all, you _must_ use `protobuf` v29 or earlier. `rules_scala` now uses
1026-
v30 by default, which removes `py_proto_library` and other symbols that Bazel
1027-
6.5.0 requires:
1086+
You _must_ use `protobuf` v29 or earlier. `rules_scala` now uses v30 by default,
1087+
which removes `py_proto_library` and other symbols that Bazel 6.5.0 requires:
10281088

10291089
```txt
10301090
ERROR: Traceback (most recent call last):
@@ -1045,47 +1105,17 @@ ERROR: .../src/java/io/bazel/rulesscala/worker/BUILD:3:13:
10451105
and referenced by '//src/java/io/bazel/rulesscala/worker:worker'
10461106
```
10471107

1048-
#### Using a prebuilt `@com_google_protobuf//:protoc` or C++ compiler flags
1049-
1050-
Newer versions of `@com_google_protobuf//:protoc` fail to compile under Bazel
1051-
6.5.0 by default. You will have to choose one of the following approaches to
1052-
resolve this problem, if migrating to Bazel 7 or 8 isn't an immediate option.
1108+
#### Configuring the protocol compiler toolchain
10531109

1054-
You may use protocol compiler toolchainization with `protobuf` v29 to avoid
1055-
recompiling `protoc`. See the [Using a precompiled protocol compiler](#protoc)
1056-
section for details.
1110+
See [Using a prebuilt @com_google_protobuf//:protoc or C++ compiler
1111+
flags][protoc-opts] for protocol compiler configuration requirements.
10571112

1058-
Otherwise, per bazelbuild/rules_scala#1647, add the following flags to
1059-
`.bazelrc` to compile the newer `abseil-cpp` versions used by newer `protobuf`
1060-
versions:
1113+
[protoc-opts]: #using-a-prebuilt-com_google_protobufprotoc-or-c-compiler-flags
10611114

1062-
```txt
1063-
common --enable_platform_specific_config
1115+
#### Using older versions of `protobuf`
10641116

1065-
common:linux --cxxopt=-std=c++17
1066-
common:linux --host_cxxopt=-std=c++17
1067-
common:macos --cxxopt=-std=c++17
1068-
common:macos --host_cxxopt=-std=c++17
1069-
common:windows --cxxopt=/std=c++17
1070-
common:windows --host_cxxopt=/std=c++17
1071-
```
1072-
1073-
Note that this example uses `common:` config settings instead of `build:`. This
1074-
seems to prevent invalidating the action cache between `bazel` runs, which
1075-
improves performance.
1076-
1077-
#### Using older `protobuf` versions
1078-
1079-
If you have a dependency that requires `protobuf` version before v28, use the
1080-
following maximum versions of key dependencies. Note that no `ScalaPB` release
1081-
supports `protobuf` v25.6, v26, or v27.
1082-
1083-
| Dependency | Max compatible version | Reason |
1084-
| :-: | :-: | :- |
1085-
| `protobuf` | v25.5 | Maximum version supported by `ScalaPB` 0.11.17. |
1086-
| `rules_java` | 7.12.4 | 8.x requires `protobuf` v27 and later. |
1087-
| `rules_cc` | 0.0.9 | 0.0.10 requires Bazel 7 to define `CcSharedLibraryHintInfo`.<br/>0.0.13 requires at least `protobuf` v27.0. |
1088-
| `ScalaPB` | 0.11.17<br/>(0.9.8 for Scala 2.11) | Later versions only support `protobuf` >= v28. |
1117+
See [Minimum of protobuf v28](#minimum-of-protobuf-v28) for details on using
1118+
older versions of protobuf.
10891119

10901120
### `scala_proto` not supported for Scala 2.11
10911121

docs/scala_toolchain.md

+7-15
Original file line numberDiff line numberDiff line change
@@ -103,29 +103,21 @@ register_toolchains("//toolchains:my_scala_toolchain")
103103

104104
#### Step 3 (optional)
105105

106-
If you use `scala_toolchains()` to instantiate other builtin toolchains, set
107-
`validate_scala_version = False`:
106+
If you use `scala_toolchains()` to instantiate other builtin toolchains like the
107+
precompiled proto compiler toolchain, set `scala = False`:
108108

109109
```py
110110
# WORKSPACE
111111
scala_toolchains(
112-
validate_scala_version = False,
112+
scala = False,
113113
# ...other toolchain parameters...
114114
)
115115
```
116116

117-
This prevents `rules_scala` from checking that the Scala versions specified via
118-
`scala_config` match the `scala_version` values from
119-
`third_party/repositories/scala_*.bzl`.
120-
121-
`scala_toolchains()` uses these `scala_*.bzl` files to instantiate dependency
122-
JAR repositories required by the builtin toolchains. It always instantiates a
123-
default Scala toolchain, along with its compiler JAR repositories, since [most
124-
users expect this behavior][1633-comment]. (As always, calling
125-
`register_toolchains()` on your own toolchain in `MODULE.bazel`, or before
126-
`scala_register_toolchains()` in `WORKSPACE`, overrides the builtin toolchain.)
127-
128-
[1633-comment]: https://github.com/bazelbuild/rules_scala/pull/1633#discussion_r1834378901
117+
Otherwise, `scala_toolchains()` will try to instantiate a default Scala
118+
toolchain and its compiler JAR dependency repositories. The build will then fail
119+
if the configured Scala version doesn't match the `scala_version` value in
120+
the corresponding `third_party/repositories/scala_*.bzl` file.
129121

130122
## Configuration options
131123

protoc/BUILD

+15
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,24 @@
1+
load(
2+
"@com_google_protobuf//bazel/toolchains:proto_lang_toolchain.bzl",
3+
"proto_lang_toolchain",
4+
)
15
load(
26
"@com_google_protobuf//bazel/toolchains:proto_toolchain.bzl",
37
"proto_toolchain",
48
)
59
load("@rules_scala_toolchains//protoc:platforms.bzl", "PROTOC_PLATFORMS")
610

11+
toolchain_type(
12+
name = "toolchain_type",
13+
visibility = ["//visibility:public"],
14+
)
15+
16+
proto_lang_toolchain(
17+
name = "protoc_scala_toolchain",
18+
command_line = "unused-because-we-pass-protoc-to-scalapb",
19+
toolchain_type = ":toolchain_type",
20+
)
21+
722
[
823
proto_toolchain(
924
name = platform,

protoc/private/protoc_toolchain.bzl

-12
Original file line numberDiff line numberDiff line change
@@ -81,18 +81,6 @@ def setup_protoc_toolchains(repository_ctx, package):
8181

8282
_PROTOC_TOOLCHAIN_BUILD = """
8383
load(":platforms.bzl", "PROTOC_PLATFORMS")
84-
load("@rules_proto//proto:defs.bzl", "proto_lang_toolchain")
85-
86-
toolchain_type(
87-
name = "toolchain_type",
88-
visibility = ["//visibility:public"],
89-
)
90-
91-
proto_lang_toolchain(
92-
name = "protoc_scala_toolchain",
93-
command_line = "unused-because-we-pass-protoc-to-scalapb",
94-
toolchain_type = ":toolchain_type",
95-
)
9684
9785
[
9886
alias(

scala/toolchains.bzl

+14-7
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ def scala_toolchains(
3636
validate_scala_version = True,
3737
scala_compiler_srcjars = {},
3838
protoc_platforms = [],
39+
scala = True,
3940
scalatest = False,
4041
junit = False,
4142
specs2 = False,
@@ -74,8 +75,9 @@ def scala_toolchains(
7475
`third_party/repositories/scala_*.bzl` file matching the Scala
7576
version.
7677
fetch_sources: whether to download dependency source jars
77-
validate_scala_version: whether to check if the configured Scala version
78-
matches the default version supported by rules_scala
78+
validate_scala_version: Whether to check if the configured Scala
79+
versions matches the default versions supported by rules_scala. Only
80+
takes effect if `scala` is `True`.
7981
scala_compiler_srcjars: optional dictionary of Scala version string to
8082
compiler srcjar metadata dictionaries containing:
8183
- exactly one "label", "url", or "urls" key
@@ -86,6 +88,8 @@ def scala_toolchains(
8688
unspecified, will use the identifier matching the `HOST_CONSTRAINTS`
8789
from `@platforms//host:constraints.bzl`. Only takes effect when
8890
`--incompatible_enable_proto_toolchain_resolution` is `True`.
91+
scala: whether to instantiate default Scala toolchains for configured
92+
Scala versions
8993
scalatest: whether to instantiate the ScalaTest toolchain
9094
junit: whether to instantiate the JUnit toolchain
9195
specs2: whether to instantiate the Specs2 JUnit toolchain
@@ -152,10 +156,13 @@ def scala_toolchains(
152156
})
153157

154158
for scala_version in SCALA_VERSIONS:
155-
version_specific_artifact_ids = {
156-
id: fetch_sources
157-
for id in scala_version_artifact_ids(scala_version)
158-
}
159+
version_specific_artifact_ids = {}
160+
161+
if scala:
162+
version_specific_artifact_ids.update({
163+
id: fetch_sources
164+
for id in scala_version_artifact_ids(scala_version)
165+
})
159166

160167
if scala_proto:
161168
version_specific_artifact_ids.update({
@@ -180,7 +187,7 @@ def scala_toolchains(
180187
fetch_sources_by_id = all_artifacts,
181188
# Note the internal macro parameter misspells "overriden".
182189
overriden_artifacts = overridden_artifacts,
183-
validate_scala_version = validate_scala_version,
190+
validate_scala_version = (scala and validate_scala_version),
184191
)
185192

186193
scala_toolchains_repo(

scala/toolchains_repo.bzl

-3
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,6 @@ def _scala_toolchains_repo_impl(repository_ctx):
7878
if repo_attr.scalafmt:
7979
toolchains["scalafmt"] = _SCALAFMT_TOOLCHAIN_BUILD
8080

81-
if len(toolchains) == 0:
82-
fail("no toolchains specified")
83-
8481
for pkg, build in toolchains.items():
8582
repository_ctx.file(
8683
pkg + "/BUILD",

scala_proto/scala_proto_toolchain.bzl

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,14 @@ load(
66
load("@com_google_protobuf//bazel/common:proto_common.bzl", "proto_common")
77
load("@rules_proto//proto:proto_common.bzl", "toolchains")
88

9-
_TOOLCHAIN_TYPE = Label("@rules_scala_toolchains//protoc:toolchain_type")
9+
_TOOLCHAIN_TYPE = Label("//protoc:toolchain_type")
1010

1111
# Inspired by: https://github.com/protocolbuffers/protobuf/pull/19679
1212
def _protoc(ctx):
1313
if proto_common.INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION:
1414
toolchain = ctx.toolchains[_TOOLCHAIN_TYPE]
1515
if not toolchain:
16-
fail("Protocol compiler toolchain could not be resolved.")
16+
fail("Couldn't resolve protocol compiler for %s" % _TOOLCHAIN_TYPE)
1717
return toolchain.proto.proto_compiler.executable
1818
else:
1919
return ctx.attr.protoc[DefaultInfo].files_to_run.executable

scripts/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -116,3 +116,11 @@ The [bazelisk](https://github.com/bazelbuild/bazelisk) wrapper for Bazel uses
116116
`.bazelversion` files select a Bazel version. While `USE_BAZEL_VERSION` can
117117
also override the Bazel version, keeping the `.bazelversion` files synchronized
118118
helps avoid suprises when not using `USE_BAZEL_VERSION`.
119+
120+
## [`update_protoc_integrity`](./update_protoc_integrity.py)
121+
122+
Updates `protoc/private/protoc_integrity.bzl`.
123+
124+
Upon a new release of
125+
[`protocolbuffers/protobuf`](https://github.com/protocolbuffers/protobuf/releases)
126+
add the new version to the `PROTOC_VERSIONS` at the top of this file and run it.

0 commit comments

Comments
 (0)