Skip to content

Commit c9a30c2

Browse files
mblandmbland
authored
Convert WORKSPACE to MODULE.bazel, update README (#312)
* Enable bzlmod, update README, fix Swift build Rather than mess with the WORKSPACE rules, the shortest path to fixing `blaze build //swift:tests` appeared to be introducing MODULE.bazel. MODULE.bazel, a.k.a. bzlmod, appears to be the new hotness, so I'll also try updating the other builds to use it as well. - https://bazel.build/external/migration - https://bazel.build/external/overview#workspace-shortcomings - bazelbuild/bazel#18958 * Convert `bazel build //python/...` to MODULE.bazel * Convert `bazel build //csharp/...` to MODULE.bazel * Convert //go/..., gazelle to MODULE.bazel * Migrate protobuf, rules_proto to MODULE.bazel Also bumps to: - protobuf: 23.1 - rules_proto: 6.0.0-rc2 * Add rules_java, rules_jvm_external to MODULE.bazel rules_java was an implicit dependency before. Bumps: - rules_java: 7.4.0 => 7.5.0. - rules_jvm_external: 4.4.2 => 6.0 The rules_jvm_external docs describe using `maven.install` in MODULE.bazel as a replacement for `maven_install` in WORKSPACE: - https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/bzlmod.md However, our current `maven_install` depends on the definition of IO_GRPC_GRPC_JAVA_ARTIFACTS from @io_grpc_grpc_java. I'll attempt that migration next. * MODULE.bazel: skylib, rules_proto_grpc, protobuf `bazel build //java/...` and `bazel test //java/...` both work with these changes. * Move grpc-java to MODULE.bazel, bump to 1.62.2 grpc-java only got MODULE.bazel support as of this most recent version: - grpc/grpc-java#11046 - bazelbuild/bazel-central-registry#1699 This grpc-java version bump exposed two issues that are fixed in this commit: 1. The //java/com/engflow/notificationqueue:client target dependency on @maven//:io_netty_netty_handler broke. The original WORKSPACE import of io_grpc_grpc_java imported this dependency directly by passing IO_GRPC_GRPC_JAVA_ARTIFACTS directly to `maven_install`. The `maven.install` call from grpc/grpc-java's MODULE.bazel sets `strict_visibility = True`. Somehow the other dependencies registered by grpc-java's MODULE.bazel are accessible to notificationqueue:client, but netty-handler isn't. The solution was to add the `io.netty:netty-handler:4.1.100.Final` artifact to the `maven.install` call in this project's MODULE.bazel. It doesn't seem an optimal solution, but it works for now. 2. grpc/grpc-java removed `io.grpc.stub.MetadataUtils.attachHeaders()` in grpc/grpc-java#10443. This caused notificationqueue:client to fail to compile, but that PR revealed the replacement for the deprecated `attachHeaders` call. This commit applies that replacement. * Move googleapis to MODULE.bazel This appears to be a fairly recent development, and isn't yet 100% officially supported in the googleapis/googleapis repo, but it works: - googleapis/googleapis#855 - bazelbuild/bazel-central-registry#1699 * Move rules_kotlin to MODULE.bazel, bump to v1.9.5 * Move rules_perl to MODULE.bazel, bump to 0.2.0 There's actually a 0.2.1 release, but it hasn't been pushed to https://registry.bazel.build/ yet. * Add rules_scala GitHub issue links rules_scala hasn't migrated to bzlmod yet, but discussion is underway. These links will help track its progress. * Migrate aspect_rules_ts to MODULE.bazel * Replace deps.bzl with go_deps This enables `bazel {build,test} //infra/...` to succeed using MODULE.bazel. See: - https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md#specifying-external-dependencies * Set test sizes to "small" as appropriate This eliminates warnings that these tests are sized too big, since the default is "medium". * Move http_{file,archive} calls to MODULE.bazel This moves the following http_file calls: - emacs - ubuntu_20.04_1.3GB and the following http_archive calls: - com_engflow_engflowapis - io_abseil_py * Update //platform rules, remove dotnet constraints The //dotnet/toolchain constraint was causing a Bazel error saying that the @@rules_dotnet//dontent/toolchain package didn't exist. Removing this constraint allowed remote execution to succeed anyway. * Update README, explain swift incompatibility Most of these changes are cosmetic, with the notable exception of the explantion behind the inability to build //swift remotely. Also added a `git` command to ignore python/requirements_lock.txt per: - https://stackoverflow.com/a/73720550
1 parent abfc7d4 commit c9a30c2

File tree

21 files changed

+15928
-422
lines changed

21 files changed

+15928
-422
lines changed

‎.bazelrc

-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# Options common for all EngFlow remote configurations.
22
common --@aspect_rules_ts//ts:skipLibCheck=honor_tsconfig
33
common --@aspect_rules_ts//ts:default_to_tsc_transpiler
4-
common --noenable_bzlmod
54
common --noincompatible_check_sharding_support
65

76
build:engflow_common --jobs=40

‎BUILD

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
load("@bazel_gazelle//:def.bzl", "gazelle")
1+
load("@gazelle//:def.bzl", "gazelle")
22

33
gazelle(name = "gazelle")
44

‎MODULE.bazel

+170
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
"""EngFlow example project"""
2+
3+
module(name = "engflow-example", version = "0.0.0")
4+
http_file = use_repo_rule(
5+
"@bazel_tools//tools/build_defs/repo:http.bzl", "http_file"
6+
)
7+
http_archive = use_repo_rule(
8+
"@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive"
9+
)
10+
11+
# Some file dependencies
12+
http_file(
13+
name = "emacs",
14+
sha256 = "1439bf7f24e5769f35601dbf332e74dfc07634da6b1e9500af67188a92340a28",
15+
urls = [
16+
"https://storage.googleapis.com/engflow-tools-public/emacs-28.1.tar.gz",
17+
"https://mirror.its.dal.ca/gnu/emacs/emacs-28.1.tar.gz",
18+
"https://mirrors.kernel.org/gnu/emacs/emacs-28.1.tar.gz",
19+
],
20+
)
21+
22+
http_file(
23+
name = "ubuntu_20.04_1.3GB",
24+
sha256 = "5035be37a7e9abbdc09f0d257f3e33416c1a0fb322ba860d42d74aa75c3468d4",
25+
urls = [
26+
"https://storage.googleapis.com/engflow-tools-public/ubuntu-20.04.5-live-server-amd64.iso",
27+
"https://mirror.math.princeton.edu/pub/ubuntu-iso/focal/ubuntu-20.04.5-live-server-amd64.iso",
28+
"https://mirror.pit.teraswitch.com/ubuntu-releases/focal/ubuntu-20.04.5-live-server-amd64.iso",
29+
],
30+
)
31+
32+
http_archive(
33+
name = "com_engflow_engflowapis",
34+
sha256 = "8721f7a0ec52c5bc120119aac090eedd671ca3b708652f88b82b44bea2b6c278",
35+
strip_prefix = "engflowapis-44fcd39598f223e8e5f6c7cbf2f73c870b2a6341",
36+
urls = [
37+
"https://github.com/EngFlow/engflowapis/archive/44fcd39598f223e8e5f6c7cbf2f73c870b2a6341.zip",
38+
],
39+
)
40+
41+
# Abseil Python can be imported through pip_import, but it has native Bazel support too.
42+
# bzlmod: https://github.com/abseil/abseil-py/issues/263
43+
http_archive(
44+
name = "io_abseil_py",
45+
sha256 = "8a3d0830e4eb4f66c4fa907c06edf6ce1c719ced811a12e26d9d3162f8471758",
46+
strip_prefix = "abseil-py-2.1.0",
47+
url = "https://github.com/abseil/abseil-py/archive/refs/tags/v2.1.0.tar.gz",
48+
)
49+
50+
bazel_dep(name = "bazel_skylib", version = "1.5.0")
51+
bazel_dep(
52+
name = "protobuf",
53+
version = "23.1",
54+
repo_name = "com_google_protobuf",
55+
)
56+
57+
# https://github.com/googleapis/googleapis/pull/855
58+
# https://github.com/bazelbuild/bazel-central-registry/pull/1699
59+
bazel_dep(
60+
name = "googleapis",
61+
version = "0.0.0-20240326-1c8d509c5",
62+
repo_name = "com_google_googleapis",
63+
)
64+
65+
bazel_dep(name = "rules_python", version = "0.31.0")
66+
67+
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
68+
69+
pip.parse(
70+
hub_name = "pip",
71+
python_version = "3.11",
72+
requirements_lock = "//python:requirements_lock.txt",
73+
)
74+
75+
use_repo(pip, "pip")
76+
77+
bazel_dep(name = "rules_proto", version = "6.0.0-rc2")
78+
79+
bazel_dep(
80+
name = "apple_support",
81+
version = "1.15.1",
82+
repo_name = "build_bazel_apple_support",
83+
)
84+
85+
bazel_dep(name = "rules_dotnet", version = "0.15.1")
86+
87+
dotnet = use_extension("@rules_dotnet//dotnet:extensions.bzl", "dotnet")
88+
dotnet.toolchain(dotnet_version = "8.0.200")
89+
use_repo(dotnet, "dotnet_toolchains")
90+
91+
register_toolchains("@dotnet_toolchains//:all")
92+
93+
# https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md
94+
bazel_dep(name = "rules_go", version = "0.46.0")
95+
bazel_dep(name = "gazelle", version = "0.36.0")
96+
97+
GO_PLATFORMS = [
98+
("darwin", "amd64"),
99+
("darwin", "arm64"),
100+
("linux", "amd64"),
101+
("linux", "arm64"),
102+
("windows", "amd64"),
103+
]
104+
105+
GO_VERSION = "1.21.6"
106+
107+
go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")
108+
109+
[
110+
go_sdk.download(
111+
name = "go_{}_{}".format(goos, goarch),
112+
goarch = goarch,
113+
goos = goos,
114+
version = GO_VERSION,
115+
)
116+
for goos, goarch in GO_PLATFORMS
117+
]
118+
119+
go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
120+
go_deps.from_file(go_mod = "//:go.mod")
121+
122+
use_repo(go_deps, "com_github_google_go_cmp")
123+
124+
bazel_dep(name = "rules_java", version = "7.5.0")
125+
126+
# https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/bzlmod.md
127+
bazel_dep(name = "rules_jvm_external", version = "6.0")
128+
bazel_dep(
129+
name = "grpc-java",
130+
version = "1.62.2",
131+
repo_name = "io_grpc_grpc_java",
132+
)
133+
134+
# Loads rules required to compile proto files
135+
bazel_dep(name = "rules_proto_grpc", version = "5.0.0-alpha2")
136+
137+
maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")
138+
maven.install(
139+
artifacts = [
140+
"commons-cli:commons-cli:1.5.0",
141+
"com.google.oauth-client:google-oauth-client:1.34.1",
142+
"io.netty:netty-handler:4.1.100.Final",
143+
],
144+
strict_visibility = True,
145+
repositories = [
146+
"https://repo.maven.apache.org/maven2/",
147+
],
148+
)
149+
use_repo(maven, "maven")
150+
151+
bazel_dep(name = "rules_kotlin", version = "1.9.5")
152+
bazel_dep(name = "rules_perl", version = "0.2.0")
153+
154+
bazel_dep(
155+
name = "rules_swift",
156+
version = "1.18.0",
157+
repo_name = "build_bazel_rules_swift"
158+
)
159+
160+
bazel_dep(name = "aspect_rules_ts", version = "2.2.0")
161+
162+
rules_ts_ext = use_extension(
163+
"@aspect_rules_ts//ts:extensions.bzl",
164+
"ext",
165+
dev_dependency = True,
166+
)
167+
168+
rules_ts_ext.deps()
169+
170+
use_repo(rules_ts_ext, "npm_typescript")

0 commit comments

Comments
 (0)