Skip to content

Commit f6ec270

Browse files
authored
Pin ML training example to x86_64-linux (#5)
It would be nice to support other platforms as well here, but the setup is too complex for what we're trying to illustrate and without CI it's also too hard to ensure that it's working as expected on all platforms.
1 parent dc8bea7 commit f6ec270

7 files changed

Lines changed: 21 additions & 1347 deletions

File tree

finetuning_on_cpu/.bazelrc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,8 @@ build --experimental_profile_include_target_label
2525
build --noexperimental_profile_include_primary_output
2626

2727
# Use the container image specified in `platforms/BUILD.bazel`.
28-
build:remote --extra_execution_platforms=@//platforms:python-remote
29-
build:remote --extra_toolchains=@zig_sdk//toolchain:linux_amd64_musl
28+
build --extra_execution_platforms=@//platforms:python-remote
29+
build --extra_toolchains=@zig_sdk//toolchain:linux_amd64_musl
3030

3131
# Add user specific flags.
3232
try-import %workspace%/user.bazelrc

finetuning_on_cpu/BUILD.bazel

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,7 @@
11
load("@rules_python//python/uv:lock.bzl", "lock")
22

3-
exports_files(
4-
["python_provider.sh"],
5-
visibility = ["//visibility:public"],
3+
lock(
4+
name = "requirements",
5+
srcs = ["pyproject.toml"],
6+
out = "requirements.lock",
67
)
7-
8-
[
9-
lock(
10-
name = "requirements_{}".format(os),
11-
srcs = ["pyproject.toml"],
12-
out = "requirements_{}.lock".format(os),
13-
build_constraints = ["@platforms//os:{}".format(os)],
14-
)
15-
for os in [
16-
"linux",
17-
"macos",
18-
]
19-
]

finetuning_on_cpu/MODULE.bazel

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,8 @@ pip.parse(
2323
},
2424
hub_name = "pypi",
2525
python_version = "3.13",
26-
# This file may be regenerated via `bazel run //:requirements_macos.update`.
27-
requirements_darwin = "//:requirements_macos.lock",
28-
# This file may be regenerated via `bazel run //:requirements_linux.update`.
29-
requirements_linux = "//:requirements_linux.lock",
26+
# This file may be regenerated via `bazel run //:requirements.update`.
27+
requirements_linux = "//:requirements.lock",
3028
)
3129
use_repo(pip, "pypi")
3230

finetuning_on_cpu/README.md

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22

33
An example project for training a machine learning model with Bazel.
44

5+
> [!NOTE]
6+
> This example only supports `x86_64-linux`.
7+
58
## ❄️ Setup
69

710
1. Either have Bazelisk installed or enter the nix flake from this repository.
@@ -31,7 +34,7 @@ Now run a remote invocation. The first time you do this it'll be slow (~15min)
3134
as the runner needs to spin up and fetch the various toolchains:
3235
3336
```bash
34-
bazel test training --config=remote --test_output=all
37+
bazel test training --test_output=all
3538
```
3639
3740
Run the above command a second time. Since the runner is now warm it'll
@@ -44,15 +47,9 @@ The dependencies for this project are tracked in `pyproject.toml` and locked in
4447
regenerate the lockfiles:
4548

4649
```bash
47-
# On linux
48-
bazel run //:requirements_linux.update
49-
50-
# On macos
51-
bazel run //:requirements_macos.update
50+
bazel run //:requirements.update
5251
```
5352

54-
On darwin you might have to disable remote execution temporarily for this.
55-
5653
## 🧹 Fixing formatting and linting
5754

5855
Use the flake if you develop on this code. Don't use local ruff installations as
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# This file was autogenerated by uv via the following command:
2-
# bazel run //:requirements_linux.update
2+
# bazel run //:requirements.update
33
accelerate==1.6.0 \
44
--hash=sha256:1aee717d3d3735ad6d09710a7c26990ee4652b79b4e93df46551551b5227c2aa \
55
--hash=sha256:28c1ef1846e690944f98b68dc7b8bb6c51d032d45e85dcbb3adb0c8b99dffb32
@@ -1015,9 +1015,9 @@ safetensors==0.5.3 \
10151015
# via
10161016
# accelerate
10171017
# transformers
1018-
setuptools==80.4.0 \
1019-
--hash=sha256:5a78f61820bc088c8e4add52932ae6b8cf423da2aff268c23f813cfbb13b4006 \
1020-
--hash=sha256:6cdc8cb9a7d590b237dbe4493614a9b75d0559b888047c1f67d49ba50fc3edb2
1018+
setuptools==80.7.0 \
1019+
--hash=sha256:d3b6ba519cc2b1e0b14195ef1a9560f675b1f873dc0d42969e139b2b58e763d6 \
1020+
--hash=sha256:e8fb83ffb2011d786ab2a38d63c33c261cc5862522a6b57ac8f7e7ffeff2336b
10211021
# via torch
10221022
six==1.17.0 \
10231023
--hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \

0 commit comments

Comments
 (0)