You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 47e1176
Browse filesBrowse the repository at this point in the historyBrowse files
docs: self-hosted runners configure rust-ci through the job env
docs/runner-services.md becomes docs/self-hosted.md: the variables each action reads, the job-started hook that hands them to every job including container jobs, the other routes, the trust notes and the limits.
README and CHANGELOG follow the actions; the private transfer measurement is gone.
Copy file name to clipboardExpand all lines: CHANGELOG.md
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,18 +7,17 @@ The format follows [Keep a Changelog](https://keepachangelog.com); releases are
7
7
8
8
### Added
9
9
10
-
-`sccache`: installs a pinned sccache (version + per-arch sha256) as `RUSTC_WRAPPER`, backend-configured by the host's `.rust-ci-env` file beside the work directory (`SCCACHE_*` lines, allowlist-checked into the job env).
11
-
`"auto"` no-ops without the file, so one workflow runs unchanged on hosted and self-hosted runners; the contract for host operators is docs/runner-services.md.
10
+
-`sccache`: installs a pinned sccache (version + per-arch sha256) as `RUSTC_WRAPPER`; the backend is whatever `SCCACHE_*` the job environment carries, and `"auto"` no-ops without one, so one workflow runs unchanged on hosted and self-hosted runners.
12
11
`mode: gha` targets GitHub's cache service instead; an explicit opt-in, since it exports the masked Actions runtime token into the job env and spends the repository's cache pool.
13
12
-`sccache-stats`: hit/miss/request facts as `cache.sccache.*` rows in timing-report's Cache section, run as a late job step; never fails a job.
14
-
-`crates-mirror`: writes the crates-io source replacement into `$CARGO_HOME/config.toml` from the host's `RUST_CI_CRATES_MIRROR`URL (source replacement is config-file-only, cargo#5416).
13
+
-`crates-mirror`: writes the crates-io source replacement into `$CARGO_HOME/config.toml` from the `url` input or `RUST_CI_CRATES_MIRROR`in the job environment (source replacement is config-file-only, cargo#5416).
15
14
Lockfile checksums keep pinning canonical crates.io hashes, so the mirror is an availability dependency, not a trust dependency.
16
15
-`cut-release`: stamps a `CITATION.cff` into the release commit, where the repository keeps one: top-level `version`, and `date-released` where the file has one, both on the cut's date.
17
16
Keys match whole-line, so `cff-version` survives; a missing file skips and `citation: ""` disables.
18
17
-`rust-cache`: opt-in `local-target` (requires `cache-target`) keeps `target/` on the runner between jobs instead of transferring it, by pointing `CARGO_TARGET_DIR` at the runner's work tree, outside the workspace where `git clean -ffdx` cannot reach it.
19
-
`"auto"` activates only when the host leaves a `.rust-ci-local-target` marker, so one workflow runs unchanged on hosted and self-hosted runners.
20
-
Measured on a self-hosted runner: 525 s of a 20.5-minute job spent transferring 9.26 GB.
18
+
`"auto"` follows `RUST_CI_LOCAL_TARGET=1` in the job environment, so one workflow runs unchanged on hosted and self-hosted runners.
21
19
Consumers must read `CARGO_TARGET_DIR` rather than assume `./target`.
20
+
-`docs/self-hosted.md`: how a host hands these variables to every job through the runner's job-started hook, which reaches `container:` jobs too.
Copy file name to clipboardExpand all lines: README.md
+14-15Lines changed: 14 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -244,12 +244,11 @@ It sets `CARGO_TARGET_DIR` to a directory in the runner's work tree, **outside t
244
244
A target directory inside the workspace cannot survive a checkout, however persistent the disk is.
245
245
**Consumers must read `CARGO_TARGET_DIR` rather than assume `./target`.**
246
246
247
-
`"auto"`activates only when the host leaves a `.rust-ci-local-target` marker beside the runner's work directory, so the same workflow runs unchanged on a hosted runner (no marker, transfer as before) and on a self-hosted one (marker, no transfer).
248
-
A job env `RUST_CI_LOCAL_TARGET=1` counts as the marker, for a workflow that controls its runner but not the host.
249
-
A marker rather than an env var because the runner does not forward its own environment into a job container, while the work tree is bind-mounted into it.
247
+
`"auto"`activates when the job environment carries `RUST_CI_LOCAL_TARGET=1`, which a host whose work tree persists provides ([docs/self-hosted.md](docs/self-hosted.md)), so the same workflow runs unchanged on a hosted runner (transfer as before) and on a prepared self-hosted one (no transfer).
248
+
Ephemeral runners must not set it: the directory dies with the instance.
250
249
251
250
Only the target entry is affected; the registry entry still uses `actions/cache`, since `$CARGO_HOME` usually lives in the toolchain image rather than the work tree.
252
-
`rust-cache-save`needs no flag, because this mode hands over no key.
251
+
`rust-cache-save`needs no flag, because this mode hands over no key and its steps skip.
253
252
The directory sits at the work root, is shared per prefix across every repo on the runner, and is never pruned: the host owns it and can wipe `.rust-ci-target` to reclaim disk.
254
253
255
254
```yaml
@@ -283,23 +282,22 @@ The restore-everywhere/save-on-main pattern: PR jobs omit this action (or pass `
283
282
284
283
### `sccache`
285
284
286
-
Installs a pinned [sccache](https://github.com/mozilla/sccache) as `RUSTC_WRAPPER`, backend configured by the **runner's host**: every `SCCACHE_*` line of the host's `.rust-ci-env` file is allowlist-checked and imported into the job environment ([docs/runner-services.md](docs/runner-services.md) is the contract).
287
-
The action is backend-agnostic; the host names the backend (WebDAV, S3, Redis, a directory).
288
-
289
-
`"auto"` activates only when that file offers `SCCACHE_*` configuration, so the same workflow runs unchanged on hosted and prepared self-hosted runners; a job env `RUST_CI_SCCACHE=1` counts as the offer.
285
+
Installs a pinned [sccache](https://github.com/mozilla/sccache) as `RUSTC_WRAPPER`; the backend is whatever `SCCACHE_*` the job environment carries, and sccache reads it itself, so every backend it supports works (WebDAV, S3, Redis, a directory).
286
+
`"auto"` activates when a non-empty `SCCACHE_*` variable is set, so the same workflow runs unchanged on hosted and self-hosted runners; [docs/self-hosted.md](docs/self-hosted.md) shows how a host provides the variables.
290
287
The binary is fetched by pinned version **and sha256** (per-arch, Linux musl) into `$RUNNER_TEMP`, so the absolute `RUSTC_WRAPPER` path works inside job containers without touching `PATH`.
291
288
`CARGO_INCREMENTAL` defaults to 0; an explicit consumer value is obeyed.
292
289
Linking (`bin`/`dylib`/`cdylib`/proc-macro crates) and build-script execution stay uncached; the wins are the `lib` compiles, shared across every job on the same backend.
290
+
The sealed Docker actions are untouched: they pin `CARGO_HOME`, forward `CARGO_.*` only and build offline.
293
291
294
-
`mode: gha` uses **GitHub's own cache service** as the backend, for runners without a host file (hosted ones).
295
-
Explicit on purpose, never part of `auto`: it exports `ACTIONS_RUNTIME_TOKEN` (masked) and `ACTIONS_RESULTS_URL` into the job environment, and it spends the repository's Actions cache pool (10 GB free, expandable paid, 7-day-unused eviction).
292
+
`mode: gha` uses **GitHub's own cache service** as the backend, the choice for hosted runners.
293
+
Explicit on purpose, never part of `auto`: it exports `ACTIONS_RUNTIME_TOKEN` (masked) and `ACTIONS_RESULTS_URL` into the job environment, and it spends the repository's Actions cache pool (10 GB per repository by default, entries removed after 7 days without access).
296
294
GitHub scopes cache writes per ref, so pull-request objects never reach the default branch's scope; cross-PR sharing therefore only flows through objects a default-branch build wrote.
297
-
Every object is a network round-trip, so a host-local backend beats it wherever one exists.
@@ -316,14 +314,15 @@ A no-op when sccache is inactive; statistics never fail a job.
316
314
### `crates-mirror`
317
315
318
316
Routes crates-io through a runner-local pull-through mirror: cargo's source replacement is config-file-only (cargo#5416), so this writes it into `$CARGO_HOME/config.toml`.
319
-
The URL comes from the host's `.rust-ci-env` (`RUST_CI_CRATES_MIRROR=sparse+http://…/`), a job env of the same name, or the `url` input; `"auto"` no-ops without one, so hosted runners keep fetching from crates.io directly.
317
+
The URL comes from the `url` input or from `RUST_CI_CRATES_MIRROR` in the job environment; `"auto"` no-ops without one, so hosted runners keep fetching from crates.io directly.
320
318
`Cargo.lock`keeps the canonical crates.io checksums under source replacement and cargo verifies every download against them, so the mirror is an availability dependency, never a trust dependency.
321
319
Place it before the first dependency resolution; an existing crates-io replacement in `$CARGO_HOME/config.toml` is refused rather than fought.
A self-hosted runner can offer per-host services to the jobs it runs: a compile-cache backend for `sccache`, a crates.io pull-through for `crates-mirror`, and a work tree that persists between jobs for `rust-cache`.
4
+
The actions read that offer from the job environment and from their inputs, nothing else.
5
+
6
+
## What the actions read
7
+
8
+
| Variable | Read by | Meaning |
9
+
| --- | --- | --- |
10
+
|`SCCACHE_*`|`sccache`| sccache's own backend configuration, for example `SCCACHE_WEBDAV_ENDPOINT`, `SCCACHE_BUCKET` with `SCCACHE_ENDPOINT`, `SCCACHE_REDIS_ENDPOINT` or `SCCACHE_DIR`; credentials such as `AWS_ACCESS_KEY_ID` are ordinary variables next to them. |
11
+
|`RUST_CI_CRATES_MIRROR`|`crates-mirror`| A `sparse+http(s)://.../` registry URL of a crates.io pull-through, written into `$CARGO_HOME/config.toml` as the crates-io source replacement. |
12
+
|`RUST_CI_LOCAL_TARGET=1`|`rust-cache` with `local-target: auto`| The runner keeps its work tree between jobs, so `target/` stays on disk instead of travelling through the cache. Never set it on ephemeral runners. |
13
+
14
+
One rule for every `auto`: the mode activates when its variable is present and non-empty, and does nothing otherwise.
15
+
Hosted runners carry none of these variables, so one workflow runs unchanged on both.
16
+
17
+
## How a host sets them
18
+
19
+
Use the runner's job-started hook.
20
+
It runs on the host at the start of every job, before any job container is created, and the lines it appends to `$GITHUB_ENV` become the job's environment.
21
+
Every later step sees them, including the steps of a `container:` job, because the runner passes the job environment into each `docker exec`.
# A credential is masked before it is written: echo "::add-mask::$secret"
31
+
```
32
+
33
+
Point the runner at the script with `ACTIONS_RUNNER_HOOK_JOB_STARTED=/usr/local/lib/ci/job-started.sh`, either in the `.env` file of the runner directory or in the environment of the runner's service, and restart the runner.
34
+
GitHub documents the hook under "Running scripts before or after a job".
35
+
36
+
Other routes, for completeness:
37
+
38
+
- A workflow may set the same variables itself, from `env:`, `vars.*` or `container.env`; an unset variable yields an empty value, which the actions treat as absent.
39
+
- Variables in the runner's `.env` alone reach the steps of bare jobs, never the steps of a `container:` job, and never the `env` expression context.
40
+
-`container.options: -e NAME` copies a variable from the runner's process environment into one workflow's container.
41
+
-`runner.environment` is `github-hosted` or `self-hosted`, so a fleet whose self-hosted runners keep their work tree can pass `local-target: ${{ runner.environment == 'self-hosted' }}` and pick `mode: gha` for sccache on hosted runners.
42
+
43
+
## Trust
44
+
45
+
Everything the hook exports is readable by every job the runner executes, third-party actions included.
46
+
Scope credentials to the cache backend alone, on a network the runners can reach and the world cannot.
47
+
A compile cache written by untrusted jobs is a poisoning surface: keep release builds cache-off, or give trust tiers separate backends.
48
+
The crates mirror serves under `Cargo.lock` checksum protection, so its requirement is availability, not trust.
49
+
50
+
## Limits
51
+
52
+
The sealed Docker actions pin `CARGO_HOME`, forward `CARGO_.*` only and build with `--network=none`, so neither sccache nor the mirror reaches them; their compile cache is the `target-dir` mount.
53
+
Inside a `container:` job use a network backend: a directory on the host is not visible there.
54
+
sccache runs one server per host and port, and the action restarts it with the current job's configuration; a runner that executes jobs concurrently sets `SCCACHE_SERVER_PORT` per job.
0 commit comments