Skip to content

Commit 0772508

Browse files
authored
osdc/hf-cache: drop --use-mmap, keep buffer-size reduction (#913)
Remove `--use-mmap` from the rclone mount, keeping the `--buffer-size 4M` read-ahead cut. Both were added together in #907; we're rolling out one lever at a time, starting with the buffer-size reduction. I want to make one change at a time for this tricky part to make sure that the cause and effect can be seen clearly
1 parent 80e5627 commit 0772508

2 files changed

Lines changed: 4 additions & 7 deletions

File tree

osdc/modules/hf-cache/README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@ rclone's memory is **reserved** (`request == limit`) and **tiered by GPU count**
2121
To keep RSS inside those reserves the mount runs with `--buffer-size 4M` (a 4x cut
2222
from rclone's 16Mi-per-open-file default read-ahead — the dominant RSS driver when
2323
a model opens many shards; kept non-zero so cold reads retain some prefetch, with
24-
`--vfs-cache-mode full` serving the rest from the on-disk cache) and `--use-mmap`
25-
(frees buffers back to the OS). Without them, concurrent large-model (safetensors)
26-
reads OOM-kill rclone, and the dead FUSE mount surfaces as a spurious
27-
`LocalEntryNotFoundError` in the job.
24+
`--vfs-cache-mode full` serving the rest from the on-disk cache). Together with the
25+
per-tier `GOMEMLIMIT` (see `deploy.sh`), this keeps concurrent large-model
26+
(safetensors) reads from OOM-killing rclone; a dead FUSE mount otherwise surfaces as
27+
a spurious `LocalEntryNotFoundError` in the job.
2828

2929
**Writes are gated by GitHub OIDC, not by the mount.** Job pods can't write the
3030
cache (read-only mount, read-only IRSA). On `ci-refresh-hf-cache` runs, the

osdc/modules/hf-cache/kubernetes/mount-daemonset.yaml.tpl

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -127,8 +127,6 @@ spec:
127127
# model opens many shards at once. Kept non-zero so cold reads
128128
# retain some prefetch (vfs-cache-mode full serves the rest from
129129
# the on-disk cache).
130-
# --use-mmap returns freed buffers to the OS instead of Go
131-
# retaining them as process RSS.
132130
rclone mount \
133131
":s3,provider=AWS,env_auth=true,region=__REGION__:__BUCKET__" \
134132
"$MOUNT" \
@@ -142,7 +140,6 @@ spec:
142140
--vfs-cache-max-age 24h \
143141
--vfs-read-chunk-size 128M \
144142
--buffer-size 4M \
145-
--use-mmap \
146143
--cache-dir "$CACHE" \
147144
--no-modtime \
148145
--umask 022 \

0 commit comments

Comments
 (0)