Skip to content

Stop chowning all of ${RUNNER_TEMP} in linux_job/linux_job_v2 - #8598

Merged
huydhn merged 1 commit into
mainfrom
fix-chown-runner-temp
Aug 20, 2026
Merged

Stop chowning all of ${RUNNER_TEMP} in linux_job/linux_job_v2#8598
huydhn merged 1 commit into
mainfrom
fix-chown-runner-temp

Conversation

@huydhn

@huydhn huydhn commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

On the gfx1100 ROCm runners docker is rootless, so a chown -R $(id -u) issued from inside a container lands on a subordinate UID. Chowning all of ${RUNNER_TEMP} therefore takes _github_workflow/event.json away from the runner, and every step after it fails with Access to the path ... is denied — see executorch#21977, where the test passed and 6 post-steps failed.

Chown only the three dirs we mount into the container instead.

@vercel

vercel Bot commented Aug 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
torchci Ignored Ignored Aug 20, 2026 9:00pm

Request Review

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 20, 2026
Comment thread .github/workflows/linux_job.yml Dismissed
Comment thread .github/workflows/linux_job.yml Dismissed
@huydhn
huydhn force-pushed the fix-chown-runner-temp branch from 16ea3a0 to 163d976 Compare August 20, 2026 20:11
${RUNNER_TEMP} also holds the runner's own control files --
_github_workflow/event.json, _runner_file_commands/ -- and the recursive
chown we issue from inside an alpine container is not guaranteed to land
on the runner's UID.

On a runner whose docker daemon remaps container UIDs it does not. The
gfx1100 ROCm runners (ctr-navi3x-b13-ws*) run rootless docker (`docker
info` reports DockerRootDir=/home/pytorchci/.local/share/docker), where
container UID 0 maps to the host runner user and container UID N>0 maps
into that user's subuid range. `chown -R $(id -u)` -- the runner's *host*
UID, expanded outside the container -- therefore lands on a subordinate
UID that nothing on the host owns. The runner then cannot read
event.json, and every remaining step of the job fails with

    ##[error]Access to the path
    '/home/pytorchci/actions-runner/_work/_temp/_github_workflow/event.json'
    is denied.

Chown only the three directories we actually hand to the container:
$RUNNER_ARTIFACT_DIR, $RUNNER_DOCS_DIR and $RUNNER_TEST_RESULTS_DIR.
Those are the bind mounts the exec container writes to as root, so they
are the only ones that need restoring, and none of them belong to the
runner. setup-linux, setup-rocm and setup-xpu all set the three env vars
to ${RUNNER_TEMP}/{artifacts,docs,test-results}.

Anything a script writes directly to ${RUNNER_TEMP} as root is no longer
chowned back. linux_job.yml has the same step but is legacy, so it is
left alone.

Hoist the repeated ALPINE_IMAGE expression to a job-level env var rather
than repeat it three times.
@huydhn
huydhn force-pushed the fix-chown-runner-temp branch from 163d976 to bd3f737 Compare August 20, 2026 21:00
@huydhn
huydhn requested a review from a team August 20, 2026 21:05
@huydhn
huydhn marked this pull request as ready for review August 20, 2026 21:07
@digantdesai

Copy link
Copy Markdown

Thanks @huydhn, let me know once this lands, I can rerun or land gfx1100 CI.

@huydhn

huydhn commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

@digantdesai Appreciate if you could try to test this out on your PR. Two small tweaks are needed for that:

  • uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@fix-chown-runner-temp
  • Add the parameter test-infra-ref: fix-chown-runner-temp to the job

@huydhn
huydhn merged commit cb4b1c6 into main Aug 20, 2026
18 checks passed
@huydhn
huydhn deleted the fix-chown-runner-temp branch August 20, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants