Skip to content

SCHED-411: use native pyxis#2208

Merged
itechdima merged 2 commits intomainfrom
SCHED-411/use-native-pyxis
Feb 26, 2026
Merged

SCHED-411: use native pyxis#2208
itechdima merged 2 commits intomainfrom
SCHED-411/use-native-pyxis

Conversation

@itechdima
Copy link
Collaborator

@itechdima itechdima commented Feb 24, 2026

Problem

Old pyxis fork instead of upstream version due to unavailability to cache squashfs files.

Solution

Update pyxis to 0.23.0 and use its logic for enroot importer, so we change our own implementation to native.

Testing

Initial load

root@login-0:~# time srun -w worker-1 --container-image=nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04 hostname
cpu-bind=MASK - worker-1, task  0  0 [2154]: mask 0x3 set
pyxis: importing docker image: nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04
pyxis: imported docker image: nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04
worker-1

real	1m57.889s
user	0m0.000s
sys	0m0.018s

Different worker uses cache (with different user):

nebius@login-0:~$ time srun -w worker-0 --container-image=nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04 hostname
cpu-bind=MASK - worker-0, task  0  0 [2434]: mask 0x3 set
pyxis: importing docker image: nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04
pyxis: imported docker image: nvidia/cuda:13.1.1-cudnn-devel-ubuntu22.04
worker-0

real	0m15.814s
user	0m0.004s
sys	0m0.015s

Release Notes

Feature: udpate pyxis to 0.23.0, enroot to 4.1.1. Use enroot importer extension in pyxis to cache squashfs files.

@itechdima itechdima force-pushed the SCHED-411/use-native-pyxis branch 3 times, most recently from 9dfd726 to 111ecff Compare February 25, 2026 14:40
@itechdima itechdima force-pushed the SCHED-411/use-native-pyxis branch 4 times, most recently from d64b082 to d4bf9ff Compare February 26, 2026 10:43
@itechdima itechdima force-pushed the SCHED-411/use-native-pyxis branch from d4bf9ff to 4b0f5bb Compare February 26, 2026 11:39
@itechdima itechdima marked this pull request as ready for review February 26, 2026 11:39
asteny
asteny previously approved these changes Feb 26, 2026
@itechdima itechdima merged commit 0f88fa8 into main Feb 26, 2026
20 checks passed
@itechdima itechdima deleted the SCHED-411/use-native-pyxis branch February 26, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants