Context
In sidecar mode, the FUSE daemon runs unprivileged (user 65534) without access to /dev/fuse. Currently clone_fd is disabled because FUSE_DEV_IOC_CLONE requires opening /dev/fuse fresh, which the sidecar cannot do.
This means sidecar mode is limited to a single FUSE reader thread, while normal mode uses N reader threads via cloned fds.
Proposal
The CSI driver (privileged) could pre-create N cloned fds and send them all via SCM_RIGHTS. The sidecar would then assign one fd per reader thread.
Blockers
-
FUSE_INIT ordering: the clone ioctl requires the fd to have completed the FUSE_INIT handshake first. Currently fuser does the handshake in Session::from_fd. The CSI driver (Go) would need to do the handshake itself, then clone, then send.
-
fuser API: fuser has no Session::from_fd_skip_init() or Session::from_cloned_fds() API. Would need an upstream contribution or a fork.
-
SCM_RIGHTS multi-fd: the Go side (SendMsg) and Rust side (connect_and_receive_fd) would need to handle multiple fds in a single message.
Alternative
Implement the FUSE_INIT handshake in Go (read FUSE_INIT request, write FUSE_INIT_OUT response), then clone N fds and send them. The sidecar would use a custom fuser session that skips the handshake.
Priority
Low. Single-threaded FUSE is sufficient for most workloads where the bottleneck is network (HF Hub downloads), not FUSE dispatch. Worth revisiting if profiling shows FUSE serialization as a bottleneck on high-IOPS workloads.
Related: huggingface/hf-csi-driver#19
Context
In sidecar mode, the FUSE daemon runs unprivileged (user 65534) without access to
/dev/fuse. Currentlyclone_fdis disabled becauseFUSE_DEV_IOC_CLONErequires opening/dev/fusefresh, which the sidecar cannot do.This means sidecar mode is limited to a single FUSE reader thread, while normal mode uses N reader threads via cloned fds.
Proposal
The CSI driver (privileged) could pre-create N cloned fds and send them all via SCM_RIGHTS. The sidecar would then assign one fd per reader thread.
Blockers
FUSE_INIT ordering: the clone ioctl requires the fd to have completed the FUSE_INIT handshake first. Currently fuser does the handshake in
Session::from_fd. The CSI driver (Go) would need to do the handshake itself, then clone, then send.fuser API: fuser has no
Session::from_fd_skip_init()orSession::from_cloned_fds()API. Would need an upstream contribution or a fork.SCM_RIGHTS multi-fd: the Go side (
SendMsg) and Rust side (connect_and_receive_fd) would need to handle multiple fds in a single message.Alternative
Implement the FUSE_INIT handshake in Go (read
FUSE_INITrequest, writeFUSE_INIT_OUTresponse), then clone N fds and send them. The sidecar would use a custom fuser session that skips the handshake.Priority
Low. Single-threaded FUSE is sufficient for most workloads where the bottleneck is network (HF Hub downloads), not FUSE dispatch. Worth revisiting if profiling shows FUSE serialization as a bottleneck on high-IOPS workloads.
Related: huggingface/hf-csi-driver#19