Move from Go‑FUSE to a full libfuse3 implementation for POSIX‑compliant workloads

[hazhhajian]

Hi everyone,

I’d like to open a discussion around the FUSE layer used by SeaweedFS CSI and the seaweedfs‑mount image.

We’re currently running:

chrislusf/seaweedfs-enterprise:4.13
chrislusf/seaweedfs-mount:1.4.4

From the source structure in weed/mount , it looks like the mount component still relies on Go‑FUSE, not the kernel‑level libfuse3 library.

That choice makes the mount lightweight, but it also limits proper handling of several POSIX functions getcwd(), fcntl(), ftruncate(), and atomic rename() which some developer tools depend on.

We’ve observed that when volumes are mounted via the CSI driver:

git clone, git commit, and git push fail with “origin does not appear to be a git repository” due to non-atomic renames or file-locking issues.

As a workaround, we deploy a DaemonSet using the seaweedfs-enterprise/seaweedfs image to mount the SeaweedFS global namespace on each node’s local filesystem. This exposes the distributed filesystem at /mnt/seaweedfs, which application pods consume via a hostPath volume. Once mounted this way, Git and other file operations run normally against the shared SeaweedFS volume.

Best,
Hazhir

[chrislusf]

git clone, git commit, and git push fail with “origin does not appear to be a git repository” due to non-atomic renames or file-locking issues.

could you show exact steps how to reproduce this?

[hazhhajian]

Here is my setup. I have mounted the SeaweedFS in two ways. One using CSI driver and the other one by mounting SeaweedFS via DaemonSet on each node using seaweedfs-enterprise image.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-git-test-csi
  labels:
    lifecycle: persistent
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: seaweedfs-csi-driver
    volumeHandle: dfs-git-test
    volumeAttributes:
      collection: git-test
      replication: "001"
      path: /dev/git-test
    readOnly: false
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-seaweedfs-git-test-csi
  namespace: tmp
  labels:
    lifecycle: persistent
spec:
  storageClassName: ""
  volumeName: pv-git-test-csi
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: seaweedfs-git-test-ds
  namespace: seaweedfs
spec:
  selector:
    matchLabels:
      app: seaweedfs-git-test-ds
  template:
    metadata:
      labels:
        app: seaweedfs-git-test-ds
    spec:
      # FUSE requires privileged mode
      securityContext:
        runAsUser: 0
      containers:
        - name: git-test-ds
          image: chrislusf/seaweedfs-enterprise:4.13
          securityContext:
            privileged: true
          command: ["weed"]
          args:
            [
              "mount",
              "-filer=seaweedfs-filer.seaweedfs.svc.cluster.local:8888",
              "-dir=/mnt/seaweedfs-git-test-ds",
              "-filer.path=/git-test",
              "-allowOthers",
              "--volumeServerAccess=direct",
              "-dirAutoCreate",
            ]
          volumeMounts:
            - name: seaweedfs-git-test-ds
              mountPath: /mnt/seaweedfs-git-test-ds
      volumes:
        - name: seaweedfs-git-test-ds
          hostPath:
            path: /mnt/seaweedfs-git-test-ds
            type: DirectoryOrCreate
      tolerations:
        - operator: Exists
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: git-test
  namespace: tmp
  labels:
    app: git-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: git-test
  template:
    metadata:
      labels:
        app: git-test
    spec:          
      containers:
        - name: git-test-pod
          image: alpine/git:latest
          command: ['tail', '-f', '/dev/null']
          volumeMounts:
            - name: pvc-seaweedfs-git-test-csi
              mountPath: /mnt/sw-csi
            - name: seaweedfs-git-test-ds
              mountPath: /mnt/sw-seaweedfs
        
      volumes:
        - name: pvc-seaweedfs-git-test-csi
          persistentVolumeClaim:
            claimName: pvc-seaweedfs-git-test-csi
        - name: seaweedfs-git-test-ds
          hostPath:
            path: /mnt/seaweedfs-git-test-ds
            type: Directory

And here is the rest of the test from within the Git pod:

➜  tmp git:(master) ✗ k get pod
NAME                              READY   STATUS        RESTARTS   AGE
git-test-6746fb568-qcdt7          1/1     Running       0          33s

➜  tmp git:(master) ✗ k exec -it git-test-6746fb568-qcdt7 -- sh
/git # ls -l /mnt/
total 4
drwxrwxrwx    1 root     root             0 Feb 28 10:41 sw-csi
drwxr-xr-x    2 root     root          4096 Feb 28 10:31 sw-seaweedfs

/git # cd /mnt/sw-csi/
/mnt/sw-csi # git clone https://github.com/seaweedfs/seaweedfs.git
Cloning into 'seaweedfs'...
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

/mnt/sw-csi # cd /mnt/sw-seaweedfs/
/mnt/sw-seaweedfs # git clone https://github.com/seaweedfs/seaweedfs.git
Cloning into 'seaweedfs'...
remote: Enumerating objects: 105751, done.
remote: Counting objects: 100% (1048/1048), done.
remote: Compressing objects: 100% (497/497), done.
remote: Total 105751 (delta 770), reused 553 (delta 551), pack-reused 104703 (from 3)
Receiving objects: 100% (105751/105751), 161.45 MiB | 14.71 MiB/s, done.
Resolving deltas: 100% (75942/75942), done.
Updating files: 100% (2665/2665), done.
/mnt/sw-seaweedfs #