What is the maximum read/write bandwidth performance currently achievable with go-fuse?

[Hpensor]

What are the theoretical maximum limits for single-threaded read, single-threaded write, and 50/50 mixed read-write workloads in terms of bandwidth and IOPS when using go-fuse's rawFileSystem with kernel FUSE? @hanwen @jmhodges @bradfitz

In our practical tests, we stripped away all backend processing logic and simply returned success immediately within the API handlers (null operation). Under this configuration, the maximum write bandwidth we observed was approximately 5 GB/s.

test case:
vdbench 2000 * 100MB file /120 thread / 128KB ioSize /

[hanwen]

If you use pass-through, you can read/write at the same speed as a normal file on (say) ext4, as that is what passthrough does.

If you don't, I don't have a good answer. I never have had the need for ultra-high-performance scenarios, so I never tested beyond my 4-core laptop (2019 vintage).

I expect that you get the best performance if you make the server run on a single CPU: the read loop (fuse package) uses a normal Go locks to synchronize access to outstanding reads/writes, and I expect scaling the server dispatch beyond one CPU will cause contention. The fs package has distributed locks for operations on inodes, but still a single lock in rawBridge (this can probably be optimized with sync.Map, though.)

REcent kernel work on FUSE performance centers around tricks like io_uring, which doesn't play well with the Go runtime.

Oh, you ask about single threaded performance? If your file system doesn't do anything, most of the overhead is in the kernel. Reading requests from the kernel creates minimal garbage, and if you write all zeroes, neither does writing to the kernel. So it's basically 2 syscalls, a couple of map lookups and a couple of uncontested mutex locks. If you measure it, you're just measuring how quickly the kernel can do context switches and user <-> kernel data copies.

hanwen@fedora:~/vc/hanwen/go-fuse$ go test ./benchmark -test.bench '.*' -test.cpu 1
read: (g=0): rw=read, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=psync, iodepth=1
fio-3.40
Starting 1 process
read: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1)
read: (groupid=0, jobs=1): err= 0: pid=51223: Fri Jan 30 09:16:05 2026
  read: IOPS=7794, BW=974MiB/s (1022MB/s)(1024MiB/1051msec)
    clat (usec): min=31, max=458, avg=126.02, stdev=96.27
     lat (usec): min=31, max=458, avg=126.28, stdev=96.48
    clat percentiles (usec):
     |  1.00th=[   33],  5.00th=[   34], 10.00th=[   34], 20.00th=[   36],
     | 30.00th=[   39], 40.00th=[   40], 50.00th=[   95], 60.00th=[  157],
     | 70.00th=[  192], 80.00th=[  225], 90.00th=[  269], 95.00th=[  293],
     | 99.00th=[  326], 99.50th=[  338], 99.90th=[  359], 99.95th=[  375],
     | 99.99th=[  457]
   bw (  KiB/s): min=584448, max=1173504, per=88.10%, avg=878976.00, stdev=416525.49, samples=2
   iops        : min= 4566, max= 9168, avg=6867.00, stdev=3254.11, samples=2
  lat (usec)   : 50=46.55%, 100=3.78%, 250=35.74%, 500=13.93%
  cpu          : usr=5.05%, sys=18.00%, ctx=8193, majf=0, minf=42
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=8192,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=974MiB/s (1022MB/s), 974MiB/s-974MiB/s (1022MB/s-1022MB/s), io=1024MiB (1074MB), run=1051-1051msec
goos: linux
goarch: amd64
pkg: github.com/hanwen/go-fuse/v2/benchmark
cpu: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
BenchmarkGoFuseMemoryRead             	   42352	     26747 ns/op	4900.41 MB/s	      75 B/op	       3 allocs/op
BenchmarkGoFuseFDRead                 	   23817	     51099 ns/op	2565.07 MB/s	     155 B/op	       3 allocs/op
BenchmarkGoFSStat                     	  530234	      2232 ns/op
BenchmarkGoFSReaddir                  	    6304	    464635 ns/op
BenchmarkLibfuseHighlevelThreadedStat 	   18655	     58435 ns/op
PASS
ok  	github.com/hanwen/go-fuse/v2/benchmark	18.311s

[rfjakob]

I/O request size could be upped to 2 MiB (I think?)