fix(fuse): switch to hanwen/go-fuse (#11272)

* test(fuse): consolidate FUSE tests into test/cli/fuse

Move FUSE integration tests from sharness shell scripts (t0030, t0031,
t0032) and test/cli/fuse_test.go into a dedicated test/cli/fuse/ Go
sub-package, ensuring all FUSE test cases run in CI.

- git mv test/cli/fuse_test.go to test/cli/fuse/ (package fuse)
- convert all sharness FUSE tests to Go subtests under TestFUSE:
  mount failure, IPNS symlink, IPNS NS map resolution, MFS file/dir
  creation, xattr (Linux), files write, add --to-files, file removal,
  nested dirs, publish-while-mounted block, sharded directory reads
- add xattr helpers with build tags (linux/other) using unix.Getxattr
- split make test_fuse into test_fuse_unit (./fuse/...) and
  test_fuse_cli (./test/cli/fuse/...) sub-targets
- set TEST_FUSE=0 in test_cli so FUSE tests skip in cli-tests CI job
- increase fuse-tests CI timeout from 5m to 10m for CLI tests
- delete sharness t0030, t0031, t0032 (were always skipped in CI)

* docs: document FUSE test split between unit and e2e

Add cross-reference comments between the unit tests in fuse/readonly/,
fuse/ipns/, fuse/mfs/ and the end-to-end CLI tests in test/cli/fuse/.
Also fix AGENTS.md to use a temp dir for fusermount symlink instead of
sudo.

* ci: prevent stale FUSE mounts from failing fuse-tests

On shared self-hosted runners, leftover mount points from previous
runs can exhaust the kernel FUSE mount limit.

- add job-level concurrency group so only one fuse-tests runs at a time
- lazy-unmount stale /tmp/fusetest* mounts before running tests

* ci: only symlink fusermount3 when fusermount is missing

* fix(fuse): remove goroutine leak in IPNS Flush handler

The Flush handler wrapped fi.fi.Flush() in a goroutine so it could
return early when the FUSE context was canceled. But the goroutine
kept running in the background, and when Release arrived it called
Close on the same file descriptor concurrently. The two paths both
entered DagModifier.Sync, racing on its internal write buffer and
causing a nil pointer panic.

The fix is to call Flush directly without a goroutine. The MFS flush
cannot be safely canceled mid-operation anyway, so the goroutine
only added the illusion of cancellation while leaking work and
masking the real error.

Also bumps boxo to pick up the matching defense-in-depth fix that
serializes FileDescriptor.Flush and Close with a mutex.

* fix(fuse): add mutex to IPNS file handle operations

bazil/fuse dispatches each FUSE request in its own goroutine.
The IPNS File handle had no synchronization, so concurrent
Read/Write/Flush/Release calls could overlap on the underlying
DagModifier which is not safe for concurrent use.

Add sync.Mutex to File, matching the pattern already used by the
MFS FileHandler.

* refactor(fuse): remove dead File.Forget method

bazil/fuse only dispatches Forget to nodes via the NodeForgetter
interface. File is a handle, not a node, so this method was never
called. The /mfs mount has no equivalent.

* fix(fuse): flush IPNS directory after Remove and Rename

The /mfs mount flushes the directory after Unlink and Rename so
changes propagate to the MFS root immediately. The /ipns mount
did not, leaving mutations pending until an unrelated flush.

Also add an empty-directory check before removing directories,
matching the /mfs mount's safety check.

* fix(fuse): inherit CID builder and flush on IPNS Create

New files created via the /ipns FUSE mount now inherit the CID
builder from their parent directory, preventing CIDv0 nodes from
appearing inside a CIDv1 tree.

The directory is also flushed after AddChild so the new entry
propagates to the MFS root immediately, matching the /mfs mount.

* test(fuse): add IPNS Remove and non-empty rmdir tests

Cover the file removal path and the empty-directory safety check
added in the previous commit. TestRemoveFile verifies a created
file can be removed and is gone afterwards. TestRemoveNonEmptyDirectory
verifies that rmdir on a directory with children fails, and succeeds
once the children are removed first.

* feat(fuse): read UnixFS mode/mtime, add StoreMtime/StoreMode config

All three FUSE mounts now read mode and mtime from UnixFS metadata
when present, falling back to POSIX defaults when absent. Most IPFS
data does not include this optional metadata.

Writing mode and mtime is opt-in via two new config flags:
- Mounts.StoreMtime: persist mtime on file create and open-for-write
- Mounts.StoreMode: persist mode on chmod

Other changes in this commit:
- align default file/dir modes across /ipns and /mfs to 0644/0755
- share mode constants via fuse/mount/mode.go
- convert Mounts.FuseAllowOther from bool to Flag for consistency
- add Setattr to /ipns FileNode and /mfs File for chmod and touch
- move dead File.Setattr from IPNS handle to FileNode (node)
- bump boxo for Directory.Mode() and Directory.ModTime() getters

* feat(fuse): add ipfs.cid xattr to all mounts

All three FUSE mounts now expose the node's CID via the ipfs.cid
extended attribute on both files and directories.

The /mfs mount also accepts the old ipfs_cid name for backward
compatibility. The /ipfs mount previously had a stub that returned
nil for all xattrs; it now returns the correct CID.

The xattr name follows the convention used by CephFS (ceph.*),
Btrfs (btrfs.*), and GlusterFS (glusterfs.*).

* feat(fuse): switch from bazil.org/fuse to hanwen/go-fuse v2

Replace the unmaintained bazil.org/fuse (last commit 2020) with
hanwen/go-fuse v2.9.0, fixing two architectural issues that could
not be solved with the old library.

ftruncate now works: hanwen/go-fuse passes the open file handle to
NodeSetattrer, so Setattr can truncate through the existing write
descriptor instead of trying to open a second one (which deadlocks
on MFS's single-writer lock).

fsync now works: FileFsyncer runs on the handle directly, flushing
the write buffer through the open descriptor. Previously a no-op
because bazil dispatched Fsync to the inode only.

mount package:
- NewMount takes (InodeEmbedder, mountpoint, *fs.Options) instead
  of (fs.FS, mountpoint, allowOther)
- mount/unmount collapses to a single fs.Mount call
- fusermount3 tried before fusermount in ForceUnmount

all three mounts:
- structs embed fs.Inode (hanwen's InodeEmbedder pattern)
- Remove split into Unlink + Rmdir (separate FUSE interfaces)
- ReadDirAll replaced with Readdir returning DirStream
- fillAttr helper shared between Getattr and Lookup responses
- kernel cache invalidation via NotifyContent after Flush
- 1s entry/attr timeout for writable mounts (matches go-fuse
  default, gocryptfs, rclone)
- O_APPEND tracked on file handle, writes seek to end
- build tags standardized to (linux || darwin || freebsd) && !nofuse

tests:
- replaced bazil fstestutil.MountedT with shared fusetest.TestMount
- fixed TestConcurrentRW: channel drain mismatch and missing sync
  between write Close and read start
- added TestFsync, TestFtruncate, TestReadlink, TestSeekRead,
  TestLargeFile, TestRmdir, TestCrossDirRename, TestUnknownXattr
- added StoreMtime disabled/enabled subtests

* fix(fuse): close fd on error in Open to prevent leak

MFS enforces a single-writer lock, so a leaked write descriptor
blocks all subsequent opens of that file until GC.

* fix(fuse): detect external unmount via server.Wait

Without this, IsActive stays true after `fusermount -u` and
Unmount returns nil instead of ErrNotMounted.

* fix(fuse): return actual error from Unlink/Rmdir, not ENOENT

After confirming the child exists, an Unlink failure could be an
IO error. Returning ENOENT would hide the real cause.

* fix(fuse): reuse DagReader per open, pass ctx to all reads

Readonly Open now returns a file handle holding a DagReader instead
of recreating one per Read call. Sequential reads no longer
re-traverse the DAG from the root on each kernel request.

All three mounts now use CtxReadFull with the kernel's per-request
context so killing a process mid-read cancels in-flight block
fetches instead of letting them complete uselessly.

* chore(fuse): cleanup dead code, add var comments

- remove dead `_ = mntDir` in TestXattrCID
- comment why immutableAttrCacheTime and mutableCacheTime are var
- add TODO for using IPNS record TTL as cache timeout

* chore(fuse): replace OSXFUSE 2.x check with macFUSE detection

The old check tried to verify OSXFUSE >= 2.7.2 to avoid a kernel
panic from 2015. It used sysctl, tried to `go install` a third-party
tool at runtime, and referenced paths that no longer exist.

Replace with a simple check for the macFUSE mount helper, matching
the same paths go-fuse looks for. If neither macFUSE nor OSXFUSE is
found, point the user to the install page.

Also standardize build tags to (linux || darwin || freebsd) && !nofuse
and use strings.ReplaceAll.

* fix(fuse): include mountpoint path in mount errors

go-fuse's fusermount errors don't include the path, so tools that
check error messages for the mountpoint name couldn't tell which
mount failed.

* chore(ci): remove bazil fusermount workaround

go-fuse finds fusermount3 natively, no symlink needed. The stale
mount cleanup was for bazil's fstestutil which we no longer use.

* docs: update v0.41 changelog for FUSE rewrite

* chore(deps): bump boxo for full FileDescriptor serialization

boxo@64be0815 extends the mutex from Flush/Close to all
FileDescriptor operations (Read, Write, Seek, Truncate, Size),
preventing data races on the underlying DagModifier.

* chore(deps): bump boxo to merged ipfs/boxo#1133

Picks up full FileDescriptor serialization: the mutex now covers
all operations (Read, Write, Seek, Truncate, Size), not just
Flush and Close.

* feat(fuse): CAP_ATOMIC_O_TRUNC, new integration tests

Advertise CAP_ATOMIC_O_TRUNC so the kernel sends O_TRUNC inside
Open instead of doing a separate SETATTR(size=0) first. Without
this, the kernel's SETATTR needs to open a write descriptor inside
Setattr, which deadlocks on MFS's single-writer lock.

Move kernel cache invalidation from Flush to Release because
mfsFD.Close (in Release) is where the final DAG node is committed.

Upgrade go-fuse to latest for ExtraCapabilities support.

New tests for both MFS and IPNS:
- TestOpenTrunc, TestSeekAndWrite, TestOverwriteExisting
- TestTempFileRename, TestVimSavePattern, TestRsyncPattern
  (skipped pending rename-over-existing and cache fixes)

* fix(fuse): rename-over-existing, bump boxo for flushUp race fix

IPNS Rename now unlinks the target before AddChild, matching MFS.
Without this, renaming onto an existing name returned "directory
already has entry".

Bump boxo to pick up the flushUp unlinked-entry fix (ipfs/boxo@8ae46d5):
when a file descriptor outlives its directory entry (FUSE RELEASE
racing with RENAME), flushUp no longer re-adds the stale name.

Unskip TestTempFileRename and TestRsyncPattern on both mounts.

* fix(fuse): unskip VimSavePattern, bump boxo for setNodeData fix

boxo@552d8e7 fixes File.setNodeData dropping content links when
updating metadata (mode, mtime). chmod or touch after write no
longer makes the file appear empty.

Unskip TestVimSavePattern on both mounts. Remove debug logging
and temporary test functions added during investigation.

* fix(fuse): build tags for cross-compilation

go-fuse does not compile on windows/openbsd/netbsd/plan9. Move
WritableMountCapabilities (which imports go-fuse) from mode.go
(no build tag) to caps.go (platform-gated). Align build tags on
fusetest and core/commands/mount stubs so unsupported platforms
don't pull in go-fuse transitively.

* fix(test): use fusermount3 in CLI FUSE tests

The doUnmount helper hardcoded fusermount, but systems with only
fuse3 installed have fusermount3. Try fusermount3 first, matching
what go-fuse and our ForceUnmount already do.

* feat(fuse): symlink support on writable mounts

Add NodeSymlinker to MFS and IPNS directories. Symlinks are stored
as UnixFS TSymlink nodes in the DAG, the same format used by
`ipfs add` for directories containing symlinks. The readonly /ipfs
mount already rendered existing symlinks; now /mfs and /ipns can
create them too.

The target string is cached at Lookup time to avoid re-parsing the
DAG node on every Readlink call. Symlink permissions are always
0777 per POSIX convention (access control uses the target's mode).

* fix(fuse): checked type assertion in MFS Rename

The direct type assertion on newParent could panic if the kernel
passed a non-directory inode. Use a checked assertion with EINVAL
fallback, matching the type-switch pattern in the IPNS mount.

* fix(test): add missing continue in stress test

Missing continue after error sends let execution fall through to
nil type assertions (read.(files.File)) that would panic on error.
Also cancel the context before continuing to avoid leaking it.

* fix(fuse): return error from Readdir when DAG.Get fails

Abort the directory listing instead of silently omitting the
unretrievable entry. Callers get EIO, which is more honest than
a partial listing that hides missing blocks.

* docs: remove duplicate fsync bullet in changelog

* ci: clean up stale FUSE mounts in fuse-tests job

On shared self-hosted runners, leftover mounts from crashed runs
can exhaust the kernel mount_max limit. Lazy-unmount kubo-test
and harness temp mounts before and after tests.

* chore(deps): bump boxo to merged ipfs/boxo#1134

Picks up flushUp unlinked-entry guard and setNodeData content link
preservation.

* docs: add build tag comments, normalize tag style

Add a one-line comment above every //go:build directive explaining
why the constraint exists.

Normalize tag style: positive platform constraints first, then
feature flags/negations. Simplify redundant expressions.

* fix(fuse): add Setattr to directories for chmod and mtime

Tools like tar and rsync call utimensat on directories after
extraction. Without Setattr on Dir, this returned ENOTSUP.

Add Setattr to Dir (MFS) and Directory (IPNS) that handles mode
and mtime the same way as the file-level Setattr. When StoreMtime
or StoreMode is disabled the call succeeds silently, matching the
file-level behavior.

* docs: clarify directory support and spec link for StoreMtime/StoreMode

- mention that touch and chmod work on both files and directories
- note tar and rsync as practical use cases
- link to UnixFS spec for optional metadata storage

* fix(fuse): use proper mode conversion, document 9-bit limit

Use files.UnixPermsToModePerms and files.ModePermsToUnixPerms for
converting between FUSE kernel mode (unix 12-bit layout) and Go's
os.FileMode (different bit positions for setuid/setgid/sticky).

The UnixFS spec supports all 12 permission bits, but boxo's MFS
layer (File.Mode, Directory.Mode) exposes only the lower 9. FUSE
mounts are always nosuid so the upper 3 bits would have no effect.

Add TestSetuidBitsStripped to both mounts confirming the behavior.

* feat(fuse): symlink Setattr with mtime persistence

Wire the backing mfs.File into the FUSE Symlink struct so Setattr
can call SetModTime when StoreMtime is enabled. boxo's File methods
(SetModTime, ModTime) already work on TSymlink nodes since they
operate on the FSNode protobuf without checking the type.

Without Setattr, rsync -a fails with "failed to set times" on
symlinks. Every major FUSE filesystem (gocryptfs, rclone, sshfs,
s3fs) implements Setattr on symlinks for this reason.

Mode is always 0777 per POSIX convention, so chmod requests are
silently accepted but not stored.

* fix(fuse): return EIO instead of panicking on unknown node type

Replace panic with log.Errorf + syscall.EIO in IPNS Directory.Lookup
for unexpected MFS node types. Also remove duplicate comment block
on File.Flush.

* docs: update FUSE docs for go-fuse migration

- fuse.md: replace stale OSXFUSE section with macFUSE, remove
  obsolete go-fuse-version tool, fix broken FreeBSD sudo echo,
  update xattr example to ipfs.cid with CIDv1, add mode/mtime
  section, add unixfs-v1-2025 tip, add debug logging section,
  add TOC, link to hanwen/go-fuse
- changelog: refine bullet wording, link to fuse.md
- config.md: fix double space, update fuse.md link text
- experimental-features.md: fix double space, soften wording
- README.md: add FUSE to features list and docs table

* refactor(fuse): extract shared writable types and test suite

Extract duplicated code from fuse/mfs and fuse/ipns into a shared
fuse/writable package, and consolidate duplicated tests into a
reusable suite in fuse/fusetest.

- fuse/writable: Dir, FileInode, FileHandle, Symlink types with all
  FUSE interface methods, shared by both mounts
- fuse/fusetest: RunWritableSuite with helpers, exercised by both
  mfs and ipns via mount-specific factories
- fix cache invalidation race: NotifyContent in Flush (synchronous)
  in addition to Release (async), so stat after close sees new size
- drop deprecated ipfs_cid xattr, log error guiding users to ipfs.cid
- mfs_unix.go: 632 -> 19 lines (thin wrapper over writable.Dir)
- ipns_unix.go: 795 -> 170 lines (Root + key resolution only)
- mfs_test.go: 1183 -> 95 lines (factory + persistence test)
- ipns_test.go: 1309 -> 162 lines (factory + IPNS-specific tests)
- tests that were only in one mount now run on both

* feat(fuse): add macOS-specific mount options

Set volname, noapplexattr, and noappledouble on macOS via
PlatformMountOpts, applied in NewMount so all three mounts
benefit automatically.

- volname: shows mount name in Finder instead of "macfuse Volume 0"
- noapplexattr: suppresses Finder's com.apple.* xattr probes
- noappledouble: prevents ._ resource fork sidecar files

* fix(fuse): detect symlinks in readdir, fix stale refs

Readdir on writable mounts now checks the underlying DAG node type
for TFile entries, reporting S_IFLNK for symlinks instead of regular
file. This makes ls -l and find -type l work correctly.

- writable: Readdir checks SymlinkTarget for TFile entries
- writablesuite: add SymlinkReaddir regression test
- readonly: add TestReaddirSymlink regression test
- test/cli/fuse: fix stale bazil.org/fuse reference in doc comment

* fix(fuse): normalize deprecated ipfs_cid xattr to ipfs.cid

Getxattr for the old "ipfs_cid" name now returns the CID instead of
ENOATTR, keeping existing tooling working during the deprecation
period. A log error is emitted on each access to nudge migration.

* fix(fuse): serialize concurrent reads on readonly file handles

The go-fuse server dispatches each FUSE request in its own goroutine.
On files larger than 128 KB the kernel issues concurrent readahead
Read requests on the same file handle, racing on the shared DagReader's
Seek+CtxReadFull sequence and corrupting its internal state.

Add sync.Mutex to roFileHandle (matching the existing pattern in
writable.FileHandle) and lock in Read and Release.

- fuse/readonly/readonly_unix.go: add mu sync.Mutex to roFileHandle
- fuse/readonly/ipfs_test.go: add TestConcurrentLargeFileRead
- fuse/fusetest/writablesuite.go: add LargeFileConcurrentRead to
  shared writable suite (exercised by both /mfs and /ipns tests)

* fix(fuse): bypass MFS locking for read-only opens

MFS uses an RWMutex (desclock) that holds RLock for the lifetime of a
read descriptor and requires exclusive Lock for writes. Tools like
rsync --inplace open the same file for reading and writing from
separate processes, deadlocking on this mutex.

For O_RDONLY opens, create a DagReader directly from the current DAG
node instead of going through MFS. The reader gets a point-in-time
snapshot and never touches desclock, so writers proceed independently.

- fuse/writable/writable.go: add roFileHandle with DagReader for
  read-only opens, add DAG field to Config
- fuse/mfs/mfs_unix.go: pass ipfs.DAG to writable Config
- fuse/ipns/ipns_unix.go: pass ipfs.Dag() to writable Config
- fuse/fusetest/writablesuite.go: add ConcurrentReadWrite test
  exercising simultaneous read and write on the same file

* fix(fuse): support truncate(path, size) without open fd

Open a temporary write descriptor in Setattr when the kernel sends a
size change without a file handle (the truncate(2) syscall, as opposed
to ftruncate(fd) which passes the handle). Previously this returned
ENOTSUP.

- fuse/writable: open, truncate, flush, close in Setattr else branch
- fuse/fusetest: add TruncatePath to the shared writable suite
- test/cli/fuse: add end-to-end truncation test covering ftruncate(fd),
  syscall.Truncate(path), and open(O_TRUNC) through a real daemon

* ci(fuse): get stack traces on test hangs

The fuse-tests job was being silently cancelled by GitHub at 10min
because Go's per-test timeout (5m) was the same order as the job
timeout, and GOTRACEBACK=single hid the hung goroutines anyway.

- shrink TEST_FUSE_TIMEOUT to 4m so Go's panic fires first
- shrink job timeout-minutes to 6 (normal run is ~3min)
- set GOTRACEBACK=all so the panic dumps every goroutine, not just the timer

* fix(fuse): fill attrs in FileInode.Setattr response

Without this, the kernel could cache zero attrs after a chmod, touch, or
ftruncate until AttrTimeout (1s) expired. Dir.Setattr and Symlink.Setattr
already fill out.Attr; FileInode.Setattr now matches.

* docs(config): clarify Mounts.IPNS writability scope

Only directories backed by keys the node holds are writable. All other
names resolve via IPNS to read-only symlinks into the /ipfs mount.

* fuse: review cleanup for go-fuse migration

Final pass on #11272 addressing review feedback.

- writable: panic in NewDir if Config.DAG is nil. Both call sites
  already supply it, but a nil value silently fell back to the MFS
  path in FileInode.Open, re-introducing the rsync --inplace deadlock
  the read-only fast path was added to fix.
- writable: document Dir.Rename non-atomicity. Source unlink happens
  before destination add, so any failure between the two loses the
  source. An atomic fix requires changes in boxo/mfs.
- writable: add unit test locking in that Symlink.Setattr accepts a
  mode-only request without erroring and does not store the requested
  mode (POSIX symlinks have no meaningful permission bits).
- docs/config: correct StoreMode default modes; the previous text
  listed 0666 for files, which the code never uses.

* docs(config): list StoreMtime and StoreMode in Mounts TOC

* fix(fuse): fill EntryOut attrs in Dir.Create and Dir.Mkdir

Without this, fstat on the file handle returned by Create reports
mode 0 and size 0 for up to AttrTimeout (1s), because the kernel
caches the empty attrs from the Create response. Path-based stat
goes through Lookup which already fills attrs, so the bug only
shows up via fstat. Mirrors the same fix already applied to
FileInode.Setattr.

Dir.Mkdir gets the same fillAttr treatment for consistency, plus
a TODO noting that boxo's mfs.Directory.Mkdir accepts no mode arg
so the caller's mode is dropped on creation.

Adds CreateAttrsImmediate and MkdirAttrsImmediate to the shared
writable suite to guard both paths against future regressions.

* fix(fuse): map context cancellation to EINTR in read paths

When a userspace process is killed mid-read (Ctrl-C, SIGKILL on a
stuck cat) the kernel sends FUSE_INTERRUPT and go-fuse cancels the
per-request context. fs.ToErrno does not recognise context.Canceled
and falls through to "function not implemented", which the kernel
cannot act on. Map context.Canceled and DeadlineExceeded to EINTR
so the syscall is correctly aborted.

- mount/errno.go: new ReadErrno helper used by all context-aware
  read paths in both readonly and writable mounts
- readonly: applied to Node.Open, Node.Readdir, roFileHandle.Read
- writable: applied to FileInode.Open, FileHandle.Read, roFileHandle.Read
- readonly/ipfs_test.go: TestReadCancellationUnblocks guards the
  contract via a blocking DagReader fake; without ReadErrno the
  test reports "function not implemented" instead of EINTR

* test(fuse): add OExcl, DirRename, SparseWrite, FsyncCrossHandle

Coverage gaps in the shared writable suite:

- OExcl: lock files and atomic-create patterns rely on the second
  open with O_CREATE|O_EXCL failing with EEXIST
- DirRename: previously only file rename and cross-dir file rename
  were tested; this exercises Rename on a directory inode
- SparseWrite: WriteAt past the end of an empty file must report
  the correct size and return zeros for the gap
- FsyncCrossHandle: a reader on a fresh fd must see data flushed by
  fsync on the writer fd, not just after close

* test(fuse): cover external unmount on /ipns and /mfs

Previously TestExternalUnmount only exercised /ipfs, leaving the
goroutine that watches fuse.Server.Wait() untested for the other
two mounts. Refactor into a table-driven test that runs the same
fusermount/umount-then-IsActive flow against all three mounts.

Switch to coremock.NewMockNode so the node is online: doMount only
attaches the /ipns mount when node.IsOnline is true, and the table
needs all three populated.

* fix(commands): align 'ipfs mount' output columns

MountCmd's LongDescription has "MFS  mounted at:" with two spaces
so the column lines up with the 4-char "IPFS" and "IPNS" rows above,
but the runtime encoder and the daemon's startup print used a single
space and produced misaligned output.

Bring both runtime sites in line with the help text, and update the
two existing test fixtures (test/cli/fuse and the sharness test-lib
helper that t0040-add-and-cat.sh still uses) to expect the aligned
form.

* fix(fuse): invalidate kernel cache on Fsync

FileHandle.Fsync only flushed the MFS file descriptor and left the
kernel's cached attrs and content for the inode untouched. A fresh
reader on the same path then saw the size cached from the original
Create response (zero), reading zero bytes regardless of how much
the writer had synced.

Mirror the cache invalidation already done in Flush via
inode.NotifyContent(0, 0) so a writer that fsyncs while another
process opens the file (vim then a follow-up cat, IDE then a
language server) sees consistent state.

Sharpen the FsyncCrossHandle assertion to report the size delta on
failure; the bug surfaced as got=0/want=500 only after switching
from bytes.Equal to require.Equal.

* chore(gitignore): ignore test_fuse_unit and test_fuse_cli json output

The new test_fuse_unit and test_fuse_cli make targets emit
test/fuse/fuse-unit-tests.json and test/fuse/fuse-cli-tests.json
respectively, the same gotestsum --jsonfile pattern that test_unit
and test_cli already use. Add them to the same .gitignore section
so a local test run does not leave the working tree dirty.

* test(fuse): end-to-end coverage with real POSIX tools

Adds TestFUSERealWorld in test/cli/fuse/realworld_test.go: a single
shared-daemon test with 18 subtests that exercise the writable /mfs
mount through the actual binaries users invoke (sh, cat, seq, wc,
ls, stat, cp, mv, rm, ln, readlink, find, dd, sha256sum, tar,
rsync, vim). Each subtest verifies the result both via the FUSE
filesystem and via 'ipfs files read|stat|ls' so both views agree.

Synthetic payloads default to 1 MiB + 1 byte so multi-chunk
read/write paths are exercised, not just single-chunk fast paths.

External tools are required, not optional: a missing binary fails
the test loudly so a CI image change cannot silently turn the suite
green. The whole-suite TEST_FUSE gate is the only place a developer
is allowed to skip.

runCmd forces LC_ALL=C so locale-sensitive tool output (date
formats in 'ls -l', decimal separators in 'wc', localized error
messages, find/ls collation) is deterministic regardless of the
runner's locale settings.

One shared daemon across all 18 subtests keeps total runtime under
two seconds; isolation comes from per-subtest subdirectories under
the mount.

Author: lidel

.github/workflows/gotest.yml

@@ -150,12 +150,21 @@ jobs:
         if: failure() || success()
 
   # FUSE filesystem tests (require /dev/fuse and fusermount)
+  # Runs both FUSE unit tests (./fuse/...) and CLI integration tests (./test/cli/fuse/...)
   fuse-tests:
     if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'
     runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '["self-hosted", "linux", "x64", "2xlarge"]' || '"ubuntu-latest"') }}
-    timeout-minutes: 5
+    concurrency:
+      group: fuse-tests-${{ github.repository }}
+      cancel-in-progress: false
+    # A normal run takes ~3min. 6min gives roughly 2x and lets Go's 4min
+    # test timeout fire first (printing a stack trace) on a hang, instead
+    # of GitHub silently cancelling the job.
+    timeout-minutes: 6
     env:
-      GOTRACEBACK: single
+      # Dump all goroutines on a test panic, not just the panicking one,
+      # so we can see which test is actually hung.
+      GOTRACEBACK: all
       TEST_FUSE: 1
     defaults:
       run:
@@ -169,14 +178,29 @@ jobs:
           go-version-file: 'go.mod'
       - name: Install FUSE
         run: |
-          if ! command -v fusermount &>/dev/null; then
+          if ! command -v fusermount3 &>/dev/null && ! command -v fusermount &>/dev/null; then
             sudo apt-get update
             sudo apt-get install -y fuse3
-            # bazil.org/fuse looks for "fusermount", fuse3 only ships "fusermount3"
-            sudo ln -sf /usr/bin/fusermount3 /usr/local/bin/fusermount
           fi
+      - name: Clean up stale FUSE mounts
+        run: |
+          # On shared self-hosted runners, leftover mounts from previous
+          # runs can exhaust the kernel FUSE mount limit (mount_max).
+          # Unit tests mount with FsName "kubo-test"; CLI tests mount
+          # under the harness temp dir (ipfs/ipns/mfs subdirectories).
+          awk '$1 == "kubo-test" || $2 ~ /\/tmp\/.*\/(ipfs|ipns|mfs)$/ { print $2 }' /proc/mounts 2>/dev/null \
+            | while read -r mp; do
+                fusermount3 -uz "$mp" 2>/dev/null || fusermount -uz "$mp" 2>/dev/null || true
+              done
       - name: Run FUSE tests
         run: make test_fuse
+      - name: Clean up FUSE mounts
+        if: always()
+        run: |
+          awk '$1 == "kubo-test" || $2 ~ /\/tmp\/.*\/(ipfs|ipns|mfs)$/ { print $2 }' /proc/mounts 2>/dev/null \
+            | while read -r mp; do
+                fusermount3 -uz "$mp" 2>/dev/null || fusermount -uz "$mp" 2>/dev/null || true
+              done
 
   # Example tests (kubo-as-a-library)
   example-tests:

.gitignore

@@ -28,10 +28,12 @@ go-ipfs-source.tar.gz
 docs/examples/go-ipfs-as-a-library/example-folder/Qm*
 /test/sharness/t0054-dag-car-import-export-data/*.car
 
-# test artifacts from make test_unit / test_cli
+# test artifacts from make test_unit / test_cli / test_fuse
 /test/unit/gotest.json
 /test/unit/gotest.junit.xml
 /test/cli/cli-tests.json
+/test/fuse/fuse-unit-tests.json
+/test/fuse/fuse-cli-tests.json
 
 # ignore build output from snapcraft
 /ipfs_*.snap

AGENTS.md

@@ -124,10 +124,10 @@ If you see "version (N) is lower than repos (M)", the `ipfs` binary in `PATH` is
 
 ### Running FUSE Tests
 
-FUSE tests require `/dev/fuse` and `fusermount` in `PATH`. On systems with only fuse3, create a symlink:
+FUSE tests require `/dev/fuse` and `fusermount` in `PATH`. On systems with only fuse3, create a symlink in a temp directory (never use `sudo` to install system-wide):
 
 ```bash
-ln -s /usr/bin/fusermount3 /tmp/fusermount && PATH="/tmp:$PATH" make test_fuse
+FUSE_BIN="$(mktemp -d)" && ln -s /usr/bin/fusermount3 "$FUSE_BIN/fusermount" && PATH="$FUSE_BIN:$PATH" make test_fuse
 ```
 
 Set `TEST_FUSE=1` to make mount failures fatal (CI does this). Without it, tests auto-detect and skip when FUSE is unavailable.

README.md

@@ -34,6 +34,7 @@ Kubo was the first [IPFS](https://docs.ipfs.tech/concepts/what-is-ipfs/) impleme
 - [HTTP Gateway](https://specs.ipfs.tech/http-gateways/) for trusted and [trustless](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval) content retrieval
 - [HTTP RPC API](https://docs.ipfs.tech/reference/kubo/rpc/) to control the daemon
 - [HTTP Routing V1](https://specs.ipfs.tech/routing/http-routing-v1/) client and server for [delegated routing](./docs/delegated-routing.md)
+- [FUSE mounts](./docs/fuse.md) for mounting `/ipfs`, `/ipns`, and `/mfs` as local filesystems (experimental)
 - [Content blocking](./docs/content-blocking.md) for public node operators
 
 **Other IPFS implementations:** [Helia](https://github.com/ipfs/helia) (JavaScript), [more...](https://docs.ipfs.tech/concepts/ipfs-implementations/)
@@ -178,6 +179,7 @@ Kubo is available in community-maintained packages across many operating systems
 | [HTTP RPC clients](docs/http-rpc-clients.md) | Client libraries for Go, JS |
 | [Delegated routing](docs/delegated-routing.md) | Multi-router and HTTP routing |
 | [Metrics & monitoring](docs/metrics.md) | Prometheus metrics |
+| [FUSE mounts](docs/fuse.md) | Mount `/ipfs`, `/ipns`, `/mfs` as local filesystems |
 | [Content blocking](docs/content-blocking.md) | Denylist for public nodes |
 | [Customizing](docs/customizing.md) | Unsure if use Plugins, Boxo, or fork? |
 | [Debug guide](docs/debug-guide.md) | CPU profiles, memory analysis, tracing |

cmd/ipfs/kubo/daemon.go

@@ -1239,9 +1239,11 @@ func mountFuse(req *cmds.Request, cctx *oldcmds.Context) error {
 	if err != nil {
 		return err
 	}
+	// Extra space after "MFS" so "mounted at:" lines up with IPFS and
+	// IPNS in the column above. Matches MountCmd's output formatter.
 	fmt.Printf("IPFS mounted at: %s\n", fsdir)
 	fmt.Printf("IPNS mounted at: %s\n", nsdir)
-	fmt.Printf("MFS mounted at: %s\n", mfsdir)
+	fmt.Printf("MFS  mounted at: %s\n", mfsdir)
 	return nil
 }
 

cmd/ipfs/kubo/daemon_linux.go

@@ -1,3 +1,4 @@
+// Systemd readiness notification (sd_notify). Linux only.
 //go:build linux
 
 package kubo

cmd/ipfs/kubo/daemon_other.go

@@ -1,3 +1,4 @@
+// No-op readiness notification on non-Linux platforms.
 //go:build !linux
 
 package kubo

cmd/ipfs/runmain_test.go

@@ -1,3 +1,4 @@
+// Only built when collecting coverage via "go test -tags testrunmain".
 //go:build testrunmain
 
 package main_test

cmd/ipfs/util/signal.go

@@ -1,3 +1,4 @@
+// Signal handling. Excluded from wasm where os.Signal is unavailable.
 //go:build !wasm
 
 package util

cmd/ipfs/util/ui.go

@@ -1,3 +1,4 @@
+// GUI detection stub. Windows has its own implementation.
 //go:build !windows
 
 package util