Update Mountpoint's semantics to enable opening a new file handle on an inode if all the existing open handles have been flushed (#1704)

Update Mountpoint's semantics to enable opening a new file handle on an
inode if **all** the existing open handles have been flushed.

This allows for the filesystem to not depend on a `release` request to
complete pending (delayed) upload and cleanup the internal state of the
inode within Mountpoint; alleviating situations where an `open` request
made immediately after a close(`flush`) request leads to a race
condition between the `release` following the close and the new `open`,
which is sometimes an issue when the upload to S3 has been delayed until
`release`.

### Does this change impact existing behavior?

The semantics continue to allow **only one writer OR one/many readers**
concurrently active for a file; however, now a new handle can be opened
which can override the current active handles (despite not being
officially released) if all the active handles are marked "flushed".
As part of opening the new handle, Mountpoint will also attempt
uploading any pending data written for the previous file handle. All the
requests to the overridden handle(s) will then start to fail or be
no-op.
Multiple concurrent writers or concurrent readers and writers are still
not allowed.

A handle is marked "flushed" when a close/`flush` is called on a file
descriptor mapped to that handle. A following `read`/`write` request
will revert that flushed state and signify that the handle is actively
in use and can not be overridden. We maintain this information at the
individual handle level and also in an inode-locked map of handles.

Breaking changes:
- Requests made to a duplicate file descriptor for a flushed file handle
will start to fail (for e.g. a `read`/`write` would fail with `EBADF:
file handle has been invalidated by a newer handle opened`) or be no-op
(for e.g. `flush`, `release`) if a new `open` has overridden the flushed
handle(s).
- A race condition can occur between a `read`/`write` request for a
duplicate file descriptor on an existing (flushed) handle and multiple
concurrent `open` requests, and any of them might succeed due to
parallel processing of FUSE requests within Mountpoint. However, only
one of them will ever succeed and there cannot be two concurrent writers
or reader+writer for the inode at any point in time.
- An `open` request for an inode might fail if the pending upload to S3
fails. This is independent of whether the file has been truncated in the
second `open`.

### Does this change need a changelog entry? Does it require a version
change?

Yes, it needs a changelog entry and update to the semantics.md. Yes it
also requires a version change.

---

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and I agree to the terms of
the [Developer Certificate of Origin
(DCO)](https://developercertificate.org/).

---------

Signed-off-by: Mansi Pandey <mansipnd@amazon.co.uk>
Signed-off-by: Alessandro Passaro <alexpax@amazon.co.uk>
Co-authored-by: Mansi Pandey <mansipnd@amazon.com>
Co-authored-by: Alessandro Passaro <alexpax@amazon.co.uk>

Author: 3 people

Cargo.lock

@@ -210,6 +210,12 @@ impl ProvideErrorMetadata for RenameObjectError {
     }
 }
 
+impl ProvideErrorMetadata for PutObjectError {
+    fn meta(&self) -> ClientErrorMetadata {
+        Default::default()
+    }
+}
+
 /// Shorthand type for the result of an object client request
 pub type ObjectClientResult<T, S, C> = Result<T, ObjectClientError<S, C>>;
 

mountpoint-s3-client/src/object_client.rs

@@ -1,5 +1,6 @@
-## Unreleased (v0.8.5)
+## Unreleased (v0.9.0 (January 06, 2025))
 
+* Update Mountpoint's semantics to enable opening a new file handle on an inode if all the existing open handles have been flushed. ([#1704](https://github.com/awslabs/mountpoint-s3/pull/1704))
 * Upgrade cargo dependencies.
 
 ## v0.8.4 (December 22, 2025)

mountpoint-s3-fs/CHANGELOG.md

@@ -1,7 +1,7 @@
 [package]
 name = "mountpoint-s3-fs"
 # See `/doc/PUBLISHING_CRATES.md` to read how to publish new versions.
-version = "0.8.5"
+version = "0.9.0"
 edition = "2024"
 license = "Apache-2.0"
 repository = "https://github.com/awslabs/mountpoint-s3"

mountpoint-s3-fs/Cargo.toml

@@ -188,6 +188,7 @@ impl ToErrno for InodeError {
             InodeError::OperationNotSupportedOnSyntheticInode { .. } => libc::EIO,
             InodeError::OutOfOrderReadDir { .. } => libc::EBADF,
             InodeError::NoSuchDirHandle { .. } => libc::EINVAL,
+            InodeError::FlexibleRetrievalObjectNotAccessible(_) => libc::EACCES,
         }
     }
 }

mountpoint-s3-fs/src/fs.rs

@@ -85,6 +85,18 @@ libc_flags! {
     }
 }
 
+impl OpenFlags {
+    #[cfg(not(target_os = "linux"))]
+    pub fn direct_io(&self) -> bool {
+        false
+    }
+
+    #[cfg(target_os = "linux")]
+    pub fn direct_io(&self) -> bool {
+        self.contains(OpenFlags::O_DIRECT)
+    }
+}
+
 /// Flags used in [rename](super::S3Filesystem::rename).
 #[derive(Clone, Copy, PartialEq, Eq)]
 pub struct RenameFlags(u32);