Skip to content

Conversation

@marcbowes
Copy link
Contributor

Summary

Adds two related features behind unstable feature flags:

  1. Simulated filesystem (unstable-fs) - Drop-in replacements for std::fs and tokio::fs with crash-consistency testing support
  2. Barriers (unstable-barriers) - Observable synchronization points for testing (see Add barriers to turmoil #229)

Filesystem Usage

use turmoil::fs::shim::std::fs::{File, create_dir, write, sync_dir};

// Configure filesystem behavior
let mut sim = Builder::new()
    .fs()
        .sync_probability(0.1)  // 10% chance writes randomly sync
        .block_size(4096)       // Enable torn write simulation
    .build();

sim.client("db", async {
    create_dir("/data")?;
    write("/data/file.txt", b"hello")?;
    
    // POSIX durability: must sync both file AND directory
    File::open("/data/file.txt")?.sync_all()?;
    sync_dir("/data")?;  // Directory entry now durable
    
    Ok(())
});

// Simulate crash - unsynced data is lost
sim.crash("db");

Key Design Choices

POSIX durability model: File data and directory entries are tracked separately. sync_all() makes file data durable; sync_dir() makes directory entries durable. Both are required for a new file to survive a crash.

Torn writes: With block_size configured, pending writes may be partially applied on crash (0 to N blocks survive), simulating real disk behavior.

I/O latency + page cache (tokio shim only): Async operations can simulate realistic I/O timing with cache hits (~100ns) vs misses (configurable latency). O_DIRECT bypasses the cache. Sync operations (std::fs shim) complete immediately.

Worker thread support: FsHandle::current() captures context for use in spawn_blocking threads.

Limitations

  • Permissions are informational only (not enforced)
  • All paths must be absolute
  • No file locking (flock/fcntl)
  • I/O latency simulation only available for async (tokio::fs) operations

Test Coverage

86 tests covering durability, crash recovery, torn writes, symlinks, hard links, metadata, page cache, and worker threads.

@marcbowes marcbowes requested a review from mcches December 8, 2025 17:55
@mcches
Copy link
Contributor

mcches commented Dec 8, 2025

#15 for tracking

Copy link
Contributor

@mcches mcches left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing is blocking. Thank you for this contribution.

Comment on lines 155 to 158
pub struct FsHandleGuard {
_private: (),
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub struct FsHandleGuard {
_private: (),
}
pub struct FsHandleGuard;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is pub, keeping the private field prevents external construction (users must go through FsHandle::enter()). This also preserves forward compatibility if we ever need to add state to the guard. Added a comment explaining this.

src/fs/mod.rs Outdated
}

/// Clear entire cache.
#[allow(dead_code)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed - YAGNI.

src/fs/mod.rs Outdated
Comment on lines 927 to 929
/// - Files in `persisted_files` but not in `synced_entries` are orphaned on crash
pub(crate) struct Fs {
/// Persisted file data (survives crash)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These comments conflict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - the comments were ambiguous. Updated to clarify the inode vs directory entry distinction: data in persisted_* survives crashes, but is only reachable if the path is also in synced_entries. Without a synced directory entry, the data is orphaned and cleaned up on crash.

/// O_DIRECT flag value for bypassing page cache.
/// Platform-specific: Linux=0x4000, macOS doesn't have O_DIRECT (we use F_NOCACHE via fcntl).
/// For our simulation, we just need a consistent non-zero value users can pass.
#[cfg(target_os = "linux")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why bother with cfg here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point - simplified. All branches had the same value because this is a simulation constant, not the real platform flag. macOS doesn't have O_DIRECT at all (uses fcntl F_NOCACHE), so we just use Linux's 0x4000 as the canonical value for portability.

Filesystem (`unstable-fs` feature):
- Drop-in replacements for std::fs and tokio::fs with durability simulation
- POSIX semantics: separate file data sync (fsync) and directory entry sync
- Crash consistency testing: pending writes lost on crash unless synced
- Configurable behaviors: sync probability, capacity limits, I/O errors,
  silent corruption, torn writes, I/O latency, and page cache simulation
- Per-host isolation with FsHandle for worker thread support

Barriers (`unstable-barriers` feature):
- Inject hooks at specific code points for deterministic test control
- Support for suspend/resume, panic injection, and observation
- Integrates with fs corruption events for targeted testing
@marcbowes marcbowes merged commit 0b8edea into tokio-rs:main Dec 8, 2025
6 checks passed
@marcbowes marcbowes deleted the feat-fs branch December 8, 2025 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants