Skip to content

NIOFileSystem: Add on-disk size to FileInfo #3656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thliu21 wants to merge 3 commits into
base: main
Choose a base branch
from
+54 −0
Open

NIOFileSystem: Add on-disk size to FileInfo #3656

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2d59d27
Add on-disk size to FileInfo
thliu21 Jul 10, 2026
62084dc
Add sparse file on-disk size test
thliu21 Jul 16, 2026
93757ec
Guard FileInfo onDiskSize on Windows
thliu21 Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions Sources/NIOFS/FileInfo.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ import CNIOLinux
import CNIOLinux
#endif

#if canImport(Darwin)
private let S_BLKSIZE = Darwin.S_BLKSIZE
#elseif canImport(Glibc)
private let S_BLKSIZE = Glibc.S_BLKSIZE
#elseif canImport(Musl)
private let S_BLKSIZE = Musl.DEV_BSIZE
#elseif canImport(Android)
private let S_BLKSIZE = Android.S_BLKSIZE
#endif

/// Information about a file system object.
///
/// The information available for a file depends on the platform, ``FileInfo`` provides
Expand Down Expand Up @@ -59,6 +69,11 @@ public struct FileInfo: Hashable, Sendable {
/// The size of the file in bytes.
public var size: Int64

/// The number of bytes allocated for the file on disk.
///
/// This may differ from ``size`` for files which are sparse or compressed.
public var onDiskSize: Int64

/// User ID of the file.
public var userID: UserID

Expand All @@ -76,6 +91,9 @@ public struct FileInfo: Hashable, Sendable {

/// Creates a ``FileInfo`` by deriving values from a platform-specific value.
public init(platformSpecificStatus: CInterop.Stat) {
#if os(Windows)
fatalError("FileInfo.onDiskSize is not implemented on Windows")
#endif
self._platformSpecificStatus = Stat(platformSpecificStatus)
#if os(Windows)
self.type = FileType(platformSpecificMode: platformSpecificStatus.nioMode)
Expand All @@ -91,6 +109,7 @@ public struct FileInfo: Hashable, Sendable {
self.type = FileType(platformSpecificMode: CInterop.Mode(platformSpecificStatus.st_mode))
self.permissions = FilePermissions(masking: CInterop.Mode(platformSpecificStatus.st_mode))
self.size = Int64(platformSpecificStatus.st_size)
self.onDiskSize = Int64(S_BLKSIZE) * Int64(platformSpecificStatus.st_blocks)
self.userID = UserID(rawValue: platformSpecificStatus.st_uid)
self.groupID = GroupID(rawValue: platformSpecificStatus.st_gid)

Expand All @@ -114,6 +133,7 @@ public struct FileInfo: Hashable, Sendable {
type: FileType,
permissions: FilePermissions,
size: Int64,
onDiskSize: Int64,
userID: UserID,
groupID: GroupID,
lastAccessTime: Timespec,
Expand All @@ -124,6 +144,7 @@ public struct FileInfo: Hashable, Sendable {
self.type = type
self.permissions = permissions
self.size = size
self.onDiskSize = onDiskSize
self.userID = userID
self.groupID = groupID
self.lastAccessTime = lastAccessTime
Expand Down
21 changes: 21 additions & 0 deletions Sources/_NIOFileSystem/FileInfo.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ import CNIOLinux
import CNIOLinux
#endif

#if canImport(Darwin)
private let S_BLKSIZE = Darwin.S_BLKSIZE
#elseif canImport(Glibc)
private let S_BLKSIZE = Glibc.S_BLKSIZE
#elseif canImport(Musl)
private let S_BLKSIZE = Musl.DEV_BSIZE
#elseif canImport(Android)
private let S_BLKSIZE = Android.S_BLKSIZE
#endif

@Lukasa Lukasa Jul 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakepetroules To avoid an in-flight regression of Windows, can you identify what the appropriate operation is there?

@thliu21 thliu21 Jul 21, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the Windows port. The corresponding API for actual allocated bytes is GetCompressedFileSizeW. Because FileInfo currently receives only BY_HANDLE_FILE_INFORMATION while that API is path-based, I need to thread the value through the path/handle information flow and add a Windows sparse-file test rather than add an inaccurate fallback.


/// Information about a file system object.
///
/// The information available for a file depends on the platform, ``FileInfo`` provides
Expand Down Expand Up @@ -59,6 +69,11 @@ public struct FileInfo: Hashable, Sendable {
/// The size of the file in bytes.
public var size: Int64

/// The number of bytes allocated for the file on disk.
///
/// This may differ from ``size`` for files which are sparse or compressed.
public var onDiskSize: Int64

/// User ID of the file.
public var userID: UserID

Expand All @@ -76,6 +91,9 @@ public struct FileInfo: Hashable, Sendable {

/// Creates a ``FileInfo`` by deriving values from a platform-specific value.
public init(platformSpecificStatus: CInterop.Stat) {
#if os(Windows)
fatalError("FileInfo.onDiskSize is not implemented on Windows")
#endif
self._platformSpecificStatus = Stat(platformSpecificStatus)
#if os(Windows)
self.type = FileType(platformSpecificMode: platformSpecificStatus.nioMode)
Expand All @@ -91,6 +109,7 @@ public struct FileInfo: Hashable, Sendable {
self.type = FileType(platformSpecificMode: CInterop.Mode(platformSpecificStatus.st_mode))
self.permissions = FilePermissions(masking: CInterop.Mode(platformSpecificStatus.st_mode))
self.size = Int64(platformSpecificStatus.st_size)
self.onDiskSize = Int64(S_BLKSIZE) * Int64(platformSpecificStatus.st_blocks)
self.userID = UserID(rawValue: platformSpecificStatus.st_uid)
self.groupID = GroupID(rawValue: platformSpecificStatus.st_gid)

Expand All @@ -114,6 +133,7 @@ public struct FileInfo: Hashable, Sendable {
type: FileType,
permissions: FilePermissions,
size: Int64,
onDiskSize: Int64,
userID: UserID,
groupID: GroupID,
lastAccessTime: Timespec,
Expand All @@ -124,6 +144,7 @@ public struct FileInfo: Hashable, Sendable {
self.type = type
self.permissions = permissions
self.size = size
self.onDiskSize = onDiskSize
self.userID = userID
self.groupID = groupID
self.lastAccessTime = lastAccessTime
Expand Down
11 changes: 11 additions & 0 deletions Tests/NIOFSIntegrationTests/FileHandleTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1228,6 +1228,17 @@ final class FileHandleTests: XCTestCase {
}
}

func testOnDiskSizeOfSparseFile() async throws {
try await self.withTemporaryFile { handle in
try await handle.write(contentsOf: Array("hi".utf8), toAbsoluteOffset: 0)
try await handle.resize(to: .mebibytes(128))

let info = try await handle.info()
XCTAssertEqual(info.size, 128 * 1024 * 1024)
XCTAssertLessThan(info.onDiskSize, 1024 * 1024)
}
}

func testResizeFileErrors() async throws {
try await self.withTemporaryFile { handle in
try await handle.write(
Expand Down
1 change: 1 addition & 0 deletions Tests/NIOFSTests/FileInfoTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ final class FileInfoTests: XCTestCase {
XCTAssertEqual(info.userID, FileInfo.UserID(rawValue: 5))
XCTAssertEqual(info.groupID, FileInfo.GroupID(rawValue: 6))
XCTAssertEqual(info.size, 8)
XCTAssertEqual(info.onDiskSize, 9 * 512)

@weissi weissi Jul 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we maybe add a test case which creates a file with a hole? Should be pretty easy: FileSystem.shared.withFileOpen(forWriting: ".../stuff") { handle in handle.write("hi"); handle.resize(128 MB) } then check that the on-disk size is under 1MB for UNIX platforms

@thliu21 thliu21 Jul 16, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an integration regression test that writes hi, resizes the file to 128 MiB, and verifies the logical size while asserting the on-disk allocation remains below 1 MiB.


XCTAssertEqual(info.lastAccessTime, FileInfo.Timespec(seconds: 0, nanoseconds: 0))
XCTAssertEqual(info.lastDataModificationTime, FileInfo.Timespec(seconds: 1, nanoseconds: 0))
Expand Down