FileStream performance guidelines

[samuel-lucas6]

Back in 2021, there was a .NET blog post asking whether people were interested in reading .NET File IO performance guidelines. Several people expressed an interest in the comments, and I'm also interested.

1. Was this ever written? It would be a beneficial resource. (tagging @adamsitnik since he wrote the post)
2. If not, please could someone with expertise explain how to optimise FileStream performance for different file sizes (e.g. the bufferSize, preallocation, SequentialScan vs RandomAccess, SetLength, and so on). I'm not interested in async but others might be.

[huoyaoyuan]

@dotnet/area-system-io
I'm also interested in this. Generally, good API design should make best practices simple and straight.

[AlgorithmsAreCool]

Not an answer but I've had a lot of success using the new Random Access class instead of trying to guess the best way to get FileStream to buffer. I also found that the Sync overloads were a bit faster than the Async versions, but that gets tricker if your workload is processing a lot of concurrent requests.

[samuel-lucas6]

Ok I'm in the process of doing benchmarks with random files on my machine, and this is what I've found for reading files sequentially into a 16 KiB span:

• For small files (e.g. bytes to 128 KiB), a buffer size of 0 seems to be best, closely followed by 2,048 and 4,096. SequentialScan does nothing.
• For 512 KiB files, a buffer size of 65,536, closely followed by 81,920 seems best. SequentialScan does nothing.
• For 1 MiB files, a buffer size of 81,920 seems best, closely followed by 65,536. SequentialScan does nothing.
• For 10 MiB files, a buffer size of 131,072 seems best, closely followed by 262,144. SequentialScan does nothing.
• For 32 MiB files, a buffer size of 131,072 seems best, closely followed by 524,288 and 262,144. SequentialScan does nothing.
• For 100 MiB files, a buffer size of 524,288 seems best, closely followed by 1,048,576 and 262,144. SequentialScan does nothing.
• For 781 MiB, a buffer size of 1,048,576 is best, closely followed by 524,288. SequentialScan does nothing.
• For 1.7 GiB and 7.19 GiB, 1,048,576 is still best, followed by 524,288. SequentialScan does nothing.

Conclusions:

• Reading only the first few bytes from a file is most efficient with a buffer size of 0.
• Using a large buffer size compared to small files is bad.
• Using a large buffer size (e.g. 10 MiB) for large files (e.g. 1.7 GiB) is bad.
• Using a buffer size as large as the file (e.g. 1 MiB buffer for 1 MiB files) is bad.
• Using a small buffer size (e.g. 4,096) or no buffering for large files is bad.
• The default of 4,096 is not optimal for many file sizes, and neither is 81,920, which is used by CopyTo().
• SequentialScan results are either slower or barely any different. Perhaps it's better for very large files.

If someone else could confirm these sort of figures are accurate, that would be greatly appreciated. I'm inexperienced with benchmarking.

Now I have to do writing to files.

[danmoseley]

Thanks for sharing. What OS were you using? Trade offs can vary by OS and file system.

[samuel-lucas6]

I believe that was on Windows 10.

[samuel-lucas6]

Was this ever written? It would be a beneficial resource.

Managed to find this PDF. There's also a video. Both by @adamsitnik.

Preallocating for writing seems to do very little in my benchmarks, but increasing the user buffer from 16 KiB to 64 KiB for reading makes a big difference, especially with large files.

[huoyaoyuan]

There may also be difference caused by storage device (HDD or SSD) and OS cache strategy. I'd like to see a conclusion suitable for major type of scenarios.

[AlexeiScherbakov]

I recently discover that for network shares large buffer can help improve perfomance.
For 10 years I used 64kbytes buffers and it was optimal for local disks. But recently I detected problem with file operation on network share on 10Gbit network, especially with write through flag. Now for files in remote shares I use 1-8Mb buffers. On old SMB servers 64Kbyte was max chunk, today SMB2+ block size can be up to 1048576-8388608 (https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/a64e55aa-1152-48e4-8206-edd96444e7f7#Appendix_A_259). In fast networks it is better to write data in one large single block

Network access is not covered with any tests, but filestreams sometimes can access files on network shares

[AartBluestoke]

Network access is not covered with any tests - and how many times you are exposed to network latency will be a major contributor here - moderate size chunks proactively fetched ahead of the current read could be best - and the cluster of disks typically at the end of a network fetch could allow parallel requests to be more efficiently serviced than any local access pattern ...