Results are not repeatable

[lamyergeier]

Following test are conducted on same external harddrive (same partition) on the same laptop (running Linuxmint 20.3) from same port. We can observe that the test shows widely varying values for Sequential and random read and Memory copy. Is it expected?

External HDD 1 (NTFS partition)

Test 1

1/5 Sequential write [4MB] block Avg: 45.46MB/s Min÷Max: 21.86 ÷ 87.74, Time: 0m24s
2/5 Sequential read [4MB] block Avg: 426.19MB/s Min÷Max: 76.47 ÷ 2570.87, Time: 0m03s
3/5 Random write [4KB] block Avg: 1.01MB/s Min÷Max: 0.18 ÷ 8.83, Time: 0m07s
4/5 Random read [4KB] block Avg: 115.80MB/s Min÷Max: 0.12 ÷ 1584.69, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 3772.37MB/s Min÷Max: 982.27 ÷ 10246.98, Time: 0m04s

Test 2

1/5 Sequential write [4MB] block Avg: 44.06MB/s Min÷Max: 18.34 ÷ 79.22, Time: 0m25s
2/5 Sequential read [4MB] block Avg: 976.09MB/s Min÷Max: 115.66 ÷ 3086.32, Time: 0m02s
3/5 Random write [4KB] block Avg: 1.05MB/s Min÷Max: 0.18 ÷ 8.71, Time: 0m07s
4/5 Random read [4KB] block Avg: 3.71MB/s Min÷Max: 0.11 ÷ 262.38, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 3452.52MB/s Min÷Max: 1131.59 ÷ 10590.42, Time: 0m04s

Test 3

1/5 Sequential write [4MB] block Avg: 44.97MB/s Min÷Max: 20.61 ÷ 106.92, Time: 0m25s
2/5 Sequential read [4MB] block Avg: 527.74MB/s Min÷Max: 81.72 ÷ 3118.75, Time: 0m03s
3/5 Random write [4KB] block Avg: 1.08MB/s Min÷Max: 0.19 ÷ 8.68, Time: 0m07s
4/5 Random read [4KB] block Avg: 8.68MB/s Min÷Max: 0.13 ÷ 293.81, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 2214.92MB/s Min÷Max: 622.90 ÷ 8879.18, Time: 0m04s

External HDD 2: (ExFat partition)

Test 1

1/5 Sequential write [4MB] block Avg: 103.01MB/s Min÷Max: 51.06 ÷ 164.85, Time: 0m11s
2/5 Sequential read [4MB] block Avg: 943.43MB/s Min÷Max: 375.20 ÷ 2438.07, Time: 0m02s
3/5 Random write [4KB] block Avg: 0.97MB/s Min÷Max: 0.22 ÷ 10.84, Time: 0m07s
4/5 Random read [4KB] block Avg: 2.22MB/s Min÷Max: 0.43 ÷ 206.53, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 2351.91MB/s Min÷Max: 520.16 ÷ 8831.35, Time: 0m04s

Test 2

1/5 Sequential write [4MB] block Avg: 101.16MB/s Min÷Max: 50.04 ÷ 162.04, Time: 0m12s
2/5 Sequential read [4MB] block Avg: 425.75MB/s Min÷Max: 59.78 ÷ 2843.71, Time: 0m03s
3/5 Random write [4KB] block Avg: 1.00MB/s Min÷Max: 0.02 ÷ 11.98, Time: 0m07s
4/5 Random read [4KB] block Avg: 1.68MB/s Min÷Max: 0.13 ÷ 197.63, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 5501.92MB/s Min÷Max: 454.50 ÷ 10996.59, Time: 0m03s

Test 3

1/5 Sequential write [4MB] block Avg: 104.20MB/s Min÷Max: 48.11 ÷ 145.57, Time: 0m11s
2/5 Sequential read [4MB] block Avg: 946.09MB/s Min÷Max: 471.89 ÷ 2563.48, Time: 0m02s
3/5 Random write [4KB] block Avg: 0.99MB/s Min÷Max: 0.09 ÷ 12.11, Time: 0m07s
4/5 Random read [4KB] block Avg: 2.30MB/s Min÷Max: 0.31 ÷ 208.18, Time: 0m07s
5/5 Memory copy [4MB] block Avg: 3100.12MB/s Min÷Max: 305.35 ÷ 10961.79, Time: 0m04s

[maxim-saplin]

 Indeed, write tests are stable while read tests are not. Most likely that's related to cahced reads and OS ignoring apps request not to use file cache:
 

if (RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Linux))
  Syscall.posix_fadvise((int)SafeFileHandle.DangerousGetHandle(), 0, 0, PosixFadviseAdvice.POSIX_FADV_DONTNEED);

There's no easy fix for that other than experimenting with various hacks and trying it out on your config to see which hack plays out and disables cached reads.

[lamyergeier]

If cache is the issue, am I correct in assuming that the first measurements are correct and subsequent measurements not for each device. Also, can cache affect the calculations of the first measurement itself? 21 May 2022 20:24:24 Maxim Saplin ***@***.***>:

…

 Indeed, write tests are stable while read tests are not. Most likely that's related to cahced reads and OS ignoring apps request not to use file cache:   *  if (RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Linux)) Syscall.posix_fadvise((int)SafeFileHandle.DangerousGetHandle(), 0, 0, PosixFadviseAdvice.POSIX_FADV_DONTNEED); * There's no easy fix for that other than experimenting with various hacks and trying it out on your config to see which hack plays out and disables cached reads. — Reply to this email directly, view it on GitHub[#10 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKBEQUVK54OIZV4STPVQAV3VLES5PANCNFSM5WSD3YAA]. You are receiving this because you authored the thread.[Tracking image][https://github.com/notifications/beacon/AKBEQUW2WOL3HDCOHCSHO7TVLES5PA5CNFSM5WSD3YAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIOJZ2VI.gif]

[maxim-saplin]

That is not correct. Please notice that all read tests follow write ones. The write tests essentialy create the test file and thus let OS cache this file. Read tests do their measurements against file created by the very first Sequential Write test.

[lamyergeier]

Will it be better if the software creates a new test file after each write or read? 22 May 2022 11:24:30 Maxim Saplin ***@***.***>:

…

That is not correct. Please notice that all read tests follow write ones. The write tests essentialy create the test file and thus let OS cache this file. Read tests do their measurements against file created by the very first Sequential Write test. — Reply to this email directly, view it on GitHub[#10 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKBEQUXAMK6Z2VIOCFQKKCLVLH4M3ANCNFSM5WSD3YAA]. You are receiving this because you authored the thread.[Tracking image][https://github.com/notifications/beacon/AKBEQUSCILQGQPUY7GGDXFLVLH4M3A5CNFSM5WSD3YAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIOKT6EA.gif]

[maxim-saplin]

Software always creates a new file when starting a test session and deletes it afterwards. You can't workaround file caching with any sequences of creation of file. The file is writen twice (Seq and Random test) and read twice (Seq and Random tests).

If OS doesn't respect cache invalidation there're 2 ways to go:

1. Do a write test, reboot, do the consequent read test
2. Fill RAM with random data hoping that OS will drop most of the caches - that's how the app works on Android

Neither of the options is implemented in Linux port

[lamyergeier]

Actually, I meant within a test session for each write and each read, the software could create a new test file. 22 May 2022 11:45:29 Maxim Saplin ***@***.***>:

…

Software always creates a new file when starting a test session and deletes it afterwards. You can't workaround file caching with any sequences of creation of file. The file is writen twice and read twice. — Reply to this email directly, view it on GitHub[#10 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKBEQUXMA5YLKP24JASWJFTVLH63RANCNFSM5WSD3YAA]. You are receiving this because you authored the thread.[Tracking image][https://github.com/notifications/beacon/AKBEQUU5G5ZC34FB3ZAM65DVLH63RA5CNFSM5WSD3YAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIOKU2EQ.gif]

[maxim-saplin]

And I tried to explain it doesn't matter how many files you create, as long as read tests do read what has just been written

[maxim-saplin]

Anyways, the best course of action is finding out how to make the OS ignore file cache and set up test file the proper way. There might be some distro specific API call.

For the moment I don't have capacity to follow-up with reproduction and investigation. Keeping the issue open