20251023 steady state add latency #2003
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
|
Seems like some rebasing went awry. |
mcgrof
force-pushed
the
20251023-steady-state-add-latency
branch
2 times, most recently
from
6604c32 to
61894dd
Compare
Author
yikes yes, sorry about that. Fixed and and pushed. |
vincentkfu
reviewed
Oct 28, 2025
mcgrof
force-pushed
the
20251023-steady-state-add-latency
branch
from
61894dd to
3954bbb
Compare
vincentkfu
reviewed
Oct 31, 2025
vincentkfu
reviewed
Oct 31, 2025
Collaborator
|
There is still a bit more than needs to be added to get this working in client/server mode.
|
vincentkfu
reviewed
Oct 31, 2025
mcgrof
force-pushed
the
20251023-steady-state-add-latency
branch
from
3954bbb to
950daee
Compare
gnutls is required newwer version of libnfs, and so we need to check for this. libnfs version 6.0.0 and later added TLS support. Earlier versions like 5.0.x do not require gnutls. Add a version check in the configure script to detect the libnfs version and only add gnutls as a dependency when libnfs >= 6.0.0 is detected. For systems with older libnfs versions, the build will work without requiring gnutls. For systems with libnfs 6.0.0+, gnutls will be properly detected and linked. Generated-by: Claude AI Reviewed-by: Vincent Fu <[email protected]> Signed-off-by: Luis Chamberlain <[email protected]>
Extract the long if condition in str_steadystate_cb() into a separate helper function is_valid_steadystate() for better readability and maintainability. Generated-by: Claude AI Reviewed-by: Vincent Fu <[email protected]> Signed-off-by: Luis Chamberlain <[email protected]>
Extract the common logic from steadystate_bw_mean() and steadystate_iops_mean() into a helper function steadystate_data_mean() to reduce code duplication and improve maintainability. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <[email protected]>
Add fio latency steady state support. The implementation calculates
weighted average latency across all I/O directions and supports both
maximum mean deviation and slope-based detection methods.
We bump FIO_SERVER_VER since struct thread_stat has changed.
Tested successfully against NVMe device with debug output confirming
proper latency calculation and steady state evaluation.
Quick demo with the null engine:
fio --name=test --ioengine=null --size=1G --rw=randread --bs=4k --iodepth=64 --steadystate=lat:100us --ss_dur=10s --runtime=120s --time_based=1
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=64
fio-3.41-20-gf2b2e-dirty
Starting 1 process
Jobs: 1 (f=1): [f(1)][100.0%][r=15.6GiB/s][r=4078k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=1207074: Tue Oct 7 11:47:27 2025
read: IOPS=4054k, BW=15.5GiB/s (16.6GB/s)(153GiB/9899msec)
slat (nsec): min=18, max=33347, avg=22.20, stdev=64.49
clat (nsec): min=135, max=42283, avg=152.74, stdev=173.53
lat (nsec): min=156, max=42306, avg=174.94, stdev=185.32
clat percentiles (nsec):
| 1.00th=[ 139], 5.00th=[ 141], 10.00th=[ 141], 20.00th=[ 143],
| 30.00th=[ 143], 40.00th=[ 145], 50.00th=[ 147], 60.00th=[ 153],
| 70.00th=[ 155], 80.00th=[ 157], 90.00th=[ 163], 95.00th=[ 171],
| 99.00th=[ 217], 99.50th=[ 221], 99.90th=[ 239], 99.95th=[ 270],
| 99.99th=[ 2096]
bw ( MiB/s): min=15344, max=15944, per=100.00%, avg=15835.92, stdev=132.94, samples=19
iops : min=3928298, max=4081694, avg=4053996.63, stdev=34033.65, samples=19
lat (nsec) : 250=99.93%, 500=0.06%, 750=0.01%, 1000=0.01%
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=99.93%, sys=0.03%, ctx=88, majf=0, minf=6
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=40129848,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
steadystate : attained=yes, bw=15.2GiB/s (15.9GB/s), iops=3976k, lat=152nsec, lat mean dev=2.700
Run status group 0 (all jobs):
READ: bw=15.5GiB/s (16.6GB/s), 15.5GiB/s-15.5GiB/s (16.6GB/s-16.6GB/s), io=153GiB (164GB), run=9899-9899msec
Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <[email protected]>
Introduce a mock test framework to validate specific algorithms and calculations in isolation. This allows testing numerical precision, edge cases, and algorithmic correctness without requiring the full FIO infrastructure. Features: - TAP (Test Anything Protocol) output for CI integration - Isolated testing of specific calculations - Clear documentation of test purposes and goals - Simple make target: 'make mock-tests' The first test validates the numerical precision improvements in steady state latency calculations, demonstrating how the framework can catch subtle precision and overflow issues. Structure: - mock-tests/lib/tap.h: TAP output support - mock-tests/tests/: Individual test programs - mock-tests/Makefile: Build system - mock-tests/README.md: Comprehensive documentation This framework complements the existing integration tests by focusing on unit-level validation of critical calculations. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <[email protected]>
When fio has issued 884 billion IOs it is still aable to recover per second latencies with less than 0.1% error using a latency value of 13ms. This is evidence that fio's strategy for estimating per second latency for steady state detection works reasonably well. Signed-off-by: Vincent Fu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds fio steady state latency support.