You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The cost of starting and stopping the timers is substantial, about 0.5μs for a start/stop pair, due to the overhead of reading the performance counters in the kernel. In the case you use manual time measurement with start/stop_measuring_time you should ensure that the timed region is substantially longer than time in order to reduce the error imposed by this overhead: for example by using a loop inside the timed region and returning the number of iterations from the test method.
112
+
The cost of starting and stopping the timers is substantial, about 1μs for a start/stop pair, due to the overhead of reading the performance counters in the kernel. When using manual time measurement with `start_measuring_time()`/`stop_measuring_time()`, you should ensure that the timed region is substantially longer than this overhead to reduce measurement error. A common approach is to use a loop inside the timed region and return the number of iterations from the test method.
111
113
112
114
If you do _not_ use these manual methods, the overhead is very low (a few instructions) as the test method is already called in such a loop by the framework, with the time manipulation methods outside that.
115
+
116
+
##### Overhead column
117
+
118
+
The framework tracks and reports the estimated measurement overhead as a percentage in the `overhead` column. This is calculated by:
119
+
120
+
1. Calibrating the cost of a single `start_measuring_time()`/`stop_measuring_time()` pair at startup
121
+
2. Counting how many times these functions are called during each test run
A high overhead percentage (e.g., >10%) indicates that the timing instrumentation is consuming a significant portion of the measured time, which reduces the accuracy of the results. This typically happens when:
125
+
- The timed region is very short (comparable to the ~1μs overhead)
126
+
-`start_measuring_time()`/`stop_measuring_time()` are called many times with little work between them
127
+
128
+
To reduce overhead, either:
129
+
- Increase the amount of work in each timed region
130
+
- Use an inner loop and return the iteration count, rather than calling start/stop for each iteration
131
+
132
+
##### Overhead warnings
133
+
134
+
By default, a warning is printed if any test has median overhead exceeding 10%:
135
+
136
+
```
137
+
WARNING: test 'example.my_test' has high measurement overhead: 15.2% (threshold: 10.0%)
138
+
```
139
+
140
+
You can adjust the threshold with `--overhead-threshold <percent>`, or fail the test run entirely when overhead is too high with `--fail-on-high-overhead`.
0 commit comments