Judgement percentage always 5% and 1%

[tmigot]

I am running PkgBenchmark.jl using the test file. Then,

commit = benchmarkpkg(repo_name)  # current state of repository
main = benchmarkpkg(repo_name, "main")
judgement = judge(commit, main)

commit_stats = bmark_results_to_dataframes(commit)
main_stats = bmark_results_to_dataframes(main)
judgement_stats = judgement_results_to_dataframes(judgement)
export_markdown("judgement_test.md", judgement)

The percentage of memory and time ratio in the results invariably return 5% and 1%:

## Results
A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less
than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

| ID                                                 | time ratio                   | memory ratio |
|----------------------------------------------------|------------------------------|--------------|
| `["trigonometry", "circular", "(\"cos\", 0.0)"]`   | 0.22 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "circular", "(\"cos\", π)"]`     | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "circular", "(\"sin\", 0.0)"]`   | 0.20 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "circular", "(\"sin\", π)"]`     | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "circular", "(\"tan\", 0.0)"]`   | 0.20 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "circular", "(\"tan\", π)"]`     | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"cos\", 0.0)"]` | 0.22 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"cos\", π)"]`   | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"sin\", 0.0)"]` | 0.20 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"sin\", π)"]`   | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"tan\", 0.0)"]` | 0.19 (5%) :white_check_mark: |   1.00 (1%)  |
| `["trigonometry", "hyperbolic", "(\"tan\", π)"]`   | 0.08 (5%) :white_check_mark: |   1.00 (1%)  |

Even when I am changing the benchmark or modifying the package.

[BoZenKhaa]

That's because the percentages are tolerances set in the BenchmarkTools.jl setup of the individual benchmarks, see the linked PR.

[tmigot]

Oh, I see. Thanks @BoZenKhaa !