ci: Add codspeed for performance monitoring

[FBruzzesi]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Related issue [Enh]: Better benchmarking routine #805

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕 22 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_benchmark_scripts[query_path0] (323.5 µs)
• test_benchmark_scripts[query_path10] (322.9 µs)
• test_benchmark_scripts[query_path11] (325 µs)
• test_benchmark_scripts[query_path12] (327.3 µs)
• test_benchmark_scripts[query_path13] (319.7 µs)
• test_benchmark_scripts[query_path14] (320.7 µs)
• test_benchmark_scripts[query_path15] (321 µs)
• test_benchmark_scripts[query_path16] (323 µs)
• test_benchmark_scripts[query_path17] (319.8 µs)
• test_benchmark_scripts[query_path18] (320.7 µs)
• test_benchmark_scripts[query_path19] (319.5 µs)
• test_benchmark_scripts[query_path1] (320.4 µs)
• test_benchmark_scripts[query_path20] (320.6 µs)
• test_benchmark_scripts[query_path21] (375.8 µs)
• test_benchmark_scripts[query_path2] (319.8 µs)
• test_benchmark_scripts[query_path3] (322.5 µs)
• test_benchmark_scripts[query_path4] (321.9 µs)
• test_benchmark_scripts[query_path5] (320.2 µs)
• test_benchmark_scripts[query_path6] (319 µs)
• test_benchmark_scripts[query_path7] (319 µs)
• test_benchmark_scripts[query_path8] (325.1 µs)
• test_benchmark_scripts[query_path9] (319.2 µs)

[FBruzzesi]

Ok, so:

• even with 10% of data, it takes 30 mins to run
• in the codspeed website I can see that these benchmarks come with a warning:

Warning

This benchmark contains 32 system calls, totalling 39.1 s of execution time. Since they cannot be consistently instrumented, those calls are not included in the measure. Please switch to the Walltime instrument to accurately measure system calls. Learn more about measurement and system calls.

which to me indicates that the numbers in the report here are not tracking what we would like to see.

Additionally, we don't get the split by backend, which is also something I would like to see if we integrate performance tooling.
To do that we would need to have the benchmark on a lower level such as in execute_query

[dangotbanned]

@FBruzzesi (#805 (comment))

In #972 we were using TPCH with 0.25 ratio and it was taking ~40mins to run IIRC. That's a bit much for what I would consider fast iteration - maybe a ratio of 0.1 is more reasonable to start with

IIRC the docs for the duckdb TPCH tests used 0.01 - so we can go lower

I found the bit in the docs that used 0.01 (https://duckdb.org/docs/0.10/extensions/tpch#listing-expected-answers)

To produced the expected results for all queries on scale factors 0.01, 0.1, and 1, run

If we can run these with 10x less data, surely we should right?

The current run has been going for almost 2 hours 😅
(https://github.com/narwhals-dev/narwhals/actions/runs/15098359607/job/42436026213?pr=2516)

[FBruzzesi]

The current run has been going for almost 2 hours 😅

Yes I have been monitoring it - it's a bit odd, isn't it? I am not fully sure what's going on 🤔