Skip to content

Test benchmarking#8

Closed
kaitj wants to merge 6 commits into
mainfrom
ci/benchmark
Closed

Test benchmarking#8
kaitj wants to merge 6 commits into
mainfrom
ci/benchmark

Conversation

@kaitj
Copy link
Copy Markdown
Owner

@kaitj kaitj commented May 6, 2026

No description provided.

kaitj added 5 commits May 6, 2026 13:44
- Mark tests with "cloud" and / or "benchmark" as needed
- Combine both "dev" and "benchmark" dependencies, was causing issues with the pytest due to imports (alternatively, use `try-except` block for optional dependency import)
- Replace pandas with polars in dev dependency (for benchmarking)
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py60100% 
__main__.py69888%101, 110, 135, 137–138, 164, 168, 172
_entities.py112199%129
_indexing.py212597%150, 159–160, 409, 447
_logging.py31487%30, 37, 39–40
_metadata.py48491%39–40, 66, 71
_pathlib.py170100% 
_version.py110100% 
TOTAL5062295% 

Tests Skipped Failures Errors Time
56 0 💤 0 ❌ 0 🔥 15.469s ⏱️

@kaitj kaitj force-pushed the ci/benchmark branch 15 times, most recently from 0dfa8d9 to 2f94d6b Compare May 6, 2026 19:18
Repository owner deleted a comment from github-actions Bot May 6, 2026
Repository owner deleted a comment from github-actions Bot May 6, 2026
@kaitj kaitj force-pushed the ci/benchmark branch 2 times, most recently from 5201e77 to 7a9bb0b Compare May 6, 2026 19:29
Repository owner deleted a comment from github-actions Bot May 6, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

Benchmark Results

Local index (1 workers) Local index (4 workers) Bold query Metadata query Subject query
PR 104.229 (119.968 ± 49.252) ms 112.730 (127.484 ± 46.721) ms 0.177 (0.182 ± 0.016) ms 0.263 (0.264 ± 0.014) ms 0.031 (0.034 ± 0.009) ms
main 147.933 (170.620 ± 40.987) ms 153.858 (165.798 ± 41.628) ms 0.249 (0.255 ± 0.023) ms 0.351 (0.365 ± 0.035) ms 0.056 (0.062 ± 0.012) ms
PR vs main ⚪ -43.705ms ⚪ -41.127ms ⚪ -0.072ms ⚪ -0.088ms ⚪ -0.026ms

median (mean ± std)

🔴 >250ms slower   ⚪ within 250ms   🟢 >250ms faster

Repository owner deleted a comment from github-actions Bot May 6, 2026
- Switch to shortened SHA for PR
- Add PR for unique output file artifact
- Disable comparison against tag due to lack of dependency group
- Add step to comment on PR
- Sort labels for comment
@kaitj kaitj closed this May 6, 2026
@kaitj kaitj deleted the ci/benchmark branch May 20, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant