community(benchmarks): add component benchmark results from Intel i3-6006U Linux#2065
Open
Om-Rohilla wants to merge 2 commits into
Open
community(benchmarks): add component benchmark results from Intel i3-6006U Linux#2065Om-Rohilla wants to merge 2 commits into
Om-Rohilla wants to merge 2 commits into
Conversation
…6006U Ran all four component benchmark scripts on my local Linux machine (Intel i3-6006U, 4 cores, 3.7 GB RAM, Python 3.14.5) and recorded the output. Added a Community-submitted benchmarks section to docs/benchmarks/BENCHMARKS.md with actual measured numbers. Contributes to sipyourdrink-ltd#787.
Reviewer's GuideAdds a new community-submitted component benchmark section for an Intel i3-6006U Linux setup to BENCHMARKS.md, documenting hardware details and detailed benchmark results in a structured table following the existing benchmarks documentation style. Flow diagram for community-submitted benchmark contribution processflowchart LR
actor_user["Community contributor"]
run_scripts["Run benchmark scripts
(bench_orchestrator.py,
bench_task_store.py,
bench_quality_gates.py,
bench_startup.py)"]
record_results["Record hardware details
and benchmark results"]
update_section["Add row to
Community-submitted benchmarks
in BENCHMARKS.md"]
submit_pr["Open PR or comment
on issue 787"]
actor_user --> run_scripts --> record_results --> update_section --> submit_pr
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a “Community-submitted benchmarks” section to document real-world benchmark results and provide a contribution path for others to submit their own data.
Changes:
- Introduced a new documentation section for community benchmark submissions
- Added a concrete benchmark dataset (hardware/software context + results table)
- Included a submission workflow link (issue #787)
Comment on lines
+146
to
+148
| | Benchmark | Result | Command | | ||
| |-----------|--------|---------| | ||
| | Orchestrator tick latency (100-task backlog) — avg | 5.89 ms | `bench_orchestrator.py` | |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a "Community-submitted benchmarks" section to docs/benchmarks/BENCHMARKS.md (lines 132–163) that records a community real-run entry with hardware/software metadata, measured orchestrator/task-store/quality-gate/startup timings, and a short notes paragraph. ChangesBenchmark Documentation
Estimated code review effort: 🎯 1 (Trivial) | ⏱️ ~2 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In the hardware metadata block under the new section, add blank lines between the bolded lines (Hardware / Bernstein version / Python / Submitted by) so they render as separate paragraphs rather than a single wrapped line in some Markdown renderers.
- For the
Commandcolumn, consider showing the full invocation (e.g.,uv run python benchmarks/bench_orchestrator.py) to match the earlier sections and make it directly copy-pastable for readers. - Since Python 3.14 is not yet generally available, it might help to briefly note that this was run on a pre-release or dev build, or alternatively re-run and document with the latest stable Python to avoid confusion for readers comparing results.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In the hardware metadata block under the new section, add blank lines between the bolded lines (Hardware / Bernstein version / Python / Submitted by) so they render as separate paragraphs rather than a single wrapped line in some Markdown renderers.
- For the `Command` column, consider showing the full invocation (e.g., `uv run python benchmarks/bench_orchestrator.py`) to match the earlier sections and make it directly copy-pastable for readers.
- Since Python 3.14 is not yet generally available, it might help to briefly note that this was run on a pre-release or dev build, or alternatively re-run and document with the latest stable Python to avoid confusion for readers comparing results.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/benchmarks/BENCHMARKS.md`:
- Around line 141-143: The benchmark metadata block contains
incorrect/fabricated versions (the "Hardware:", "Bernstein version:", and
"Python:" entries) — specifically "Ubuntu (kernel 6.17.0)" and "Python: 3.14.5"
are invalid and duplicate the Python entry on line 139; update those entries to
the actual kernel and Python versions used when running the benchmark (or remove
them until you can provide accurate values), remove the duplicate Python line,
and ensure any URLs referenced for kernel or Python releases point to valid
upstream pages (verify links resolve) so the BENCHMARKS.md metadata is truthful
and self-consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1aa1467a-8073-4006-85fd-d4dc78f2dc33
📒 Files selected for processing (1)
docs/benchmarks/BENCHMARKS.md
- Add blank lines between metadata fields so they render as separate paragraphs (Sourcery) - Expand command column to full uv invocations, e.g. 'uv run python benchmarks/bench_orchestrator.py' (Sourcery/Copilot) - Clarify Python 3.14.5 is a pre-release build; add note that stable 3.12/3.13 should give lower startup latency (Sourcery/CodeRabbit) - Include full distro kernel string: 6.17.0-35-generic (CodeRabbit)
Author
|
Thanks for the reviews — addressed all the feedback in the follow-up commit (28d9a77):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds a community-submitted component benchmark section to
docs/benchmarks/BENCHMARKS.mdwith real numbers from my machine.I ran all four benchmark scripts (
bench_orchestrator.py,bench_task_store.py,bench_quality_gates.py,bench_startup.py) on my local Linux setup and recorded the output. Added the results under a new "Community-submitted benchmarks" section as suggested in issue #787.Contributes to #787.
Hardware
Results summary
Startup is a bit slow — probably cold import overhead with Python 3.14. Everything else looks reasonable for this class of hardware.
Checklist
Summary by Sourcery
Documentation:
Summary by CodeRabbit