Add grid-power traces and metric glossary to eval PR comments by tomquist · Pull Request #468 · tomquist/AstraMeter

tomquist · 2026-06-12T21:53:24Z

Enhances the steering evaluation PR comment with visual and explanatory improvements to help reviewers understand control performance.

Summary

The evaluation comparison now includes:

A collapsible metric glossary explaining what each reported metric means
Mermaid xychart charts showing grid-power traces over time for each scenario, with base and head overlaid for easy visual comparison

Key changes

Added GRAPH_POINTS constant (80 points) to downsample grid-power traces to a fixed length, ensuring base and head traces align by index regardless of poll cadence
Implemented _downsample_grid() function to bucket samples into evenly spaced mean values, carrying forward empty buckets to avoid chart gaps
Added grid_trace field to scenario results containing the downsampled grid-power values
Created _METRIC_GLOSSARY list pairing each metric key with a human-readable description, kept in sync with _REPORT_METRICS
Implemented _grid_chart() function to render Mermaid xychart visualizations, gracefully handling missing traces from older JSON artifacts
Updated render_markdown_compare() to include the collapsible glossary and grid-power charts in the PR comment

Implementation details

The downsampling algorithm uses fixed-size buckets to ensure consistent trace lengths across runs with different poll cadences, enabling direct index-based overlay in charts
Empty buckets forward the previous value to maintain continuity in the visualization
The metric glossary is rendered as a collapsible <details> section with a markdown table for clarity
Grid charts render only when trace data is available, maintaining backward compatibility with older evaluation results
All metrics are rounded appropriately for readability in the markdown output

https://claude.ai/code/session_01DSRYD8MSs85X4HGnWAwaUZ

Summary by CodeRabbit

New Features
- Reports now embed per-scenario grid-power comparison charts (baseline vs modified) for visual trend comparison.
- Metrics now include a fixed-length downsampled grid trace so overlays align consistently.
Documentation
- CLI compare docs mention the new grid-power trace visualization and include a collapsible metric glossary explaining reported metrics.
Tests
- Expanded tests covering glossary contents, chart rendering, and fixed-length grid-trace generation.

Export a downsampled grid-power trace from each scenario run and render it as a Mermaid xychart (base vs head overlaid) inside the scenario's details block in the CI comment.

coderabbitai · 2026-06-12T21:53:35Z

Warning

Review limit reached

@tomquist, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 17 minutes and 17 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9135bc34-afa7-437f-9d54-3a6876f8878d

📥 Commits

Reviewing files that changed from the base of the PR and between 2db5d79 and f4dfce8.

📒 Files selected for processing (1)

src/astrameter/simulator/evaluation.py

Walkthrough

The PR extends the evaluation harness to add visualizations and context for CI metric comparisons. Grid power traces are downsampled to a fixed length for overlay alignment, metrics gain descriptive glossary entries in the CI report, and comparative grid-power charts are rendered for each scenario using Mermaid xychart.

Changes

Grid trace visualization and metric glossary for CI reporting

Layer / File(s)	Summary
Grid trace downsampling `src/astrameter/simulator/evaluation.py`	`GRAPH_POINTS` constant and `_downsample_grid()` bucket sampled grid values into fixed-length, evenly spaced series with forward-fill. Metrics dict extended with `grid_trace` field.
Test updates and imports `tests/test_steering_eval.py`	Tests import `_METRIC_GLOSSARY`, `_REPORT_METRICS`, and `GRAPH_POINTS`. Added assertions verifying `grid_trace` length/types and strengthened markdown checks for glossary and embedded Mermaid chart structure.
Metric glossary and CI documentation `src/astrameter/simulator/evaluation.py`	`_METRIC_GLOSSARY` list maps each metric to descriptions; `render_markdown_compare()` inserts a collapsible glossary section populated from that list.
Grid power visualization in reports `src/astrameter/simulator/evaluation.py`	`_grid_chart()` renders a Mermaid `xychart` showing per-scenario grid-power traces and optionally overlays base vs head. Integrated into scenario details rendering when head trace is present.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

tomquist/AstraMeter#466: Earlier PR that extended steering-quality evaluation/CI reporting; this PR builds on that foundation by adding grid-trace downsampling, metric glossary, and comparative visualization.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately describes the main changes: adding grid-power traces and metric glossary to evaluation PR comments, which aligns perfectly with the implementation of Mermaid charts, downsampling logic, and glossary rendering.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/dreamy-dirac-06udz4

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-12T21:54:57Z

Steering evaluation (base vs head)

Lower is better for every metric. See src/astrameter/simulator/evaluation.py for definitions.

What do these metrics mean?

Metric	Meaning
`settle_mean_s`	Mean seconds after a load/PV step for grid power to return inside the ±25 W settle band and hold for 10 s (reaction speed).
`settle_p95_s`	95th-percentile settle time — the slow tail of reactions.
`unsettled_events`	Number of disturbance events that never settled within the 10-minute measurement window.
`overshoot_mean_w`	Mean overshoot (W): how far grid power swings past zero to the opposite sign after an event.
`overshoot_max_w`	Worst-case overshoot (W) across all events.
`band_crossings_per_h`	Sign flips per hour across the ±20 W hysteresis band — oscillation / hunting.
`steady_rms_w`	RMS grid power (W) during steady state (excluding the 120 s after each event) — residual jitter when nothing is changing.
`mean_abs_grid_w`	Mean absolute grid power (W) over the whole run — overall tracking accuracy.
`avoidable_import_wh`	Energy imported from the grid (Wh) the battery could have supplied (it had charge and discharge headroom) — missed self-consumption.
`avoidable_export_wh`	Energy exported to the grid (Wh) an AC-chargeable battery could have absorbed (it had room and charge headroom) — missed charging.
`battery_travel_w_per_h`	Total absolute change in battery setpoints per hour (W/h) — control effort / actuator wear; lower is smoother.

mixed_cadence/eff — settle 105.7→105.7s, overshoot 1949.5→1949.5W, RMS 111.7→111.7W

Metric	Base	Head	Δ
settle_mean_s	105.7	105.7	=
settle_p95_s	241.7	241.7	=
unsettled_events	1	1	=
overshoot_mean_w	766.7	766.7	=
overshoot_max_w	1949.5	1949.5	=
band_crossings_per_h	446.0	446.0	=
steady_rms_w	111.7	111.7	=
mean_abs_grid_w	100.0	100.0	=
avoidable_import_wh	75.2	75.2	=
avoidable_export_wh	25.1	25.1	=
battery_travel_w_per_h	180519.0	180519.0	=