Commit 907c1b6
Epic 6.2: SLOs - Error Budgets, Burn Rates, Multi-Window Alerting (#51)
* feat: Epic 6.2 - SLO module with error budgets, burn rates, and MCP tool
- SLOTarget dataclass with validation, ErrorBudget with float-tolerant exhaustion
- BurnRateCalculator with Google SRE multi-window alerting model
- SLOEvaluator reads from MetricsRegistry via Prometheus .collect() API
- 3 default targets: latency_p99 <30s, error_rate <5%, availability >99%
- check_slos MCP tool (8 total tools)
- 35+ new tests (1,858 total passed)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address all review findings - 5 agents, 12 fixes, 22 new tests
Security:
- Wrap check_slos in try/except + handle_mcp_exception (error oracle)
- Clamp burn_rate to MAX_BURN_RATE=1000.0 (no float('inf') in JSON)
- Add SLOTarget validation: non-empty name, finite non-negative threshold
- Filter _get_request_counts to _total suffix (PROMETHEUS_ENABLE_CREATED_SERIES safety)
Correctness:
- Fix objective=1.0 contradictory state (exhausted + 100% remaining)
- Fix p99 multi-label aggregation (sum by le across agent × status)
- Add p99 linear interpolation between bucket boundaries
- Fix p99 overflow: return highest bucket boundary instead of None
- Correct docstring: simplified burn-rate ratio, not Google multi-window
Hardening:
- DEFAULT_TARGETS immutable tuple (was mutable list)
- SLOTarget frozen=True dataclass
- ErrorBudget zero-budget path consistent status
Tests: 22 new adversarial tests (1,884 total passed)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: R2 hardening - NaN-safe JSON, isfinite guard, coupling docs
- json.dumps(allow_nan=False) prevents NaN/Infinity in JSON output
- math.isfinite() guard on p99 interpolation result
- ValueError/OverflowError catch in _get_request_counts
- Cross-reference comment for status label coupling (execution.py)
R2 review: 2 APPROVE (sec, opus), 2 REQUEST_CHANGES on documented
design limitations (in-process counters, hardcoded dispatch) - v2 items.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Brian Krafft <bkrafft@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 9a6acd2 commit 907c1b6
9 files changed
Lines changed: 1156 additions & 16 deletions
File tree
- openspace
- mcp
- observability
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
5 | | - | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
547 | 547 | | |
548 | 548 | | |
549 | 549 | | |
550 | | - | |
551 | | - | |
| 550 | + | |
| 551 | + | |
552 | 552 | | |
553 | 553 | | |
554 | 554 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
829 | 829 | | |
830 | 830 | | |
831 | 831 | | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
832 | 851 | | |
833 | 852 | | |
834 | 853 | | |
| |||
845 | 864 | | |
846 | 865 | | |
847 | 866 | | |
| 867 | + | |
| 868 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
| 5 | + | |
5 | 6 | | |
6 | | - | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| 12 | + | |
11 | 13 | | |
12 | 14 | | |
13 | 15 | | |
| |||
16 | 18 | | |
17 | 19 | | |
18 | 20 | | |
| 21 | + | |
19 | 22 | | |
0 commit comments