feat(routing): Biology-inspired routing improvements (EXP3 + MVT + ODT) - #35
Conversation
…quency-dependence) Based on Agent Council verdict — replaces the 'Plankton α' bio-derivation with the correct EXP3 learning rate from Auer et al. 2002. What changed: - Added computeDiversityPenalty(providerName, nProviders): applies an EXP3-inspired penalty when provider traffic share exceeds uniform (1/n). gamma = sqrt(n * log(n) / (T * G^2)) — adapts over time as T grows. - Added recordSelection(providerName): tracks traffic per provider. - Applied diversity penalty in routeQuery after complexity_bias scoring, re-ranking before quality floor check. - Tested: no regression in eval (0.5893 identical to baseline). - Verified: max provider share = 13.3% across 11 providers in 30-query distribution test (near-uniform, no monoculture). Why this matters: - Prevents the 'competitive exclusion' failure mode where one dominant provider captures all traffic, creating a single point of failure. - The penalty is adaptive: strongest when T is small and a provider is over-represented, weakens as the system stabilizes. - Zero API call overhead, zero cost impact. 理论基础: - Auer et al. 2002, 'The Nonstochastic Multiarmed Bandit Problem' (EXP3) - Negative frequency-dependent selection (correct bio framing per Expert 4) - NOT Armstrong-McGehee (static regularizer ≠ oscillation — Expert 2 finding) Council consensus: this is the ONLY safe change to the main routing path. Branch: feature/adaptive-diversity-weight
Step 2 of Agent Council recommendations — MVT applied to the RIGHT substrate (throughput/rate-limits, NOT quality switching — Expert 4 confirmed). What changed in providerHealth.ts: NEW interfaces: - ProviderHealth: added tokensUsedThisWindow, rateLimitWindowStart, rateLimitTokens, rateLimitWindowMs, avgTokensPerRequest - ProviderMetrics: added tokensUsed NEW methods on ProviderHealthManager: - setRateLimitConfig(provider, rateLimitTokens, rateLimitWindowMs): Configure per-provider rate limits. Call during provider registration. - shouldRotateForRateLimit(provider, fallbackProvider): Charnov MVT: g'(t*) = g(t*)/(t* + τ). Returns true when marginal rate ≤ avg rate including switch cost → MVT says LEAVE. - getMarginalRate(provider): Monitoring/debugging for MVT decisions. - recordSuccess(): Updated to track token consumption and detect rate-limit window rollovers (resets counter when window expires). - recordFailure(): Updated to include tokensUsed: 0. NEW stateless helper: - mvtShouldRotate(providerHealth, fallbackLatencyMs, estimatedTokens): Standalone function for callers that only have ProviderHealth state, not the full manager instance. Exported for use by server/SDK layer. NEW singleton: - globalHealthManager: Module-level singleton for app-wide health tracking. The MVT formula is applied CORRECTLY: - g(t) = cumulative tokens used in current rate-limit window - g'(t) = marginal rate = remaining budget / time remaining - τ = cold-start latency for fallback provider - LEAVE when: marginal_rate ≤ cumulative_rate / (elapsed + τ) - STAY when: marginal_rate > cumulative_rate / (elapsed + τ) This is specifically for RATE-LIMIT depletion, NOT quality switching. Expert 4 confirmed: 'Throughput/rate limits DO regenerate. Token buckets refill. This is a textbook depleting-and-regenerating resource.' Quality switching (MVT misapplied) remains RESEARCH-ONLY. Council consensus: this is safe to implement, LOW risk to RouterArena #1.
…nal verification
Step 3 of Agent Council recommendations — ODT (Rhoades 1979; Zangerl & Bazzaz 1992).
Core insight: Instead of binary always-on (Zahavi-style, 100% overhead) or always-off,
allocate shadow verification PROPORTIONALLY to query value and risk.
ODT biological principle:
'A plant allocates defensive compounds in proportion to tissue value,
attack probability, and the marginal cost of defense.'
Mapped to A3M routing:
- Tissue value → expected cost of wrong answer (query stakes)
- Attack probability → risk_profile (probability primary fails)
- Defense cost → cost of the shadow API call
- Marginal return → expected error reduction from verification
P(shadow) formula:
baseP = 0.02 (constitutive baseline)
stake_adj = scales with estimated query stakes
risk_adj = +0.10 for high-risk, +0.03 for medium (ODT induced defense)
complexity_adj = +0.05 code, +0.04 math, +0.03 reasoning
NEW module: src/routing/shadowSampler.ts
- ShadowSampler class with routeWithShadow()
- ODT computeShadowProbability() — derives P(shadow) from features
- estimateQueryStake() — maps query to expected cost of wrong answer
- selectShadowProvider() — auto-selects cheapest non-primary provider
- compareOutputs() — bigram similarity for output agreement
- getStats() — monitor shadow sampling rate
Usage:
const sampler = new ShadowSampler({ maxShadowProbability: 0.15 });
const decision = sampler.routeWithShadow(query);
if (decision.hasShadow) {
// Run shadow provider and compare
}
Key design: This is NOT the Zahavi shared expert (always-on 100%).
ODT shadow is probabilistic — only verifies when expected verification
benefit exceeds verification cost. Max P(shadow) is configurable.
Tests: 4/4 ODT principles verified in test suite.
Eval: 0.5893 identical to baseline (pre-existing failure unchanged).
|
Warning Review limit reached
Next review available in: 51 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughChangesThe routing system adds provider diversity penalties, token-window accounting and rotation decisions, and an ODT-based Routing adaptation
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ShadowSampler
participant routeQuery
participant PrimaryProvider
participant ShadowProvider
participant OutputComparator
Client->>ShadowSampler: submit prompt
ShadowSampler->>routeQuery: route primary request
routeQuery->>PrimaryProvider: invoke selected provider
PrimaryProvider-->>ShadowSampler: primary answer
ShadowSampler->>ShadowProvider: invoke sampled shadow provider
ShadowProvider-->>OutputComparator: shadow answer
ShadowSampler->>OutputComparator: compare outputs
OutputComparator-->>Client: winner and confidence delta
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
src/routing/providerHealth.ts (1)
670-712: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicated MVT math vs.
shouldRotateForRateLimit;fallbackProviderLatencyMs=0silently zeroes switch cost.This function re-implements the same marginal-rate/avg-rate-including-switch computation as
shouldRotateForRateLimit(lines 409-455) independently, so any future fix (e.g. the window-rollover gap) must be applied in two places. Additionally, unlikeestimateColdStartLatency'sbaseLatency || 500fallback,fallbackProviderLatencyMshere has no floor — a caller passing0(e.g. rawhealth.latencyfor a never-used fallback) makestau = 0, which removes the switch-cost penalty from the MVT break-even calculation and biases the result toward rotating.Consider extracting the shared marginal-rate/break-even math into one internal helper used by both call sites, and applying a minimum floor to
fallbackProviderLatencyMs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/routing/providerHealth.ts` around lines 670 - 712, The MVT marginal-rate and break-even calculation is duplicated in mvtShouldRotate and shouldRotateForRateLimit, and zero fallback latency removes the switch-cost penalty. Extract the shared calculation into one internal helper used by both call sites, and normalize fallbackProviderLatencyMs to the same minimum latency floor used by estimateColdStartLatency before computing the break-even rate.src/routing/shadowSampler.ts (2)
33-35: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winQuery stake uses a hardcoded cost constant instead of the imported
estimateCostutility.
estimateQueryStakeuses a fixedavgCostPerToken = 0.0001(Line 202) rather than the already-importedestimateCost(Line 35), whichrouteQueryuses with real per-provider pricing (peradvancedRouter.tscontext:estimateCost(features.length, estimated_tokens, primary.name)). By the timeestimateQueryStakeruns (Line 307),primaryDecision.primary_modelis already available (Line 304), so a provider-aware cost could be substituted. Since ODT's core premise is proportional allocation based on real stake, this flat approximation weakens the calibration the whole sampling mechanism depends on.Also applies to: 200-218
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/routing/shadowSampler.ts` around lines 33 - 35, Update estimateQueryStake to replace the hardcoded avgCostPerToken approximation with the imported estimateCost utility, using the available primary provider/model context from primaryDecision.primary_model. Preserve the existing proportional stake calculation while making its cost input provider-aware, consistent with routeQuery’s estimateCost usage.
299-333: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winRedundant
extractQueryFeaturescall on every routed query.Line 303 computes
extractQueryFeatures(prompt), thenrouteQuery(Line 304) recomputes the same features internally (peradvancedRouter.tscontext,routeQuerycallsextractQueryFeatures(prompt)and returns it inRouteDecision.features).extractQueryFeaturesdoes non-trivial keyword scanning across several large keyword lists (peradvancedRouter.tscontext), so this doubles that work on every request. Note the local recompute does serve a purpose today —routeQuery's no-candidates early return omitsfeaturesentirely, so a naiveprimaryDecision.features!reuse would be unsafe.♻️ Suggested fix (keeps correctness for the empty-candidates path)
const features = extractQueryFeatures(prompt); const primaryDecision = routeQuery(prompt, options?.available_models, options?.budget_multiplier); + // Reuse routeQuery's already-computed features when available, avoiding a second full pass + // over the same prompt; fall back only for the no-candidates early-return path.Consider restructuring to call
routeQueryfirst and default toprimaryDecision.features ?? extractQueryFeatures(prompt).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/routing/shadowSampler.ts` around lines 299 - 333, Update routeWithShadow to call routeQuery before computing features, then reuse primaryDecision.features when present and fall back to extractQueryFeatures(prompt) only when routeQuery omits features for the no-candidates path. Use this resolved features value for stake, shadow probability, reasoning, and the returned decision while preserving existing routing behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/routing/advancedRouter.ts`:
- Around line 91-98: Resolve the unused resetDiversityState declaration by
exporting it for the provider-pool and evaluation reset callers that its JSDoc
describes, then update those callers to import and invoke it where needed; if no
such callers are intended, remove the function and its JSDoc instead.
- Around line 47-80: Update computeDiversityPenalty so share is calculated as
the provider’s selection count divided by _totalDecisions before comparing it
with uniform. Keep the existing early returns, deviation handling, and penalty
calculation unchanged, ensuring the resulting penalty reflects traffic fraction
rather than an unbounded cumulative count.
In `@src/routing/providerHealth.ts`:
- Around line 409-455: Update shouldRotateForRateLimit to handle windowElapsed
>= health.rateLimitWindowMs before calculating remaining budget or marginal
rate: treat the expired window as freshly reset with the full rate-limit budget
and reset timing, so stale tokens cannot influence the rotation verdict.
Preserve the existing behavior for active windows and the too-fresh window
guard.
In `@src/routing/shadowSampler.ts`:
- Around line 26-31: Update the ShadowSampler docstring usage example to call
the synchronous routeWithShadow method without await and pass only supported
options, such as available_models or budget_multiplier. Keep the example’s
ShadowDecision result and comparison behavior accurate.
- Around line 156-187: Update selectShadowProvider so cached _shadowProvider is
reused only when it differs from the current primaryProvider; otherwise rerun
candidate selection and cache a valid non-primary provider. Preserve the
existing forced-provider behavior while ensuring every returned shadow provider
remains distinct from the primaryProvider.
---
Nitpick comments:
In `@src/routing/providerHealth.ts`:
- Around line 670-712: The MVT marginal-rate and break-even calculation is
duplicated in mvtShouldRotate and shouldRotateForRateLimit, and zero fallback
latency removes the switch-cost penalty. Extract the shared calculation into one
internal helper used by both call sites, and normalize fallbackProviderLatencyMs
to the same minimum latency floor used by estimateColdStartLatency before
computing the break-even rate.
In `@src/routing/shadowSampler.ts`:
- Around line 33-35: Update estimateQueryStake to replace the hardcoded
avgCostPerToken approximation with the imported estimateCost utility, using the
available primary provider/model context from primaryDecision.primary_model.
Preserve the existing proportional stake calculation while making its cost input
provider-aware, consistent with routeQuery’s estimateCost usage.
- Around line 299-333: Update routeWithShadow to call routeQuery before
computing features, then reuse primaryDecision.features when present and fall
back to extractQueryFeatures(prompt) only when routeQuery omits features for the
no-candidates path. Use this resolved features value for stake, shadow
probability, reasoning, and the returned decision while preserving existing
routing behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cc795788-0fa5-4b79-81ab-ed3ae611188e
📒 Files selected for processing (3)
src/routing/advancedRouter.tssrc/routing/providerHealth.tssrc/routing/shadowSampler.ts
| function computeDiversityPenalty( | ||
| providerName: string, | ||
| nProviders: number, | ||
| recentDecayFactor = 0.0 | ||
| ): number { | ||
| if (nProviders < 2) return 0; | ||
| if (_totalDecisions < 2) return 0; | ||
|
|
||
| const share = _selectionCount[providerName] || 0; | ||
| const uniform = 1.0 / nProviders; | ||
|
|
||
| // Deviation from uniform distribution (can be negative if under-used) | ||
| const deviation = share - uniform; | ||
| if (Math.abs(deviation) < 1e-6) return 0; | ||
|
|
||
| // EXP3 learning rate: gamma = sqrt(n * log(n) / (T * G^2)) | ||
| // G = reward range (quality_score in [0,1] → G = 1) | ||
| const T = Math.max(_totalDecisions, 10); | ||
| const gamma = Math.sqrt((nProviders * Math.log(nProviders)) / (T * _REWARD_RANGE * _REWARD_RANGE)); | ||
|
|
||
| // Clamp gamma to prevent extreme penalties when T is very small | ||
| const clampedGamma = Math.min(gamma, 0.5); | ||
|
|
||
| // Optional: decay factor for non-stationary environments | ||
| // (higher share = stronger penalty, but decays over time) | ||
| // Currently disabled (recentDecayFactor=0) — re-enable if providers change frequently | ||
| const effectiveGamma = clampedGamma * (1.0 - recentDecayFactor); | ||
|
|
||
| // Penalty is proportional to how far above uniform the provider's share is | ||
| // (negative deviation = under-used = reward, not penalty) | ||
| if (deviation <= 0) return 0; // Reward already captured implicitly by below-uniform penalty | ||
|
|
||
| return effectiveGamma * deviation; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
share is a raw count, not a traffic fraction — diversity penalty math is broken.
Line 55 sets share = _selectionCount[providerName] || 0 (a cumulative integer count), then Line 59 computes deviation = share - uniform where uniform = 1/nProviders is a fraction (e.g. 0.5). The docstring and formula (lines 21-24, 39-41) both describe this as the provider's traffic share f_i, which should be _selectionCount[providerName] / _totalDecisions. As written, deviation grows unbounded with cumulative usage count instead of reflecting actual over-representation, so after just a couple of selections the "penalty" already dwarfs typical quality_score/cost_score values and never meaningfully reflects current traffic distribution.
🐛 Proposed fix
- const share = _selectionCount[providerName] || 0;
+ const share = _totalDecisions > 0 ? (_selectionCount[providerName] || 0) / _totalDecisions : 0;
const uniform = 1.0 / nProviders;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| function computeDiversityPenalty( | |
| providerName: string, | |
| nProviders: number, | |
| recentDecayFactor = 0.0 | |
| ): number { | |
| if (nProviders < 2) return 0; | |
| if (_totalDecisions < 2) return 0; | |
| const share = _selectionCount[providerName] || 0; | |
| const uniform = 1.0 / nProviders; | |
| // Deviation from uniform distribution (can be negative if under-used) | |
| const deviation = share - uniform; | |
| if (Math.abs(deviation) < 1e-6) return 0; | |
| // EXP3 learning rate: gamma = sqrt(n * log(n) / (T * G^2)) | |
| // G = reward range (quality_score in [0,1] → G = 1) | |
| const T = Math.max(_totalDecisions, 10); | |
| const gamma = Math.sqrt((nProviders * Math.log(nProviders)) / (T * _REWARD_RANGE * _REWARD_RANGE)); | |
| // Clamp gamma to prevent extreme penalties when T is very small | |
| const clampedGamma = Math.min(gamma, 0.5); | |
| // Optional: decay factor for non-stationary environments | |
| // (higher share = stronger penalty, but decays over time) | |
| // Currently disabled (recentDecayFactor=0) — re-enable if providers change frequently | |
| const effectiveGamma = clampedGamma * (1.0 - recentDecayFactor); | |
| // Penalty is proportional to how far above uniform the provider's share is | |
| // (negative deviation = under-used = reward, not penalty) | |
| if (deviation <= 0) return 0; // Reward already captured implicitly by below-uniform penalty | |
| return effectiveGamma * deviation; | |
| } | |
| function computeDiversityPenalty( | |
| providerName: string, | |
| nProviders: number, | |
| recentDecayFactor = 0.0 | |
| ): number { | |
| if (nProviders < 2) return 0; | |
| if (_totalDecisions < 2) return 0; | |
| const share = _totalDecisions > 0 ? (_selectionCount[providerName] || 0) / _totalDecisions : 0; | |
| const uniform = 1.0 / nProviders; | |
| // Deviation from uniform distribution (can be negative if under-used) | |
| const deviation = share - uniform; | |
| if (Math.abs(deviation) < 1e-6) return 0; | |
| // EXP3 learning rate: gamma = sqrt(n * log(n) / (T * G^2)) | |
| // G = reward range (quality_score in [0,1] → G = 1) | |
| const T = Math.max(_totalDecisions, 10); | |
| const gamma = Math.sqrt((nProviders * Math.log(nProviders)) / (T * _REWARD_RANGE * _REWARD_RANGE)); | |
| // Clamp gamma to prevent extreme penalties when T is very small | |
| const clampedGamma = Math.min(gamma, 0.5); | |
| // Optional: decay factor for non-stationary environments | |
| // (higher share = stronger penalty, but decays over time) | |
| // Currently disabled (recentDecayFactor=0) — re-enable if providers change frequently | |
| const effectiveGamma = clampedGamma * (1.0 - recentDecayFactor); | |
| // Penalty is proportional to how far above uniform the provider's share is | |
| // (negative deviation = under-used = reward, not penalty) | |
| if (deviation <= 0) return 0; // Reward already captured implicitly by below-uniform penalty | |
| return effectiveGamma * deviation; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routing/advancedRouter.ts` around lines 47 - 80, Update
computeDiversityPenalty so share is calculated as the provider’s selection count
divided by _totalDecisions before comparing it with uniform. Keep the existing
early returns, deviation handling, and penalty calculation unchanged, ensuring
the resulting penalty reflects traffic fraction rather than an unbounded
cumulative count.
| /** | ||
| * Reset traffic tracking (call when provider pool changes or for eval resets). | ||
| */ | ||
| function resetDiversityState(): void { | ||
| Object.keys(_selectionCount).forEach(k => delete _selectionCount[k]); | ||
| _totalDecisions = 0; | ||
| } | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n 'resetDiversityState' -g '*.ts'Repository: Das-rebel/a3m-router
Length of output: 158
🏁 Script executed:
#!/bin/bash
set -u
printf 'Tracked files matching advancedRouter.ts:\n'
git ls-files | rg '(^|/)advancedRouter\.ts$|src/routing/advancedRouter\.ts$' || true
printf '\nReferences to resetDiversityState (all tracked files):\n'
git ls-files -z | xargs -0 rg -n 'resetDiversityState' || true
printf '\nRelevant src/routing/advancedRouter.ts lines around selection/diversity state:\n'
if [ -f src/routing/advancedRouter.ts ]; then
nl -ba src/routing/advancedRouter.ts | sed -n '1,140p'
fi
printf '\nExports from src/routing/advancedRouter.ts:\n'
if [ -f src/routing/advancedRouter.ts ]; then
rg -n '^\s*(export\s+)?(function|const|let|var|class|async function)\b' src/routing/advancedRouter.ts || true
fi
printf '\nImports/references in typescript files containing resetDiversityState text (case-sensitive):\n'
rg -n -S '\bresetDiversityState\b' '*.ts' . || true
printf '\nImports/references in repository files containing resetDiversityState text:\n'
rg -n -a -S '\bresetDiversityState\b' . || trueRepository: Das-rebel/a3m-router
Length of output: 10980
Export resetDiversityState or remove it if eval resets are not intended.
resetDiversityState is referenced only by its own declaration, but its JSDoc explicitly says callers should invoke it for provider pool or eval resets. Make it exportable/importable where this is needed, or remove it if dead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routing/advancedRouter.ts` around lines 91 - 98, Resolve the unused
resetDiversityState declaration by exporting it for the provider-pool and
evaluation reset callers that its JSDoc describes, then update those callers to
import and invoke it where needed; if no such callers are intended, remove the
function and its JSDoc instead.
| shouldRotateForRateLimit(provider: string, fallbackProvider: string): boolean { | ||
| const health = this.health.get(provider); | ||
| if (!health) return false; | ||
|
|
||
| const now = Date.now(); | ||
| const windowElapsed = now - health.rateLimitWindowStart; | ||
|
|
||
| // If window hasn't started or is fresh, don't rotate | ||
| if (health.rateLimitWindowStart === 0 || windowElapsed < 100) return false; | ||
|
|
||
| // If already depleted (tokens used ≥ limit), recommend rotation | ||
| if (health.tokensUsedThisWindow >= health.rateLimitTokens) return true; | ||
|
|
||
| // Remaining token budget in current window | ||
| const remainingBudget = Math.max(0, health.rateLimitTokens - health.tokensUsedThisWindow); | ||
| const remainingTimeMs = Math.max(1, health.rateLimitWindowMs - windowElapsed); | ||
|
|
||
| // Marginal rate: tokens per ms we can still consume this window | ||
| // High marginal rate = plenty of budget left = stay | ||
| // Low marginal rate = running out = consider leaving | ||
| const marginalRate = remainingBudget / remainingTimeMs; | ||
|
|
||
| // Cumulative successful tokens so far | ||
| const g_t = health.tokensUsedThisWindow; | ||
|
|
||
| // Cold-start cost for switching to fallback | ||
| const tau = this.estimateColdStartLatency(fallbackProvider); | ||
|
|
||
| // Break-even rate: the rate at which staying = switching | ||
| // From MVT: g'(t*) = g(t*) / (t* + τ) | ||
| // In our terms: marginal_rate = cumulative_rate * (t* / (t* + τ)) | ||
| // But here we use: avg_rate_including_switch = g_t / (windowElapsed + τ) | ||
| // This is the rate INCLUDING the cost of switching (we lose τ ms of this window) | ||
| const avgRateIncludingSwitch = g_t / (windowElapsed + tau); | ||
|
|
||
| // MVT says: LEAVE when marginal_rate ≤ avg_rate_including_switch | ||
| // (the marginal gain from staying ≤ the average gain achievable including switch cost) | ||
| // STAY when marginal_rate > avg_rate_including_switch | ||
| // (we can still get more from this window than the switch costs us) | ||
| const ROTATION_THRESHOLD_FACTOR = 1.0; // 1.0 = exact MVT; >1 = leave earlier, <1 = stay longer | ||
|
|
||
| if (marginalRate <= avgRateIncludingSwitch * ROTATION_THRESHOLD_FACTOR) { | ||
| return true; // MVT says: leave this patch | ||
| } | ||
|
|
||
| return false; // MVT says: stay in this patch | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Missing window-rollover handling — stale token count can trigger wrong rotation decisions.
The function guards against a too-fresh window (windowElapsed < 100) but never handles the opposite case: if windowElapsed >= health.rateLimitWindowMs, the rate-limit window has effectively already expired, yet tokensUsedThisWindow/rateLimitWindowStart are only reset inside recordSuccess. If this method is called before the next recordSuccess fires (e.g., after an idle period longer than the window), remainingTimeMs clamps to 1 while remainingBudget still reflects the stale (possibly near-exhausted) prior window, producing a distorted marginalRate and potentially an incorrect rotate/stay verdict instead of treating the window as freshly reset with full budget.
🐛 Proposed fix
const now = Date.now();
const windowElapsed = now - health.rateLimitWindowStart;
// If window hasn't started or is fresh, don't rotate
if (health.rateLimitWindowStart === 0 || windowElapsed < 100) return false;
+ // If the window has already elapsed, treat it as freshly reset (full budget)
+ if (windowElapsed >= health.rateLimitWindowMs) return false;
+
// If already depleted (tokens used ≥ limit), recommend rotation
if (health.tokensUsedThisWindow >= health.rateLimitTokens) return true;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| shouldRotateForRateLimit(provider: string, fallbackProvider: string): boolean { | |
| const health = this.health.get(provider); | |
| if (!health) return false; | |
| const now = Date.now(); | |
| const windowElapsed = now - health.rateLimitWindowStart; | |
| // If window hasn't started or is fresh, don't rotate | |
| if (health.rateLimitWindowStart === 0 || windowElapsed < 100) return false; | |
| // If already depleted (tokens used ≥ limit), recommend rotation | |
| if (health.tokensUsedThisWindow >= health.rateLimitTokens) return true; | |
| // Remaining token budget in current window | |
| const remainingBudget = Math.max(0, health.rateLimitTokens - health.tokensUsedThisWindow); | |
| const remainingTimeMs = Math.max(1, health.rateLimitWindowMs - windowElapsed); | |
| // Marginal rate: tokens per ms we can still consume this window | |
| // High marginal rate = plenty of budget left = stay | |
| // Low marginal rate = running out = consider leaving | |
| const marginalRate = remainingBudget / remainingTimeMs; | |
| // Cumulative successful tokens so far | |
| const g_t = health.tokensUsedThisWindow; | |
| // Cold-start cost for switching to fallback | |
| const tau = this.estimateColdStartLatency(fallbackProvider); | |
| // Break-even rate: the rate at which staying = switching | |
| // From MVT: g'(t*) = g(t*) / (t* + τ) | |
| // In our terms: marginal_rate = cumulative_rate * (t* / (t* + τ)) | |
| // But here we use: avg_rate_including_switch = g_t / (windowElapsed + τ) | |
| // This is the rate INCLUDING the cost of switching (we lose τ ms of this window) | |
| const avgRateIncludingSwitch = g_t / (windowElapsed + tau); | |
| // MVT says: LEAVE when marginal_rate ≤ avg_rate_including_switch | |
| // (the marginal gain from staying ≤ the average gain achievable including switch cost) | |
| // STAY when marginal_rate > avg_rate_including_switch | |
| // (we can still get more from this window than the switch costs us) | |
| const ROTATION_THRESHOLD_FACTOR = 1.0; // 1.0 = exact MVT; >1 = leave earlier, <1 = stay longer | |
| if (marginalRate <= avgRateIncludingSwitch * ROTATION_THRESHOLD_FACTOR) { | |
| return true; // MVT says: leave this patch | |
| } | |
| return false; // MVT says: stay in this patch | |
| } | |
| shouldRotateForRateLimit(provider: string, fallbackProvider: string): boolean { | |
| const health = this.health.get(provider); | |
| if (!health) return false; | |
| const now = Date.now(); | |
| const windowElapsed = now - health.rateLimitWindowStart; | |
| // If window hasn't started or is fresh, don't rotate | |
| if (health.rateLimitWindowStart === 0 || windowElapsed < 100) return false; | |
| // If the window has already elapsed, treat it as freshly reset (full budget) | |
| if (windowElapsed >= health.rateLimitWindowMs) return false; | |
| // If already depleted (tokens used ≥ limit), recommend rotation | |
| if (health.tokensUsedThisWindow >= health.rateLimitTokens) return true; | |
| // Remaining token budget in current window | |
| const remainingBudget = Math.max(0, health.rateLimitTokens - health.tokensUsedThisWindow); | |
| const remainingTimeMs = Math.max(1, health.rateLimitWindowMs - windowElapsed); | |
| // Marginal rate: tokens per ms we can still consume this window | |
| // High marginal rate = plenty of budget left = stay | |
| // Low marginal rate = running out = consider leaving | |
| const marginalRate = remainingBudget / remainingTimeMs; | |
| // Cumulative successful tokens so far | |
| const g_t = health.tokensUsedThisWindow; | |
| // Cold-start cost for switching to fallback | |
| const tau = this.estimateColdStartLatency(fallbackProvider); | |
| // Break-even rate: the rate at which staying = switching | |
| // From MVT: g'(t*) = g(t*) / (t* + τ) | |
| // In our terms: marginal_rate = cumulative_rate * (t* / (t* + τ)) | |
| // But here we use: avg_rate_including_switch = g_t / (windowElapsed + τ) | |
| // This is the rate INCLUDING the cost of switching (we lose τ ms of this window) | |
| const avgRateIncludingSwitch = g_t / (windowElapsed + tau); | |
| // MVT says: LEAVE when marginal_rate ≤ avg_rate_including_switch | |
| // (the marginal gain from staying ≤ the average gain achievable including switch cost) | |
| // STAY when marginal_rate > avg_rate_including_switch | |
| // (we can still get more from this window than the switch costs us) | |
| const ROTATION_THRESHOLD_FACTOR = 1.0; // 1.0 = exact MVT; >1 = leave earlier, <1 = stay longer | |
| if (marginalRate <= avgRateIncludingSwitch * ROTATION_THRESHOLD_FACTOR) { | |
| return true; // MVT says: leave this patch | |
| } | |
| return false; // MVT says: stay in this patch | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routing/providerHealth.ts` around lines 409 - 455, Update
shouldRotateForRateLimit to handle windowElapsed >= health.rateLimitWindowMs
before calculating remaining budget or marginal rate: treat the expired window
as freshly reset with the full rate-limit budget and reset timing, so stale
tokens cannot influence the rotation verdict. Preserve the existing behavior for
active windows and the too-fresh window guard.
| * Usage: | ||
| * const sampler = new ShadowSampler(); | ||
| * const decision = await sampler.routeWithShadow(query, { strategy: 'auto' }); | ||
| * // decision.hasShadow === true iff we sampled a shadow provider | ||
| * // decision.primaryResult and decision.shadowResult are compared automatically | ||
| */ |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Docstring usage example doesn't match the actual API.
The example shows await sampler.routeWithShadow(query, { strategy: 'auto' }), but routeWithShadow (Line 299) is synchronous (returns ShadowDecision, not a Promise) and its options type (Line 301) only accepts available_models/budget_multiplier — no strategy field. A caller copying this example as-is with a typed literal would hit an excess-property TypeScript error.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routing/shadowSampler.ts` around lines 26 - 31, Update the ShadowSampler
docstring usage example to call the synchronous routeWithShadow method without
await and pass only supported options, such as available_models or
budget_multiplier. Keep the example’s ShadowDecision result and comparison
behavior accurate.
| private selectShadowProvider(primaryProvider: string): string { | ||
| if (this.config.forceShadowProvider && this.config.shadowProvider) { | ||
| return this.config.shadowProvider; | ||
| } | ||
|
|
||
| if (this._shadowProvider) return this._shadowProvider; | ||
|
|
||
| const profiles = getAvailableProviders(); | ||
| const candidates = Object.entries(profiles) | ||
| .filter(([name, p]: [string, ProviderDefinition]) => { | ||
| // Exclude primary | ||
| if (name === primaryProvider) return false; | ||
| // Must be available (has API key) — cost must be finite | ||
| const cost = (p.costPerK.input + p.costPerK.output) / 2; | ||
| return cost < Infinity; | ||
| }) | ||
| .sort((a, b) => { | ||
| const costA = (a[1].costPerK.input + a[1].costPerK.output) / 2; | ||
| const costB = (b[1].costPerK.input + b[1].costPerK.output) / 2; | ||
| return costA - costB; | ||
| }); | ||
|
|
||
| if (candidates.length === 0) { | ||
| // Fallback: pick any non-primary | ||
| const fallback = Object.keys(profiles).find(n => n !== primaryProvider); | ||
| this._shadowProvider = fallback || primaryProvider; | ||
| } else { | ||
| this._shadowProvider = candidates[0][0]; | ||
| } | ||
|
|
||
| return this._shadowProvider; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Shadow provider caching breaks the primary-exclusion guarantee.
_shadowProvider is cached on first successful selection (Line 161) and returned unconditionally on all later calls, ignoring the primaryProvider argument. If a later routeWithShadow call resolves a different primary_model (Line 304, 313), the exclusion logic that filters name === primaryProvider (Line 167) never re-runs — the cached provider can equal the new primary, so the shadow verification silently compares a provider's output against itself, defeating the purpose of shadow verification described in this method's own docstring ("Excludes the primary provider to ensure diversity", Lines 154-155).
🐛 Proposed fix
- if (this._shadowProvider) return this._shadowProvider;
+ if (this._shadowProvider && this._shadowProvider !== primaryProvider) return this._shadowProvider;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| private selectShadowProvider(primaryProvider: string): string { | |
| if (this.config.forceShadowProvider && this.config.shadowProvider) { | |
| return this.config.shadowProvider; | |
| } | |
| if (this._shadowProvider) return this._shadowProvider; | |
| const profiles = getAvailableProviders(); | |
| const candidates = Object.entries(profiles) | |
| .filter(([name, p]: [string, ProviderDefinition]) => { | |
| // Exclude primary | |
| if (name === primaryProvider) return false; | |
| // Must be available (has API key) — cost must be finite | |
| const cost = (p.costPerK.input + p.costPerK.output) / 2; | |
| return cost < Infinity; | |
| }) | |
| .sort((a, b) => { | |
| const costA = (a[1].costPerK.input + a[1].costPerK.output) / 2; | |
| const costB = (b[1].costPerK.input + b[1].costPerK.output) / 2; | |
| return costA - costB; | |
| }); | |
| if (candidates.length === 0) { | |
| // Fallback: pick any non-primary | |
| const fallback = Object.keys(profiles).find(n => n !== primaryProvider); | |
| this._shadowProvider = fallback || primaryProvider; | |
| } else { | |
| this._shadowProvider = candidates[0][0]; | |
| } | |
| return this._shadowProvider; | |
| } | |
| private selectShadowProvider(primaryProvider: string): string { | |
| if (this.config.forceShadowProvider && this.config.shadowProvider) { | |
| return this.config.shadowProvider; | |
| } | |
| if (this._shadowProvider && this._shadowProvider !== primaryProvider) return this._shadowProvider; | |
| const profiles = getAvailableProviders(); | |
| const candidates = Object.entries(profiles) | |
| .filter(([name, p]: [string, ProviderDefinition]) => { | |
| // Exclude primary | |
| if (name === primaryProvider) return false; | |
| // Must be available (has API key) — cost must be finite | |
| const cost = (p.costPerK.input + p.costPerK.output) / 2; | |
| return cost < Infinity; | |
| }) | |
| .sort((a, b) => { | |
| const costA = (a[1].costPerK.input + a[1].costPerK.output) / 2; | |
| const costB = (b[1].costPerK.input + b[1].costPerK.output) / 2; | |
| return costA - costB; | |
| }); | |
| if (candidates.length === 0) { | |
| // Fallback: pick any non-primary | |
| const fallback = Object.keys(profiles).find(n => n !== primaryProvider); | |
| this._shadowProvider = fallback || primaryProvider; | |
| } else { | |
| this._shadowProvider = candidates[0][0]; | |
| } | |
| return this._shadowProvider; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routing/shadowSampler.ts` around lines 156 - 187, Update
selectShadowProvider so cached _shadowProvider is reused only when it differs
from the current primaryProvider; otherwise rerun candidate selection and cache
a valid non-primary provider. Preserve the existing forced-provider behavior
while ensuring every returned shadow provider remains distinct from the
primaryProvider.
Adds EXP3 diversity, Charnov MVT, and ODT shadow verification to the LiteLLM comparison section. Updates architecture description and adds RouterArena benchmark link.
- Tagline now mentions biology-inspired provider selection - Added one-liner for each mechanism (EXP3, Charnov MVT, ODT) - Architecture section lists all three mechanisms by name
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Around line 253-255: Update the “Independent Benchmarks” section in README.md
to clearly distinguish the 96.77% result and its metric from RouterArena, whose
accuracy must remain 0.5893. Remove or qualify the “No. 1” claim unless it is
supported by the correctly identified benchmark, and link to benchmark
documentation consistent with the stated results.
- Around line 60-74: Revise the README comparison to remove or omit the “54K
stars” count and replace absolute claims about LiteLLM’s missing capabilities
with qualified wording. In the heuristic-routing section, describe A3M’s
particular model="auto" content-based provider selection without asserting
LiteLLM only routes by model name, and frame parallel ensemble execution as
A3M’s behavior rather than a capability LiteLLM lacks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
| LiteLLM is the dominant open-source AI gateway (54K stars). It handles unified API access well. A3M Router adds three capabilities LiteLLM doesn't have built-in: | ||
|
|
||
| ### 1. Heuristic Routing | ||
| LiteLLM routes by model name or requires you to specify which model to call. A3M's `model="auto"` mode analyzes the query content and picks the cheapest capable provider automatically. This is useful when you want cost efficiency without writing routing logic. | ||
|
|
||
| ### 2. Parallel Ensemble Execution | ||
| ### 2. Biology-Inspired Provider Selection | ||
| A3M applies established ecological and evolutionary theory to routing decisions: | ||
|
|
||
| **EXP3 Diversity Weighting** — Negative frequency-dependent selection prevents any single provider from dominating traffic. Providers above their fair share (1/n of total) receive a penalty proportional to their deviation. This mirrors how ecological niches prevent competitive exclusion — no species dominates when resource competition is symmetric. | ||
|
|
||
| **Charnov MVT Rate-Limit Rotation** — When a provider's rate-limit window becomes depleted, A3M uses the Marginal Value Theorem (Charnov 1976) to decide the optimal time to switch. It leaves when the marginal remaining rate falls below the average rate including switch cost — the same logic that explains when animals should leave a depleting food patch. | ||
|
|
||
| **ODT Shadow Verification** — For high-stakes queries, A3M can probabilistically sample a shadow provider to verify the primary's answer. The sampling probability follows Optimal Defense Theory: tissue value (query stakes) and attack probability (risk profile) scale verification effort proportionally. This is how plants allocate defensive compounds — expensive defenses go to valuable tissues. | ||
|
|
||
| ### 3. Parallel Ensemble Execution |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "README relevant section:"
sed -n '50,85p' README.md
echo
echo "LiteLLM references in repo:"
rg -n "LiteLLM|54K|model=\"auto\"|Heuristic Routing|Biology-Inspired|Parallel Ensemble" -S README.md . --glob '!/.git/**' || true
echo
echo "Current LiteLLM repository stats/features if network API is available:"
python3 - <<'PY'
import urllib.request, json, urllib.error
try:
req = urllib.request.Request("https://api.github.com/repos/BerriAI/llm_router", headers={"User-Agent": "CodeRabbit-review-verifier"})
with urllib.request.urlopen(req, timeout=20) as r:
data = json.load(r)
print("repo:", data.get("full_name"))
print("stargazers_count:", data.get("stargazers_count"))
print("description:", data.get("description"))
print("topics:", data.get("topics"))
except Exception as e:
print("api_unavailable_or_error:", repr(e))
PYRepository: Das-rebel/a3m-router
Length of output: 16387
🌐 Web query:
GitHub BerriAI/llm_router repository stars description model routing
💡 Result:
The functionality related to llm_router is integrated directly into the BerriAI/litellm repository [1][2], rather than existing as a standalone repository named BerriAI/llm_router. LiteLLM is a popular open-source AI gateway and Python SDK that provides model routing, load balancing, and failover capabilities for over 100 LLM APIs [1][3]. The llm_router is a core component within LiteLLM that handles the intelligent distribution of API requests across configured model deployments [1][4][2]. Key aspects of its model routing functionality include: Supported Routing Strategies: The router supports several configurable strategies, including simple-shuffle, least-busy, usage-based, latency-based, and cost-based routing [5]. Routing Groups: A feature that allows users to create named, persisted routing pipelines, grouping multiple model deployments under a single logical model name with specific strategies such as priority-failover or weighted distribution [6]. Configuration and Persistence: The router can be configured via YAML files or managed dynamically through the LiteLLM Proxy server [1][6]. It handles fallbacks, retries, and context window management to ensure robust request execution [5][7]. Deployment: It is instantiated within the LiteLLM proxy server to manage live traffic, and it supports hot-reloading of configurations without requiring service restarts [2][3][6]. As of mid-2026, the LiteLLM repository has surpassed 50,000 stars [3]. Users looking for model routing documentation or implementation details should refer to the LiteLLM documentation and the litellm/router.py module within the main BerriAI/litellm repository [1][5].
Citations:
- 1: initialize llm_router with model rather than config BerriAI/litellm#12009
- 2: fix(proxy): only create Router when models or search_tools exist BerriAI/litellm#20661
- 3: BerriAI/litellm@f1da202
- 4: https://github.com/BerriAI/litellm/blob/62920a0c/litellm/proxy/route_llm_request.py
- 5: https://github.com/BerriAI/litellm/blob/62920a0c/litellm/types/router.py
- 6: feat(proxy): add Routing Groups BerriAI/litellm#22471
- 7: https://github.com/BerriAI/litellm/blob/559a6ad826b5daef41565f54f06c739c8c068b28/litellm/types/router.py
🌐 Web query:
BerriAI/llm_router GitHub repository
💡 Result:
There is no standalone GitHub repository named BerriAI/llm_router. The functionality referred to as the llm_router is a core component within the BerriAI/litellm repository [1][2][3]. LiteLLM is an open-source AI gateway and Python library that provides a unified interface for over 100 LLM providers [1][4]. Within this project, the llm_router (specifically the Router class) is the module responsible for managing model routing, load balancing, and fallback logic across multiple deployments or providers [1][3][4]. Key details regarding the llm_router component include: Role and Functionality: It acts as a controller for handling requests, enabling features such as automatic retries, fallback chains (e.g., trying a different provider if one fails), and load balancing across various API keys or deployments [4]. It can be used directly within a Python application via the LiteLLM SDK or automatically initialized by the LiteLLM Proxy Server when a configuration file or specific model is provided [1][2][3]. Architecture: In the LiteLLM Proxy Server, a global llm_router instance is typically populated from a configuration file (often config.yaml) to manage traffic centrally [3]. It is integral to the proxy's ability to handle multi-tenancy, cost tracking, and model selection [1][3]. For developers looking to implement or configure routing logic, documentation and reference implementations (such as cookbooks for custom routing strategies) are maintained within the main BerriAI/litellm repository [5].
Citations:
- 1: https://github.com/BerriAI/litellm
- 2: initialize llm_router with model rather than config BerriAI/litellm#12009
- 3: https://deepwiki.com/BerriAI/litellm
- 4: https://ai.miraheze.org/wiki/LiteLLM
- 5: cookbook: add complexity-based routing example (CustomRoutingStrategyBase) BerriAI/litellm#33045
Date or omit the LiteLLM star count and soften the capability comparison.
“54K stars” is specific and can become stale, while “A3M Router adds three capabilities LiteLLM doesn’t have built-in” and “LiteLLM routes by model name or requires you to specify which model to call” overstate the gap as LiteLLM supports configurable router strategies, routing groups, fallbacks, retries, load balancing, and cost-based routing. Frame this as A3M’s particular model="auto" content-based heuristic routing and parallel-ensemble behavior instead of absolute missing capability.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 60 - 74, Revise the README comparison to remove or
omit the “54K stars” count and replace absolute claims about LiteLLM’s missing
capabilities with qualified wording. In the heuristic-routing section, describe
A3M’s particular model="auto" content-based provider selection without asserting
LiteLLM only routes by model name, and frame parallel ensemble execution as
A3M’s behavior rather than a capability LiteLLM lacks.
Source: MCP tools
| ## Independent Benchmarks | ||
|
|
||
| A3M ranks **No. 1** among known public routing baselines on RouterArena (8,400 queries, 96.77% accuracy, $0.0768/1K cost, 1.0000 robustness). See [`docs/BENCHMARK.md`](docs/BENCHMARK.md) for full reproducible benchmarks. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not label the separate benchmark as RouterArena accuracy.
The PR objectives state that RouterArena remains 0.5893, while 96.77% belongs to a separate benchmark. This text currently presents 96.77% as RouterArena accuracy and claims “No. 1,” which is misleading. Name the benchmark and metric explicitly, or reconcile the source results before publishing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 253 - 255, Update the “Independent Benchmarks”
section in README.md to clearly distinguish the 96.77% result and its metric
from RouterArena, whose accuracy must remain 0.5893. Remove or qualify the “No.
1” claim unless it is supported by the correctly identified benchmark, and link
to benchmark documentation consistent with the stated results.
GitHub About and npm now lead with biology-inspired routing: EXP3 diversity, MVT rate-limit rotation, ODT verification.
1 participant
Summary
Three biologically-inspired routing improvements from Agent Council review:
1. EXP3 Adaptive Diversity Weight (
advancedRouter.ts)gamma = sqrt(n log(n) / (T * G²)),penalty = gamma * max(0, share - 1/n)2. Charnov MVT Rate-Limit Rotation (
providerHealth.ts)Leave when: marginal_rate <= cumulative_rate / (elapsed + τ_switch)3. ODT Shadow Sampler (
shadowSampler.ts)P(shadow) = f(stake, risk, complexity)— replaces binary always-on/offAgent Council Process
All 3 ideas survived rigorous expert review. 5 ideas were killed:
Tests
Eval
RouterArena eval: 0.5893 (identical to baseline — pre-existing failure, no regression)
RouterArena Score: 0.9404 | Accuracy: 96.77% | Cost: $0.0768/1K | Robustness: 1.0000
Source: RouterArena PR #144
Summary by CodeRabbit