[Router][CLI][Dashboard][Docs] Support decision-scoped candidate quality

config/README.md

@@ -18,6 +18,7 @@ Inside canonical `config.yaml`:
 - `routing.projections.partitions` is the canonical runtime home for exclusive domain or embedding partitions; DSL authoring uses `PROJECTION partition`
 - `routing.projections.scores` and `routing.projections.mappings` let maintained configs turn learned and heuristic signals into named routing bands that decisions can reference with `type: projection`
 - `routing.decisions[].candidateIterations` carries bounded DSL `FOR ... IN` metadata for candidate-model authoring; it is declarative selection policy input, not a general scripting runtime
+- `routing.decisions[].modelRefs[].quality_score` optionally overrides the model-card quality score for that candidate in the owning decision; `multi_factor` falls back to the model-card value only when this field is omitted
 - `routing.decisions[].emits[]` carries typed side-effect directives from DSL `EMIT` blocks; the current supported kind is `retention`, where `drop: true` skips response-side semantic-cache writes and the remaining fields stay structured/auditable for follow-up runtime consumers such as turn-aware cache TTL, current-model affinity, prefix/KV-cache warmth, and session transition telemetry
 - request-shape detectors such as `routing.signals.structure` stay in the signal layer as typed named facts; numeric thresholds live inside the detector config instead of turning decisions into a free-form expression language
 - `routing.signals.embeddings[].query_modality` declares which modality of incoming request payload the embedding rule's query is computed from. Defaults to `"text"`; `"image"` and `"audio"` require `global.model_catalog.embeddings.semantic.embedding_config.model_type=multimodal` so the query and candidate embeddings land in the same shared space. See `website/docs/tutorials/signal/learned/embedding.md` for the worked multimodal example.
@@ -46,7 +47,7 @@ Inside canonical `config.yaml`:
 - `not/`: exclusion examples
 - `composite/`: nested AND/OR/NOT cases
 
-Decision fragments may reference `modelRefs[].lora_name`, but those adapter names must be declared in the base config's `routing.modelCards[].loras`.
+Decision fragments may reference `modelRefs[].lora_name`, but those adapter names must be declared in the base config's `routing.modelCards[].loras`. A `modelRefs[].quality_score` value is decision-scoped, must be between `0` and `1`, and preserves an explicit `0`.
 Candidate iteration fragments must stay bounded to `decision.candidates` or an explicit model list and feed existing decision outputs such as `MODEL <iterator>`.
 
 `config/algorithm/` is organized by routing policy:

config/config.yaml

@@ -571,6 +571,7 @@ routing:
             - model: qwen3-8b
               lora_name: business-adapter
               weight: 1.0
+              quality_score: 0.83
               use_reasoning: false
               reasoning_description: Candidate iteration reference model for DSL authoring coverage.
               reasoning_effort: low
@@ -1135,7 +1136,9 @@ routing:
             name: other
       modelRefs:
         - model: qwen3-8b
+          quality_score: 0.72
         - model: qwen3-32b
+          quality_score: 0.94
       algorithm:
         type: multi_factor
         multi_factor:

dashboard/frontend/src/pages/ConfigPageDecisionModelQuality.test.tsx

@@ -0,0 +1,15 @@
+import { readFileSync } from 'node:fs'
+import { describe, expect, it } from 'vitest'
+
+describe('decision model quality editor', () => {
+  it('preserves and edits decision-scoped quality scores', () => {
+    const source = readFileSync(
+      new URL('./ConfigPageDecisionsSection.tsx', import.meta.url),
+      'utf8',
+    )
+
+    expect(source).toContain('Decision quality score')
+    expect(source).toContain("| 'quality_score'")
+    expect(source).toContain('modelRef.quality_score = modelRefValue.quality_score')
+  })
+})

dashboard/frontend/src/pages/ConfigPageDecisionsSection.tsx

@@ -114,6 +114,7 @@ export default function ConfigPageDecisionsSection({
       ref.reasoning_effort ? `Effort: ${ref.reasoning_effort}` : null,
       ref.lora_name ? `LoRA: ${ref.lora_name}` : null,
       typeof ref.weight === 'number' ? `Weight: ${ref.weight}` : null,
+      typeof ref.quality_score === 'number' ? `Quality: ${ref.quality_score}` : null,
     ].filter((value): value is string => Boolean(value))
 
     const details = [
@@ -355,6 +356,7 @@ export default function ConfigPageDecisionsSection({
               reasoning_effort: ref.reasoning_effort || '',
               lora_name: ref.lora_name || '',
               weight: typeof ref.weight === 'number' ? ref.weight : undefined,
+              quality_score: typeof ref.quality_score === 'number' ? ref.quality_score : undefined,
             })),
             plugins: (decision.plugins || []).map((plugin) => ({
               type: plugin.type,
@@ -471,7 +473,8 @@ export default function ConfigPageDecisionsSection({
           | 'reasoning_description'
           | 'reasoning_effort'
           | 'lora_name'
-          | 'weight',
+          | 'weight'
+          | 'quality_score',
         val: string | boolean | number | undefined,
       ) => {
         const next = rows.map((item, idx) => (idx === index ? { ...item, [key]: val } : item))
@@ -592,6 +595,27 @@ export default function ConfigPageDecisionsSection({
                     className={decisionStyles.editorInput}
                   />
                 </label>
+                <label className={decisionStyles.editorControlLabel}>
+                  <span className={decisionStyles.editorControlLabelText}>
+                    Decision quality score
+                  </span>
+                  <input
+                    type="number"
+                    value={typeof ref?.quality_score === 'number' ? ref.quality_score : ''}
+                    onChange={(e) =>
+                      updateItem(
+                        idx,
+                        'quality_score',
+                        e.target.value === '' ? undefined : Number(e.target.value),
+                      )
+                    }
+                    placeholder="Optional 0–1 override"
+                    step="0.01"
+                    min="0"
+                    max="1"
+                    className={decisionStyles.editorInput}
+                  />
+                </label>
               </div>
 
               <label className={decisionStyles.editorControlLabel}>
@@ -732,6 +756,15 @@ export default function ConfigPageDecisionsSection({
         if (typeof modelRefValue?.weight === 'number' && Number.isFinite(modelRefValue.weight)) {
           modelRef.weight = modelRefValue.weight
         }
+        if (
+          typeof modelRefValue?.quality_score === 'number' &&
+          Number.isFinite(modelRefValue.quality_score)
+        ) {
+          if (modelRefValue.quality_score < 0 || modelRefValue.quality_score > 1) {
+            throw new Error(`Model reference #${idx + 1} quality score must be between 0 and 1.`)
+          }
+          modelRef.quality_score = modelRefValue.quality_score
+        }
         return modelRef
       })
 

dashboard/frontend/src/pages/configPageSupport.ts

@@ -206,6 +206,7 @@ export interface DecisionModelRef {
   reasoning_effort?: string
   lora_name?: string
   weight?: number
+  quality_score?: number
 }
 
 export interface DecisionPluginConfig {

src/semantic-router/pkg/config/canonical_config.go

@@ -2,6 +2,7 @@ package config
 
 import (
 	"fmt"
+	"math"
 	"sort"
 	"strconv"
 	"strings"
@@ -246,6 +247,9 @@ func validateCanonicalDecisionModelRefs(decision Decision, modelsByName map[stri
 		if modelRef.Model == "" {
 			continue
 		}
+		if modelRef.QualityScore != nil && (math.IsNaN(*modelRef.QualityScore) || *modelRef.QualityScore < 0 || *modelRef.QualityScore > 1) {
+			return fmt.Errorf("routing.decisions[%s].modelRefs[%s].quality_score must be between 0 and 1", decision.Name, modelRef.Model)
+		}
 		// Reject modelRefs that point at a model the config does not define.
 		// Previously this was only checked when a lora_name was also set, so a
 		// plain modelRef to an unknown model slipped through and only failed at

src/semantic-router/pkg/config/decision_config.go

@@ -139,9 +139,12 @@ type ModelReasoningControl struct {
 }
 
 type ModelRef struct {
-	Model                 string  `yaml:"model"`
-	LoRAName              string  `yaml:"lora_name,omitempty"`
-	Weight                float64 `yaml:"weight,omitempty"`
+	Model    string  `yaml:"model"`
+	LoRAName string  `yaml:"lora_name,omitempty"`
+	Weight   float64 `yaml:"weight,omitempty"`
+	// QualityScore optionally overrides the model-card quality score for this
+	// candidate in the owning decision. A pointer preserves explicit zero.
+	QualityScore          *float64 `yaml:"quality_score,omitempty" json:"quality_score,omitempty"`
 	ModelReasoningControl `yaml:",inline"`
 }
 

src/semantic-router/pkg/config/decision_reference_validation_test.go

@@ -3,6 +3,8 @@ package config
 import (
 	"strings"
 	"testing"
+
+	"gopkg.in/yaml.v3"
 )
 
 // buildDecisionRefConfig wraps a decisions block in an otherwise-valid v0.3
@@ -91,6 +93,66 @@ func TestDecisionModelRefKnownModelAccepted(t *testing.T) {
 	}
 }
 
+func TestDecisionModelRefQualityScoreRoundTripsExplicitZero(t *testing.T) {
+	cfg, err := ParseYAMLBytes(buildDecisionRefConfig(`    - name: d1
+      priority: 1
+      rules: {operator: AND, conditions: []}
+      modelRefs:
+        - model: m1
+          quality_score: 0
+          use_reasoning: false
+`))
+	if err != nil {
+		t.Fatalf("parse decision quality score: %v", err)
+	}
+	quality := cfg.Decisions[0].ModelRefs[0].QualityScore
+	if quality == nil || *quality != 0 {
+		t.Fatalf("expected explicit zero quality score, got %#v", quality)
+	}
+
+	exported, err := yaml.Marshal(CanonicalConfigFromRouterConfig(cfg))
+	if err != nil {
+		t.Fatalf("marshal canonical config: %v", err)
+	}
+	if !strings.Contains(string(exported), "quality_score: 0") {
+		t.Fatalf("exported canonical config dropped explicit zero quality score:\n%s", exported)
+	}
+}
+
+func TestDecisionModelRefQualityScoreRejectsOutOfRange(t *testing.T) {
+	_, err := ParseYAMLBytes(buildDecisionRefConfig(`    - name: d1
+      priority: 1
+      rules: {operator: AND, conditions: []}
+      modelRefs:
+        - model: m1
+          quality_score: 1.1
+          use_reasoning: false
+`))
+	if err == nil {
+		t.Fatal("expected out-of-range decision quality score to be rejected")
+	}
+	if !strings.Contains(err.Error(), "quality_score") {
+		t.Fatalf("error should name quality_score, got: %v", err)
+	}
+}
+
+func TestDecisionModelRefQualityScoreRejectsNaN(t *testing.T) {
+	_, err := ParseYAMLBytes(buildDecisionRefConfig(`    - name: d1
+      priority: 1
+      rules: {operator: AND, conditions: []}
+      modelRefs:
+        - model: m1
+          quality_score: .nan
+          use_reasoning: false
+`))
+	if err == nil {
+		t.Fatal("expected NaN decision quality score to be rejected")
+	}
+	if !strings.Contains(err.Error(), "quality_score") {
+		t.Fatalf("error should name quality_score, got: %v", err)
+	}
+}
+
 // G2: two decisions sharing the same name are ambiguous and must be rejected.
 func TestDuplicateDecisionNamesRejected(t *testing.T) {
 	cfg := buildDecisionRefConfig(`    - name: dup

src/semantic-router/pkg/config/validator_decision.go

@@ -2,6 +2,7 @@ package config
 
 import (
 	"fmt"
+	"math"
 	"strings"
 
 	"github.com/vllm-project/semantic-router/src/semantic-router/pkg/observability/logging"
@@ -46,6 +47,9 @@ func validateDecisionModelRefs(cfg *RouterConfig, decision Decision) error {
 		if modelRef.UseReasoning == nil {
 			return fmt.Errorf("decision '%s', model '%s': missing required field 'use_reasoning'", decision.Name, modelRef.Model)
 		}
+		if modelRef.QualityScore != nil && (math.IsNaN(*modelRef.QualityScore) || *modelRef.QualityScore < 0 || *modelRef.QualityScore > 1) {
+			return fmt.Errorf("decision '%s', modelRefs[%d].quality_score must be between 0 and 1", decision.Name, i)
+		}
 		if modelRef.LoRAName == "" {
 			continue
 		}

src/semantic-router/pkg/selection/multi_factor.go

@@ -219,11 +219,16 @@ func (s *MultiFactorSelector) gatherSignals(candidates []config.ModelRef) []sign
 	out := make([]signalSet, 0, len(candidates))
 	for _, c := range candidates {
 		sig := signalSet{model: c.Model}
-		if params, ok := s.modelParams[c.Model]; ok {
+		if c.QualityScore != nil {
+			sig.quality = *c.QualityScore
+			sig.hasQ = true
+		} else if params, ok := s.modelParams[c.Model]; ok {
 			if params.QualityScore > 0 {
 				sig.quality = params.QualityScore
 				sig.hasQ = true
 			}
+		}
+		if params, ok := s.modelParams[c.Model]; ok {
 			if params.Pricing.PromptPer1M > 0 {
 				sig.cost = params.Pricing.PromptPer1M
 				sig.hasCost = true

src/semantic-router/pkg/selection/multi_factor_test.go

@@ -48,6 +48,72 @@ func candidates(names ...string) []config.ModelRef {
 	return out
 }
 
+func qualityScore(value float64) *float64 {
+	return &value
+}
+
+func TestMultiFactor_DecisionQualityOverridesGlobalQuality(t *testing.T) {
+	cfg := DefaultMultiFactorConfig()
+	cfg.Weights = MultiFactorWeights{Quality: 1.0}
+	params := map[string]config.ModelParams{
+		"globally-low":  {QualityScore: 0.1},
+		"globally-high": {QualityScore: 0.9},
+	}
+	s := buildMFSelector(cfg, params, nil, nil, nil)
+	refs := []config.ModelRef{
+		{Model: "globally-low", QualityScore: qualityScore(0.95)},
+		{Model: "globally-high", QualityScore: qualityScore(0.2)},
+	}
+
+	res, err := s.Select(context.Background(), &SelectionContext{CandidateModels: refs})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if res.SelectedModel != "globally-low" {
+		t.Fatalf("expected decision quality override to select globally-low, got %s; scores=%v", res.SelectedModel, res.AllScores)
+	}
+}
+
+func TestMultiFactor_DecisionQualityFallsBackToGlobalQuality(t *testing.T) {
+	cfg := DefaultMultiFactorConfig()
+	cfg.Weights = MultiFactorWeights{Quality: 1.0}
+	params := map[string]config.ModelParams{
+		"global-best": {QualityScore: 0.8},
+		"global-low":  {QualityScore: 0.2},
+	}
+	s := buildMFSelector(cfg, params, nil, nil, nil)
+
+	res, err := s.Select(context.Background(), &SelectionContext{CandidateModels: candidates("global-best", "global-low")})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if res.SelectedModel != "global-best" {
+		t.Fatalf("expected omitted decision quality to use global quality, got %s; scores=%v", res.SelectedModel, res.AllScores)
+	}
+}
+
+func TestMultiFactor_ExplicitZeroDecisionQualityDoesNotFallBack(t *testing.T) {
+	cfg := DefaultMultiFactorConfig()
+	cfg.Weights = MultiFactorWeights{Quality: 1.0}
+	params := map[string]config.ModelParams{
+		"overridden-zero": {QualityScore: 0.9},
+		"positive":        {QualityScore: 0.2},
+	}
+	s := buildMFSelector(cfg, params, nil, nil, nil)
+	refs := []config.ModelRef{
+		{Model: "overridden-zero", QualityScore: qualityScore(0)},
+		{Model: "positive"},
+	}
+
+	res, err := s.Select(context.Background(), &SelectionContext{CandidateModels: refs})
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if res.SelectedModel != "positive" {
+		t.Fatalf("expected explicit zero to override global quality, got %s; scores=%v", res.SelectedModel, res.AllScores)
+	}
+}
+
 func TestMultiFactor_PicksHighestQualityWhenQualityDominant(t *testing.T) {
 	cfg := DefaultMultiFactorConfig()
 	cfg.Weights = MultiFactorWeights{Quality: 1.0}

src/vllm-sr/cli/algorithms.py

@@ -16,6 +16,7 @@ class ModelRef(BaseModel):
     )
     lora_name: str | None = None  # LoRA adapter name (if using LoRA)
     weight: float | None = None
+    quality_score: float | None = Field(default=None, ge=0, le=1)
 
 
 class HybridWeightsConfig(BaseModel):

src/vllm-sr/tests/test_config_contract.py

@@ -1,3 +1,4 @@
+from cli.algorithms import ModelRef
 from cli.config_contract import (
     LEGACY_SIGNAL_KEY_TO_CANONICAL,
     build_projection_reference_index,
@@ -98,6 +99,12 @@ def test_decision_accepts_typed_output_contract_spec():
     assert decision.output_contract_spec.choice_set.values == ["A", "B", "C", "D"]
 
 
+def test_model_ref_preserves_optional_quality_score_and_explicit_zero():
+    assert ModelRef(model="model-a").quality_score is None
+    assert ModelRef(model="model-a", quality_score=0).quality_score == 0
+    assert ModelRef(model="model-a", quality_score=0.92).quality_score == 0.92
+
+
 def test_decision_accepts_terminal_action_output_contract_spec():
     decision = Decision(
         name="terminal",

website/docs/installation/configuration.md

@@ -35,6 +35,7 @@ The detailed background is in [Unified Config Contract v0.3](../proposals/unifie
   - `routing.signals`
   - `routing.projections` for partitions plus derived routing outputs
   - `routing.decisions`
+  - `routing.decisions[].modelRefs[].quality_score` is an optional decision-specific quality override for candidate selection; omission falls back to the model-card quality score, while an explicit `0` remains zero
 - `entrypoints` and `recipes` own multi-profile routing.
   - `entrypoints[].model_names` are request-facing virtual model names; they behave like auto-model aliases, never reach a backend, and are listed by `/v1/models`
   - `entrypoints[].recipe` selects which recipe evaluates matching requests
@@ -179,6 +180,7 @@ routing:
             name: support_escalated
       modelRefs:
         - model: qwen3-8b
+          quality_score: 0.92
           use_reasoning: true
           lora_name: math-adapter
       emits:

website/docs/proposals/unified-config-contract-v0-3.md

@@ -81,6 +81,7 @@ Model semantics and deployment bindings are now separated explicitly:
 - each `providers.models[].backend_refs[]` item carries its own transport and auth fields such as `endpoint`, `base_url`, `protocol`, `auth_header`, `auth_prefix`, `api_key`, and `api_key_env`
 - `providers.models[].pricing` can price prompt, cached-input, cache-write, and completion tokens independently; an omitted cache-write rate inherits the prompt rate
 - `routing.decisions[].modelRefs[].lora_name` resolves against the matching `routing.modelCards[].loras` entry, so `lora_name` is now part of the supported routing contract instead of a runtime-only escape hatch
+- `routing.decisions[].modelRefs[].quality_score` optionally overrides global model-card quality for the owning decision's candidate selection. It is constrained to `0..1`; omission means fallback, and explicit zero is preserved.
 - `routing.decisions[].output_contract` is the decision-scoped, model-visible final response format contract. Loop algorithms merge it with any format already present in the client request instead of hard-coding benchmark- or task-specific prompts inside algorithms.
 - `routing.decisions[].output_contract_spec` is the typed router-executable output contract. Use it for machine-checked post-processing such as `type: choice`, `type: structured_json` with `json_schema.schema_ref: terminal_action_v1`, or `type: reference_selection` with `postprocess: [{type: dereference_selected_reference}]`; extraction defaults to exact `content` matching and must be widened explicitly with `extract.sources`. Do not encode these runtime behaviors as prompt-text heuristics.
 - `routing.decisions[].candidateIterations` is bounded to `decision.candidates` or explicit model lists and remains declarative metadata for the selection layer, not a second policy interpreter