feat(#95): agent — metric-source primary selection + per-category overrides (#96)

* feat(#95): agent — metric-source primary selection + per-category overrides

First foundational PR for issue #95 (Phase 3 of #91). Establishes the
"who's the primary metric source on this host" mechanism so future PRs
can drop in nginx / Apache / Caddy / Traefik detectors without re-
architecting how the dashboard picks whose numbers to render.

Background: most hosts have one obvious producer per metric category
(HTTP requests, response codes, etc.). Where two coexist — e.g. HAProxy
fronting nginx, both genuinely serving traffic — the agent now picks
one as primary using a built-in preference list and surfaces the choice
in the capability manifest. The dashboard reads primary_sources and
renders that source's data; alternatives travel along so a "switch
source" UI can offer them. Operators override the pick per-category via
env var for the edge case where auto-detection chooses wrong.

Adds:
- MetricCategory + MetricSourceGear interface (gear/source.go). One
  category defined today: CategoryHTTPRequests. The interface is
  opt-in — gears that don't produce metrics simply don't implement it.
- preferenceOrder map keyed by category; first-match-wins among
  Available producers. Documented rationale for the homelab-flavoured
  ordering (HAProxy first as the L7 entry point).
- Manager.ResolvePrimarySources() — resolves the primary for each
  category, honouring an operator override if present + valid, falling
  back to auto-detection with a warning otherwise. Producers not in
  preferenceOrder still surface as alternatives (alphabetised) so
  newer gears don't disappear from the UI before someone gets around
  to ranking them.
- SourceSelection struct surfaced in CapabilitiesResponse under
  primary_sources — fields are stable JSON keys: source, reason,
  alternatives.
- Per-category override env var: GEARBOX_AGENT_HTTP_SOURCE. Names a
  gear; case-insensitive, trimmed, lowercased at load. Validated
  against registered producers; invalid names log a warning and fall
  back rather than dropping metrics for the category.
- Startup log line per resolved category so operators can confirm
  their override took effect via journalctl without hitting the API.
- HAProxy gear declares CategoryHTTPRequests so the existing source
  works end-to-end immediately.

Tests (8 new in gear/source_test.go, 3 new in config_test.go):
- Auto-detect picks first Available from preferenceOrder.
- Operator override wins over preference; reason names env var.
- Override target unavailable → fall back to auto + warning.
- Override target unknown / not a producer for the category → same.
- No available producers → category omitted from result entirely.
- Unranked producer still surfaces as alternative (alphabetised).
- End-to-end /api/v1/system/capabilities includes primary_sources.
- normaliseSourceOverride trims + lowercases edge cases.
- Load() picks up GEARBOX_AGENT_HTTP_SOURCE; default is empty.

Docs: README "Metric-source overrides" section explains the model and
override behaviour, with explicit "auto-detection is the default" copy.

Out of scope (later phases of #95):
- nginx / Apache / Caddy / Traefik / Docker gear stubs.
- Additional metric categories (backend health, container metrics).
- Dashboard consumption of primary_sources.

Refs: docs/research/metrics-source-agnostic.md (#91), issue #95.

* fix(#95): address Copilot review findings on PR #96

- manager.go: split ResolvePrimarySources (silent, called on every
  /api/v1/system/capabilities request) from ValidatePrimarySourceOverrides
  (logs warnings, called once from ProbeAll). The previous design re-logged
  the same override warning on every dashboard poll.
- manager.go: rename deps_override → overrideForCategory; the
  snake_case was inconsistent with the rest of the file.
- manager.go: replace hand-rolled containsString / withoutString with
  slices.Contains and slices.Clone + slices.DeleteFunc. Module is on
  Go 1.25; reaching for stdlib here trims maintenance surface.
- manager.go: defensive slices.Clone of the Alternatives slice in
  SourceSelection so a future change to the local `ordered` slice
  can't alias into a returned value.
- manager.go: replace overrideEnvVarFor's switch with a categoryEnvVar
  map keyed by MetricCategory; missing entry is a build-time-visible
  hole rather than a runtime "(unknown category)" placeholder leaking
  into operator-facing log messages and SourceSelection.Reason.
- config_test.go: switch new tests from os.Setenv + t.Cleanup to
  t.Setenv. Safer under panics, idiomatic since Go 1.17.

New tests:
- TestEveryCategoryHasOverrideEnvVar: guards categoryEnvVar against
  forgotten entries when a new MetricCategory is added.
- TestValidatePrimarySourceOverridesCountsMisses: confirms the
  validate-vs-resolve split — Validate reports mis-targeted overrides
  via its return value, Resolve stays silent.

Author: sarg3nt

gearbox-agent/README.md

@@ -206,6 +206,26 @@ HAPROXY_STATS_USER=admin
 HAPROXY_STATS_PASSWORD=secret
 ```
 
+### Metric-source overrides
+
+Most hosts have one obvious producer per metric category. Where two coexist — for example HAProxy fronting nginx, with both genuinely serving HTTP — the agent auto-picks a primary using a built-in preference list and surfaces it in the capability manifest. Operators can override that pick when auto-detection chooses wrong on their host.
+
+Auto-detection is the default. The override env vars below exist for the rare edge cases.
+
+```bash
+# Force a specific gear as the primary for HTTP-request metrics
+# (request volume, response codes, response times, vhost breakdowns).
+# Valid values match gear identifiers in /api/v1/system/capabilities —
+# today only "haproxy" is implemented; more land with issue #95.
+GEARBOX_AGENT_HTTP_SOURCE=nginx
+```
+
+Override behaviour:
+
+- Names are case-insensitive and trimmed (`HAProxy` and `haproxy` both work).
+- An override pointing at a gear that didn't probe Available, or doesn't produce data for the category, logs a warning at startup and **falls back to auto-detect** — locking out HTTP metrics because the override's target isn't installed on this box would be worse than serving auto-picked data.
+- The selected primary plus the chosen reason and the alternatives that were also available appear in `/api/v1/system/capabilities` under `primary_sources` so dashboards and humans can confirm the resolution.
+
 ## Authentication
 
 ### API Key

gearbox-agent/cmd/gearbox-agent/main.go

@@ -383,6 +383,7 @@ func main() {
 		HAProxyStatsPassword: cfg.HAProxyStatsPassword,
 		HAProxyConfigPath:    cfg.HAProxyConfigFile,
 		CertbotTimer:         cfg.CertbotTimer,
+		SourceOverrides:      buildSourceOverrides(cfg),
 	}
 
 	// Create plugin manager
@@ -565,3 +566,16 @@ func main() {
 
 	logger.Info("Server stopped")
 }
+
+// buildSourceOverrides packs the per-category override env vars from
+// the agent's Config into the category-keyed map that Dependencies
+// (and the manager's primary-source resolver) consume. Empty values
+// are omitted so the map only contains explicit operator picks —
+// callers downstream treat absence as "auto-detect".
+func buildSourceOverrides(cfg *config.Config) map[gear.MetricCategory]string {
+	out := make(map[gear.MetricCategory]string)
+	if cfg.HTTPSource != "" {
+		out[gear.CategoryHTTPRequests] = cfg.HTTPSource
+	}
+	return out
+}

gearbox-agent/internal/framework/config/config.go

@@ -66,6 +66,21 @@ type Config struct {
 	// endpoint + schema list to unauthenticated callers in production. See
 	// 2026-05 security audit P3-2.
 	SwaggerEnabled bool
+
+	// Metric-source overrides. Each one points a single metric category
+	// at a specific gear, bypassing auto-detection. Empty = pure
+	// auto-detection (built-in preference order). Lowercased at load
+	// time; the manager validates that each named gear is actually
+	// available before honouring the override (falls back to auto and
+	// warns otherwise).
+	//
+	// Background: most hosts have one obvious producer per metric
+	// category. When two coexist (e.g. HAProxy + nginx both serving
+	// HTTP), the agent picks one as primary; this override lets the
+	// operator force the choice for boxes where the built-in
+	// preference picks wrong. See [docs/source-detection.md] /
+	// issue #95.
+	HTTPSource string // GEARBOX_AGENT_HTTP_SOURCE — primary for CategoryHTTPRequests
 }
 
 // DefaultConfig returns the default configuration.
@@ -161,9 +176,22 @@ func Load() (*Config, error) {
 	// Swagger UI off by default; opt in for dev / API debugging.
 	cfg.SwaggerEnabled = os.Getenv("HAPROXY_AGENT_SWAGGER_ENABLED") == "true"
 
+	// Metric-source overrides. Lowercased + trimmed so 'HAProxy ' and
+	// 'haproxy' both match the gear's Info().Name. Empty = auto-detect.
+	cfg.HTTPSource = normaliseSourceOverride(os.Getenv("GEARBOX_AGENT_HTTP_SOURCE"))
+
 	return cfg, nil
 }
 
+// normaliseSourceOverride trims + lowercases an operator-supplied gear
+// name so case / whitespace differences ('HAProxy ' vs 'haproxy') don't
+// cause silent misses against Info().Name. Returns "" for an empty or
+// whitespace-only input, which downstream callers treat as "no
+// override, auto-detect".
+func normaliseSourceOverride(raw string) string {
+	return strings.ToLower(strings.TrimSpace(raw))
+}
+
 // Validate validates the configuration.
 func (c *Config) Validate() error {
 	if c.ListenAddr == "" {

gearbox-agent/internal/framework/config/config_test.go

@@ -375,3 +375,52 @@ func TestGetEnvDurationSecondsOrDefault(t *testing.T) {
 		})
 	}
 }
+
+func TestNormaliseSourceOverride(t *testing.T) {
+	cases := []struct {
+		in, want string
+	}{
+		{"", ""},
+		{"   ", ""},
+		{"haproxy", "haproxy"},
+		{"HAProxy", "haproxy"},
+		{" HAPROXY ", "haproxy"},
+		{"  Nginx  ", "nginx"},
+	}
+	for _, tc := range cases {
+		if got := normaliseSourceOverride(tc.in); got != tc.want {
+			t.Errorf("normaliseSourceOverride(%q) = %q, want %q", tc.in, got, tc.want)
+		}
+	}
+}
+
+func TestLoad_HTTPSourceOverride(t *testing.T) {
+	// t.Setenv handles save/restore automatically and prevents
+	// parallel-test interference; the old os.Setenv + t.Cleanup pattern
+	// would leak the env var if the test panicked before Cleanup ran.
+	t.Setenv("GEARBOX_AGENT_HTTP_SOURCE", " Nginx ")
+	cfg, err := Load()
+	if err != nil {
+		t.Fatalf("Load: %v", err)
+	}
+	if cfg.HTTPSource != "nginx" {
+		t.Errorf("HTTPSource = %q, want %q (trimmed + lowercased)", cfg.HTTPSource, "nginx")
+	}
+}
+
+func TestLoad_HTTPSourceDefaultsEmpty(t *testing.T) {
+	// Unset within the test; t.Setenv with an empty string only marks
+	// the env var for restoration but doesn't unset it, so explicit
+	// Unsetenv is the right tool when the test specifically needs the
+	// var absent.
+	t.Setenv("GEARBOX_AGENT_HTTP_SOURCE", "") // record original for restore
+	os.Unsetenv("GEARBOX_AGENT_HTTP_SOURCE")
+
+	cfg, err := Load()
+	if err != nil {
+		t.Fatalf("Load: %v", err)
+	}
+	if cfg.HTTPSource != "" {
+		t.Errorf("HTTPSource = %q, want empty (auto-detect)", cfg.HTTPSource)
+	}
+}

gearbox-agent/internal/framework/gear/dependencies.go

@@ -45,6 +45,14 @@ type Dependencies struct {
 
 	// CertbotTimer is the name of the certbot systemd timer.
 	CertbotTimer string
+
+	// SourceOverrides maps metric categories to an operator-chosen
+	// primary gear name, bypassing auto-detection for that category.
+	// Empty map (or missing key) means auto-detect. Sourced from
+	// per-category env vars at startup (GEARBOX_AGENT_HTTP_SOURCE for
+	// CategoryHTTPRequests, etc.). Names are pre-lowercased to match
+	// Info().Name lookups in the manager.
+	SourceOverrides map[MetricCategory]string
 }
 
 // Common event types used across plugins.