Skip to content

Commit 0384a1e

Browse files
sarg3ntclaude
andauthored
feat(#91): dashboard — multi-source metrics page (nginx/Apache/Caddy/Traefik) (#102)
* feat(#91): dashboard — multi-source metrics page (nginx/Apache/Caddy/Traefik) Closes #91. Dashboard-side companion to PR #101's agent work: DB (new source_stats table): - Aggregate rollups per (server_id, source, collected_at). Distinct table from traffic_flows because non-HAProxy sources don't emit per-IP/per-backend detail — they expose top-level counters via stub_status / mod_status / Prometheus. Adding a source column to traffic_flows would mean a forest of NULL columns for every non-HAProxy row. - Stable rollups: requests_total, response_{2xx,3xx,4xx,5xx} (where the source emits per-status-class), active_connections, plus an extra_json column for source-specific fields (nginx reading/writing/waiting; Apache worker pool; Caddy request_errors_total; Traefik entrypoints). - SaveSourceStats / GetLatestSourceStats / GetSourceStatsRange / PruneSourceStats helpers in source_stats.go. Agent client: - GetSourceStats(source) returns the agent's per-source /stats JSON as a flexible map — the four sources have different shapes; the collector normalises into the common SourceStatsSnapshot. - GetAccessLogRecent(source, statusMin, limit) hits the agent's /api/v1/access-log/{source}/recent endpoint and returns the AccessLogResponse envelope verbatim. Collector: - SourceStatsCollector polls every supported source each persistHistory tick. Sources whose gear hasn't probed Available return 503/404 from the agent — silently swallowed via isExpectedSourceMiss so the journal stays clean on HAProxy-only hosts. One source's failure doesn't skip the others. - Per-source normalisers map agent JSON → SourceStatsSnapshot: nginx requests/active + reading/writing/waiting → extra; Apache total_accesses + busy_workers + idle/cpu/uptime → extra; Caddy requests_total + request_errors_total → extra; Traefik full status-class breakdown surfaced as first-class columns. - Wired into manager.persistHistory; lifecycle-safe with the existing stats persistence. Handlers + routes: - /api/{boxID}/metrics/source/{source}/summary — per-source latest snapshot + history range for the chart cards. Gated by the agent capability for that source so empty/unknown render with reason rather than a misleading zero-data card. - /api/{boxID}/metrics/source/{source}/log-errors — Phase 5 refactor. Proxies to the agent's structured access-log endpoint rather than parsing log text dashboard-side. Accepts "haproxy" too — the agent's parser is the unification point. Existing /metrics/log-errors stays untouched for now so older dashboard clients don't break mid-rollout. UI (metrics.templ): - Four new per-source chart cards (nginx, Apache, Caddy, Traefik), each with the same data-source / data-source-card hooks the existing cards use. Hidden by default; applyCapabilities() flips them on per-source based on the agent's probe table. - "Per-Source Recent Errors" section below the existing HAProxy Error Insights panel — Phase 6's multi-block view. One block per Available non-HAProxy source via the new log-errors endpoint; built with createElement + textContent so agent-supplied fields can't inject HTML. - applyCapabilities() now iterates all sources, exposes window._availableSources so the loaders can skip absent sources rather than 404-spamming the agent. - loadSourceMetrics renders per-source Chart.js cards (nginx → stacked connection state, Apache → workers, Caddy → requests vs errors, Traefik → status-class stacked area). Phases addressed (all of remaining #91): - Phase 4 (nginx end-to-end): agent collector + dashboard card. - Phase 5 (generic access-log abstraction): dashboard log-errors now consumes the agent's structured records. - Phase 6 (multi-source Error Insights): new section. - Phase 7 (Apache / Caddy / Traefik end-to-end): same shape as nginx. - Phase 8 (cross-source aggregates): explicitly deferred per #91 — marked optional, best designed once a real second source has driven the UX. - Docker metrics (Phase 7 piece): out of scope here, lives in the feature/containers-gear branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(#91): address Copilot review findings on PR #102 Critical (charts wouldn't render): - metrics.templ: per-source cards called a nonexistent chartDefaults() helper and configured Chart.js with `type: 'time'` despite the page not loading a date adapter. Both would have thrown at render. Switch to the existing createChartOptions(0) + formatTime() string-label pattern that the other chart cards already use. No new dependencies, identical visual treatment. Functional bugs: - collector: isExpectedSourceMiss matched substring "status 503" in err.Error(), but agent.APIError.Error() returns the Message body not a status-coded string — so 503/404 from the agent surfaced as warnings every history tick instead of being silently swallowed. Use errors.As(*agent.APIError) and inspect StatusCode directly. - database: source_stats was excluded from ClearMetricsData, CleanupMetricsByAge, and CleanupMetricsBySize. Per-source rows would have grown unbounded and survived "clear metrics data". Wire into all three paths (the column is server_id, not box_id — the traffic-analysis schema predates the box_id naming). - database: GetSourceStatsRange ordered ASC LIMIT, which on long ranges with the 1-minute history interval returns the OLDEST rows in the window, hiding recent data. Switch to DESC LIMIT then reverse to chronological order before return. - metrics.templ: per-source summary URL didn't pass the selected range; every card ignored the hours-select control and used the handler's 24h default. Add hoursSelectToRange() helper + ?range= query param. - metrics.templ: chart datasets plotted monotonic counters directly (requests_total, response_*xx, etc.), so the cards showed ever-rising lines rather than per-interval rates. New counterDeltas() helper diffs successive samples (resets land as 0, matching Prometheus rate() semantics). Applied to Caddy requests + errors, Traefik status-class bands, and nginx's new requests line. - metrics.templ: nginx card was titled "Connections & Requests" but the dataset only rendered reading/writing/waiting. Add a request-rate line on a secondary y-axis so it doesn't get crushed by the connection-state stack. - handlers: log-errors source set included Traefik, but the agent has no Traefik access-log parser (Traefik logs flow through Prometheus, not the access-log endpoint). Listing Traefik always proxied to a 404 from the agent. Split into two sets — supportedSourceSummaries (nginx/apache/caddy/traefik) for the summary endpoint, supportedLogErrorSources (haproxy/nginx/ apache/caddy) for log-errors. - metrics.templ: the visibility gate for the "Per-Source Recent Errors" section used the wider chart set (anyNonHAProxy) but the renderer only walks SOURCES_WITH_ERROR_LOGS. On a Traefik-only host the section would have unhid empty. Pull the source list into two named constants (SOURCES_WITH_CHARTS, SOURCES_WITH_ERROR_LOGS) and gate the section on the renderer set. - metrics.templ: toggleFullscreen's exit branch removed `hidden` from every other chart-card, including capability-gated per-source cards. After fullscreening any card, unavailable source cards would become visible. Check data-source-card + window._availableSources during the un-hide pass; capability- hidden cards stay hidden. - agent client: GetAccessLogRecent omitted status_min when 0, causing the agent to apply its own default of 500. The handler accepts 0 as "disable filter" but couldn't actually request it. Change to pass status_min when >= 0; negative values fall back to the agent default (no caller uses that path today). Doc + a11y: - handlers: dropped a comment claiming a per-source handler test keeps the source lists in sync. No such test exists; the comment would have given a false sense of coverage. Added a cross-reference to collector.SupportedSources() instead. - metrics.templ: per-source error tables had only a tbody. Added a visually-hidden <caption> + visible <thead> with scoped column headers so screen readers announce column context. - metrics.templ: the per-source-errors refresh button was icon- only with a `title` attribute. Added aria-label for assistive technology that doesn't read title attributes reliably. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 7390cff commit 0384a1e

11 files changed

Lines changed: 1551 additions & 13 deletions

File tree

gearbox/cmd/server/main.go

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -697,6 +697,13 @@ func main() {
697697
r.Get("/{boxID}/metrics/backend/{backendName}/details", h.APIMetricsBackendDetailsHandler)
698698
r.Get("/{boxID}/metrics/log-errors", h.APIMetricsLogErrorsHandler)
699699

700+
// Per-source metrics endpoints (issue #91 phases 4/5/7).
701+
// {source} ∈ {nginx, apache, caddy, traefik} for summary;
702+
// the log-errors variant also accepts "haproxy" because
703+
// Phase 5 unifies log parsing across every source.
704+
r.Get("/{boxID}/metrics/source/{source}/summary", h.APIMetricsSourceSummaryHandler)
705+
r.Get("/{boxID}/metrics/source/{source}/log-errors", h.APIMetricsSourceLogErrorsHandler)
706+
700707
// Per-box capability manifest — exposes which agent gears probed
701708
// available so the metrics gear (and future source-aware UI) can
702709
// hide cards/KPIs that don't apply to this host.

gearbox/internal/framework/agent/client.go

Lines changed: 107 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -276,8 +276,6 @@ func (c *Client) doRequestWithBody(method, path string, reqBody interface{}) ([]
276276
return c.doRequestWithBodyAndQuery(method, path, reqBody, nil)
277277
}
278278

279-
280-
281279
// doRequestWithBodyAndQuery performs an HTTP request with a JSON body and query parameters.
282280
func (c *Client) doRequestWithBodyAndQuery(method, path string, reqBody interface{}, query url.Values) ([]byte, error) {
283281
fullURL := c.baseURL + path
@@ -1822,3 +1820,110 @@ func (c *Client) ConfigureUnattended(enabled, autoReboot bool) (*UnattendedConfi
18221820

18231821
return &resp, nil
18241822
}
1823+
1824+
// ==============================================================
1825+
// Source-aware metrics endpoints (issue #91 phases 4 + 5 + 7).
1826+
//
1827+
// These mirror the agent's /api/v1/{nginx,apache,caddy,traefik}/stats
1828+
// + /api/v1/access-log/{source}/recent shape. The dashboard collector
1829+
// calls GetSourceStats once per available source per scrape; the
1830+
// metrics-page handler calls GetAccessLogRecent when the Error
1831+
// Insights panel asks for a different source. The agent endpoints
1832+
// landed in PR #100 / #101; see gearbox-agent/docs/source-detection.md
1833+
// for the per-source surface they expose.
1834+
// ==============================================================
1835+
1836+
// SourceStats is the un-typed JSON payload the agent's /api/v1/{src}/stats
1837+
// endpoints return. Each source's Stats struct shape differs (nginx has
1838+
// active/reading/writing/waiting; Traefik has per-status-class counters;
1839+
// etc.), so the client surface keeps the payload as a flexible map and
1840+
// lets the collector normalise it into the database's SourceStatsSnapshot
1841+
// shape. This avoids re-declaring four near-identical Go types whose only
1842+
// real purpose is JSON unmarshaling.
1843+
type SourceStats map[string]any
1844+
1845+
// GetSourceStats fetches the latest stats snapshot for one source
1846+
// (nginx / apache / caddy / traefik). Returns 503 from the agent
1847+
// when the gear hasn't completed its first scrape yet; we surface
1848+
// that as an error so the collector can log + skip without
1849+
// persisting a misleading zero row.
1850+
//
1851+
// `source` must be one of the four supported identifiers; the
1852+
// agent will return 404 for anything else and we surface that too.
1853+
func (c *Client) GetSourceStats(source string) (SourceStats, error) {
1854+
body, err := c.doRequest("GET", "/api/v1/"+source+"/stats", nil)
1855+
if err != nil {
1856+
return nil, err
1857+
}
1858+
var out SourceStats
1859+
if err := json.Unmarshal(body, &out); err != nil {
1860+
return nil, fmt.Errorf("failed to parse %s stats response: %w", source, err)
1861+
}
1862+
return out, nil
1863+
}
1864+
1865+
// AccessLogRecord mirrors the agent's accesslog.Record shape — the
1866+
// dashboard renders these directly in the Error Insights panel, so
1867+
// the JSON keys here have to track the agent's. Keep alphabetical
1868+
// by field name within each grouping so adding a new field is a
1869+
// trivial inspection.
1870+
type AccessLogRecord struct {
1871+
Profile string `json:"profile"`
1872+
Timestamp string `json:"timestamp,omitempty"`
1873+
TimestampRaw string `json:"timestamp_raw,omitempty"`
1874+
SourceIP string `json:"source_ip,omitempty"`
1875+
Method string `json:"method,omitempty"`
1876+
Path string `json:"path,omitempty"`
1877+
Host string `json:"host,omitempty"`
1878+
StatusCode int `json:"status_code"`
1879+
BytesSent int64 `json:"bytes_sent,omitempty"`
1880+
DurationMs float64 `json:"duration_ms,omitempty"`
1881+
Backend string `json:"backend,omitempty"`
1882+
Server string `json:"server,omitempty"`
1883+
UserAgent string `json:"user_agent,omitempty"`
1884+
Referer string `json:"referer,omitempty"`
1885+
Raw string `json:"raw"`
1886+
}
1887+
1888+
// AccessLogResponse is the envelope the agent's
1889+
// /api/v1/access-log/{source}/recent endpoint returns. Available=false
1890+
// + a Reason populated means the host has no readable log for this
1891+
// source — the dashboard renders that as a "logs unavailable" hint
1892+
// rather than an empty panel.
1893+
type AccessLogResponse struct {
1894+
Source string `json:"source"`
1895+
Profile string `json:"profile"`
1896+
Path string `json:"path,omitempty"`
1897+
Available bool `json:"available"`
1898+
Reason string `json:"reason,omitempty"`
1899+
MatchCount int `json:"match_count"`
1900+
Records []AccessLogRecord `json:"records"`
1901+
}
1902+
1903+
// GetAccessLogRecent fetches recent parsed log records from the
1904+
// agent's access-log endpoint. statusMin = 0 is a valid value
1905+
// meaning "disable filtering" — the agent treats it that way (see
1906+
// gearbox-agent's access-log handler). To distinguish "caller
1907+
// supplied 0 explicitly" from "caller wants the agent default of
1908+
// 500" we treat any non-negative value as caller-supplied and pass
1909+
// it through; a negative value (e.g. -1) is the way to fall back
1910+
// to the agent default. limit > 0 follows the same explicit/default
1911+
// split — 0 is server-default, positive is explicit.
1912+
func (c *Client) GetAccessLogRecent(source string, statusMin, limit int) (*AccessLogResponse, error) {
1913+
q := url.Values{}
1914+
if statusMin >= 0 {
1915+
q.Set("status_min", fmt.Sprintf("%d", statusMin))
1916+
}
1917+
if limit > 0 {
1918+
q.Set("limit", fmt.Sprintf("%d", limit))
1919+
}
1920+
body, err := c.doRequest("GET", "/api/v1/access-log/"+source+"/recent", q)
1921+
if err != nil {
1922+
return nil, err
1923+
}
1924+
var resp AccessLogResponse
1925+
if err := json.Unmarshal(body, &resp); err != nil {
1926+
return nil, fmt.Errorf("failed to parse access-log response: %w", err)
1927+
}
1928+
return &resp, nil
1929+
}

gearbox/internal/framework/collector/manager.go

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,8 @@ type Manager struct {
3939
metadataCollector MetadataCollectorInterface
4040
systemCollector SystemCollectorInterface
4141
logCollector LogCollectorInterface
42-
agentClient *agent.Client // Direct access to agent client for security operations
42+
sourceCollector *SourceStatsCollector // per-source nginx/apache/caddy/traefik
43+
agentClient *agent.Client // Direct access to agent client for security operations
4344
logger *slog.Logger
4445
db *database.DB
4546
stopCh chan struct{}
@@ -60,6 +61,7 @@ func NewManager(
6061
metadataCollector := NewAgentMetadataCollector(agentClient)
6162
systemCollector := NewAgentSystemCollector(agentClient)
6263
logCollector := NewAgentLogCollector(agentClient)
64+
sourceCollector := NewSourceStatsCollector(serverID, agentClient, db, logger)
6365

6466
return &Manager{
6567
serverID: serverID,
@@ -68,6 +70,7 @@ func NewManager(
6870
metadataCollector: metadataCollector,
6971
systemCollector: systemCollector,
7072
logCollector: logCollector,
73+
sourceCollector: sourceCollector,
7174
agentClient: agentClient,
7275
logger: logger,
7376
db: db,
@@ -245,6 +248,14 @@ func (m *Manager) persistHistory() {
245248
"error", err)
246249
}
247250
}
251+
252+
// Scrape and save per-source metrics (nginx, apache, caddy,
253+
// traefik). The collector silently skips sources whose gears
254+
// haven't probed Available on this host, so this is a no-op on
255+
// hosts that only run HAProxy.
256+
if m.sourceCollector != nil {
257+
m.sourceCollector.Run()
258+
}
248259
}
249260

250261
// collectStats fetches and caches HAProxy stats.

0 commit comments

Comments
 (0)