Commit 0384a1e
* feat(#91): dashboard — multi-source metrics page (nginx/Apache/Caddy/Traefik)
Closes #91. Dashboard-side companion to PR #101's agent work:
DB (new source_stats table):
- Aggregate rollups per (server_id, source, collected_at). Distinct
table from traffic_flows because non-HAProxy sources don't emit
per-IP/per-backend detail — they expose top-level counters via
stub_status / mod_status / Prometheus. Adding a source column to
traffic_flows would mean a forest of NULL columns for every
non-HAProxy row.
- Stable rollups: requests_total, response_{2xx,3xx,4xx,5xx} (where
the source emits per-status-class), active_connections, plus an
extra_json column for source-specific fields (nginx
reading/writing/waiting; Apache worker pool; Caddy
request_errors_total; Traefik entrypoints).
- SaveSourceStats / GetLatestSourceStats / GetSourceStatsRange /
PruneSourceStats helpers in source_stats.go.
Agent client:
- GetSourceStats(source) returns the agent's per-source /stats JSON
as a flexible map — the four sources have different shapes; the
collector normalises into the common SourceStatsSnapshot.
- GetAccessLogRecent(source, statusMin, limit) hits the agent's
/api/v1/access-log/{source}/recent endpoint and returns the
AccessLogResponse envelope verbatim.
Collector:
- SourceStatsCollector polls every supported source each
persistHistory tick. Sources whose gear hasn't probed Available
return 503/404 from the agent — silently swallowed via
isExpectedSourceMiss so the journal stays clean on HAProxy-only
hosts. One source's failure doesn't skip the others.
- Per-source normalisers map agent JSON → SourceStatsSnapshot:
nginx requests/active + reading/writing/waiting → extra; Apache
total_accesses + busy_workers + idle/cpu/uptime → extra; Caddy
requests_total + request_errors_total → extra; Traefik full
status-class breakdown surfaced as first-class columns.
- Wired into manager.persistHistory; lifecycle-safe with the
existing stats persistence.
Handlers + routes:
- /api/{boxID}/metrics/source/{source}/summary — per-source latest
snapshot + history range for the chart cards. Gated by the agent
capability for that source so empty/unknown render with reason
rather than a misleading zero-data card.
- /api/{boxID}/metrics/source/{source}/log-errors — Phase 5
refactor. Proxies to the agent's structured access-log endpoint
rather than parsing log text dashboard-side. Accepts "haproxy"
too — the agent's parser is the unification point. Existing
/metrics/log-errors stays untouched for now so older dashboard
clients don't break mid-rollout.
UI (metrics.templ):
- Four new per-source chart cards (nginx, Apache, Caddy, Traefik),
each with the same data-source / data-source-card hooks the
existing cards use. Hidden by default; applyCapabilities() flips
them on per-source based on the agent's probe table.
- "Per-Source Recent Errors" section below the existing HAProxy
Error Insights panel — Phase 6's multi-block view. One block per
Available non-HAProxy source via the new log-errors endpoint;
built with createElement + textContent so agent-supplied fields
can't inject HTML.
- applyCapabilities() now iterates all sources, exposes
window._availableSources so the loaders can skip absent sources
rather than 404-spamming the agent.
- loadSourceMetrics renders per-source Chart.js cards
(nginx → stacked connection state, Apache → workers, Caddy →
requests vs errors, Traefik → status-class stacked area).
Phases addressed (all of remaining #91):
- Phase 4 (nginx end-to-end): agent collector + dashboard card.
- Phase 5 (generic access-log abstraction): dashboard log-errors
now consumes the agent's structured records.
- Phase 6 (multi-source Error Insights): new section.
- Phase 7 (Apache / Caddy / Traefik end-to-end): same shape as nginx.
- Phase 8 (cross-source aggregates): explicitly deferred per #91 —
marked optional, best designed once a real second source has
driven the UX.
- Docker metrics (Phase 7 piece): out of scope here, lives in the
feature/containers-gear branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(#91): address Copilot review findings on PR #102
Critical (charts wouldn't render):
- metrics.templ: per-source cards called a nonexistent chartDefaults()
helper and configured Chart.js with `type: 'time'` despite the page
not loading a date adapter. Both would have thrown at render. Switch
to the existing createChartOptions(0) + formatTime() string-label
pattern that the other chart cards already use. No new dependencies,
identical visual treatment.
Functional bugs:
- collector: isExpectedSourceMiss matched substring "status 503" in
err.Error(), but agent.APIError.Error() returns the Message body
not a status-coded string — so 503/404 from the agent surfaced as
warnings every history tick instead of being silently swallowed.
Use errors.As(*agent.APIError) and inspect StatusCode directly.
- database: source_stats was excluded from ClearMetricsData,
CleanupMetricsByAge, and CleanupMetricsBySize. Per-source rows
would have grown unbounded and survived "clear metrics data".
Wire into all three paths (the column is server_id, not box_id —
the traffic-analysis schema predates the box_id naming).
- database: GetSourceStatsRange ordered ASC LIMIT, which on long
ranges with the 1-minute history interval returns the OLDEST
rows in the window, hiding recent data. Switch to DESC LIMIT
then reverse to chronological order before return.
- metrics.templ: per-source summary URL didn't pass the selected
range; every card ignored the hours-select control and used the
handler's 24h default. Add hoursSelectToRange() helper +
?range= query param.
- metrics.templ: chart datasets plotted monotonic counters
directly (requests_total, response_*xx, etc.), so the cards
showed ever-rising lines rather than per-interval rates. New
counterDeltas() helper diffs successive samples (resets land as
0, matching Prometheus rate() semantics). Applied to Caddy
requests + errors, Traefik status-class bands, and nginx's new
requests line.
- metrics.templ: nginx card was titled "Connections & Requests"
but the dataset only rendered reading/writing/waiting. Add a
request-rate line on a secondary y-axis so it doesn't get
crushed by the connection-state stack.
- handlers: log-errors source set included Traefik, but the agent
has no Traefik access-log parser (Traefik logs flow through
Prometheus, not the access-log endpoint). Listing Traefik
always proxied to a 404 from the agent. Split into two sets —
supportedSourceSummaries (nginx/apache/caddy/traefik) for the
summary endpoint, supportedLogErrorSources (haproxy/nginx/
apache/caddy) for log-errors.
- metrics.templ: the visibility gate for the "Per-Source Recent
Errors" section used the wider chart set (anyNonHAProxy) but
the renderer only walks SOURCES_WITH_ERROR_LOGS. On a
Traefik-only host the section would have unhid empty. Pull the
source list into two named constants (SOURCES_WITH_CHARTS,
SOURCES_WITH_ERROR_LOGS) and gate the section on the renderer
set.
- metrics.templ: toggleFullscreen's exit branch removed `hidden`
from every other chart-card, including capability-gated
per-source cards. After fullscreening any card, unavailable
source cards would become visible. Check data-source-card +
window._availableSources during the un-hide pass; capability-
hidden cards stay hidden.
- agent client: GetAccessLogRecent omitted status_min when 0,
causing the agent to apply its own default of 500. The handler
accepts 0 as "disable filter" but couldn't actually request it.
Change to pass status_min when >= 0; negative values fall back
to the agent default (no caller uses that path today).
Doc + a11y:
- handlers: dropped a comment claiming a per-source handler test
keeps the source lists in sync. No such test exists; the
comment would have given a false sense of coverage. Added a
cross-reference to collector.SupportedSources() instead.
- metrics.templ: per-source error tables had only a tbody. Added
a visually-hidden <caption> + visible <thead> with scoped column
headers so screen readers announce column context.
- metrics.templ: the per-source-errors refresh button was icon-
only with a `title` attribute. Added aria-label for assistive
technology that doesn't read title attributes reliably.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 7390cff commit 0384a1e
11 files changed
Lines changed: 1551 additions & 13 deletions
File tree
- gearbox
- cmd/server
- internal/framework
- agent
- collector
- database
- handler
- templates/pages
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
697 | 697 | | |
698 | 698 | | |
699 | 699 | | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
700 | 707 | | |
701 | 708 | | |
702 | 709 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
276 | 276 | | |
277 | 277 | | |
278 | 278 | | |
279 | | - | |
280 | | - | |
281 | 279 | | |
282 | 280 | | |
283 | 281 | | |
| |||
1822 | 1820 | | |
1823 | 1821 | | |
1824 | 1822 | | |
| 1823 | + | |
| 1824 | + | |
| 1825 | + | |
| 1826 | + | |
| 1827 | + | |
| 1828 | + | |
| 1829 | + | |
| 1830 | + | |
| 1831 | + | |
| 1832 | + | |
| 1833 | + | |
| 1834 | + | |
| 1835 | + | |
| 1836 | + | |
| 1837 | + | |
| 1838 | + | |
| 1839 | + | |
| 1840 | + | |
| 1841 | + | |
| 1842 | + | |
| 1843 | + | |
| 1844 | + | |
| 1845 | + | |
| 1846 | + | |
| 1847 | + | |
| 1848 | + | |
| 1849 | + | |
| 1850 | + | |
| 1851 | + | |
| 1852 | + | |
| 1853 | + | |
| 1854 | + | |
| 1855 | + | |
| 1856 | + | |
| 1857 | + | |
| 1858 | + | |
| 1859 | + | |
| 1860 | + | |
| 1861 | + | |
| 1862 | + | |
| 1863 | + | |
| 1864 | + | |
| 1865 | + | |
| 1866 | + | |
| 1867 | + | |
| 1868 | + | |
| 1869 | + | |
| 1870 | + | |
| 1871 | + | |
| 1872 | + | |
| 1873 | + | |
| 1874 | + | |
| 1875 | + | |
| 1876 | + | |
| 1877 | + | |
| 1878 | + | |
| 1879 | + | |
| 1880 | + | |
| 1881 | + | |
| 1882 | + | |
| 1883 | + | |
| 1884 | + | |
| 1885 | + | |
| 1886 | + | |
| 1887 | + | |
| 1888 | + | |
| 1889 | + | |
| 1890 | + | |
| 1891 | + | |
| 1892 | + | |
| 1893 | + | |
| 1894 | + | |
| 1895 | + | |
| 1896 | + | |
| 1897 | + | |
| 1898 | + | |
| 1899 | + | |
| 1900 | + | |
| 1901 | + | |
| 1902 | + | |
| 1903 | + | |
| 1904 | + | |
| 1905 | + | |
| 1906 | + | |
| 1907 | + | |
| 1908 | + | |
| 1909 | + | |
| 1910 | + | |
| 1911 | + | |
| 1912 | + | |
| 1913 | + | |
| 1914 | + | |
| 1915 | + | |
| 1916 | + | |
| 1917 | + | |
| 1918 | + | |
| 1919 | + | |
| 1920 | + | |
| 1921 | + | |
| 1922 | + | |
| 1923 | + | |
| 1924 | + | |
| 1925 | + | |
| 1926 | + | |
| 1927 | + | |
| 1928 | + | |
| 1929 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
| 42 | + | |
| 43 | + | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| |||
60 | 61 | | |
61 | 62 | | |
62 | 63 | | |
| 64 | + | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| |||
68 | 70 | | |
69 | 71 | | |
70 | 72 | | |
| 73 | + | |
71 | 74 | | |
72 | 75 | | |
73 | 76 | | |
| |||
245 | 248 | | |
246 | 249 | | |
247 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
248 | 259 | | |
249 | 260 | | |
250 | 261 | | |
| |||
0 commit comments