feat(scan): redesigned, trustworthy scan UX — cards, full distribution, OOM-safe memory, concurrency#16
Merged
Conversation
…D + rich --explain
Real-cluster feedback: with no per-row detail you couldn't tell WHY a reduction
was recommended; with the dense distribution under every row the table was a wall
of text. This makes every row self-explanatory while staying clean.
Table (clean, one line per workload):
WORKLOAD REQUESTED USES PROPOSED SAVE CONF
neevai-rackbank-console 2000m/4Gi 13m/147Mi 15m/170Mi $60/mo ▒ low
Reads left-to-right: asks 2000m/4Gi, actually uses 13m/147Mi → propose 15m/170Mi.
USES is the p95 cpu / peak mem the new request is sized to. No per-row evidence
wall; a footer points to --explain.
--explain gains a "why this recommendation" block: requested vs the full observed
distribution (avg/p95/p99/peak for cpu & mem) vs proposed, plus a verdict
("~133× over-allocated on cpu"), then savings/confidence/tier/derivation.
Plumbing to support it:
- model.Percentiles gains Avg + P99; model.Recommendation carries the observed Usage.
- histogram.Mean(); operator records avg/p99 (UsageProfile CRD MetricHistory gains
avg/p99 — additive, regenerated via controller-gen; needs operator redeploy to
populate, snapshot/Prometheus populate immediately).
- Prometheus adds avg_over_time + quantile_over_time(0.99); snapshot sets all stats
to its single reading.
- robust display: an unpopulated stat renders "—", never a misleading "0m".
Coverage: report 92.6%, scan 85.6%, histogram 91.7%, model 100%, operator 84.8%.
Full build/vet/fmt/test/lint green.
…on-explain # Conflicts: # internal/report/report.go
… USES header, per-row samples Real-cluster feedback: the table showed a USES number with no indication of what it was (p95? peak? avg?) or how much data backed it. Now it's self-documenting: - banner: '... data: 0 (kubetidy operator) · observed over 48m' (the time span) - header: 'USES (cpu p95 · mem peak)' names exactly what the proposal is sized to - CONF column: '▒ low 31% · 31 smp' — the sample count behind each recommendation Legend trimmed accordingly; full avg/p95/p99/peak still under --explain.
… tighter rows Per real-cluster feedback: window and sample count are uniform across a scan, so repeating '· 202 smp' on all 102 rows was clutter that widened the table. - banner: '... data: 0 (kubetidy operator) · 5h history, ~202 samples/workload' (stated once; median sample count is robust to multi-pod outliers) - CONF: back to a compact '▒ low 34%' (no per-row samples) - USES header shortened (defined in the legend) to reclaim width - exact per-workload samples remain in --explain The table now carries only what varies per workload; provenance is in the header.
Two changes from real-cluster feedback.
1. UX — a flat table can't hold the full distribution (avg/p95/p99/peak for cpu
AND mem) plus the change, so render one CARD per recommendation. The workload
name goes on its own line (fixing long-name width), then an aligned block:
▸ neevai-rackbank-console · Deployment/neevai-system save $60/mo · ▒ low 34%
avg p95 p99 peak request → proposed
cpu 1m 2m 2m 2m 2000m → 10m
mem 140Mi 148Mi 149Mi 149Mi 4Gi → 311Mi
basis 5h history · 201 samples · memory kept conservative (short history — extra OOM safety)
Uniform provenance (window, typical samples) stays in the banner; cards show
top-N with a "showing top N of M" note. --explain keeps the full derivation.
2. Safety — memory sized to peak+15% over a SHORT window can OOM (the window may
not have seen the true peak). Memory headroom now scales inversely with data
maturity: a young history keeps a large cushion (e.g. peak×2), shrinking to the
base only once a representative window of samples backs it. CPU stays lean (it
only throttles). New Policy.MemoryImmatureSafety (default +100%); maturity calc
shared with the confidence model.
Coverage: report 92%+, rightsizer 90%+, model 100%. Full gate + lint green.
Per feedback, the borderless space-aligned block read ambiguously. Each card's usage block is now a proper box-drawing table (grid lines), for both the operator tier (avg/p95/p99/peak) and snapshot tier (now), e.g.: ┌─────┬───────┬───────┬───────┬───────┬────────────────────┐ │ │ avg │ p95 │ p99 │ peak │ request → proposed │ ├─────┼───────┼───────┼───────┼───────┼────────────────────┤ │ cpu │ 1m │ 2m │ 2m │ 2m │ 2000m → 10m │ │ mem │ 140Mi │ 148Mi │ 149Mi │ 149Mi │ 4Gi → 311Mi │ └─────┴───────┴───────┴───────┴───────┴────────────────────┘ Column widths measured in runes so the ·/→ glyphs don't skew alignment.
…lumns
Real-cluster feedback on speed, noise, and empty columns.
- Concurrency: workloads are analyzed in a bounded worker pool (was a sequential
loop), so a large cluster scans ~10× faster and the spinner shows live X/N
progress instead of sitting frozen. (Progress was also never being called.)
- Quiet output: raise client-go QPS/Burst (50/100) so concurrent read-only
queries aren't client-side throttled, and silence klog so internal k8s logs
(e.g. rate-limit warnings) never pollute the CLI output.
- Adaptive columns: the avg/p99 distribution columns are shown only when there's
data for them — an older operator that records just p95/peak now renders a clean
table instead of a column of "—". They appear once the new operator/Prometheus
provides them.
- --explain reuses the same bordered table + a clean key/value block ("how it's
sized"), consistent with the scan cards.
- docs: README rightsizing policy + trust section updated; CHANGELOG v0.1.2 entry
(cards, distribution, OOM-safe memory, concurrency); demo workloads are now tiny
load generators (real cpu+mem) so the demo shows a real distribution.
Full gate + race + lint green.
…sh demo GIF - Header: short banner (`kubetidy · <context>`) with the data source moved to an aligned hero "Source" line using a friendly tier name (e.g. "kubetidy operator (Tier 0, no Prometheus)") instead of the bare "0 (kubetidy operator)". Waste line now also carries the workload count. - Adaptive columns: avg/p99 columns appear only when there's data, so an older operator (p95/peak only) renders a clean table, not a column of "—". - Geeky progress: while the (now concurrent) scan runs, show a live bar + count + a phrase that advances with progress — "⟦▰▰▰▰▰▱▱▱▱▱⟧ 51/102 · pricing the waste…". - Demo GIF re-recorded against the new card UX with real cpu+mem load generators. Full gate + lint green.
mayur-tolexo
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A end-to-end overhaul of the scan experience, driven by live real-cluster feedback. Every recommendation is now self-explanatory, memory recommendations are OOM-safe, and the scan is fast and quiet.
Output: a card per recommendation
avg / p95 / p99 / peakfor CPU and memory — in a bordered table, beside therequest → proposedchange. The long workload name sits on its own line.--explainreuses the same bordered table + a clean key/value block ("how it's sized").avg/p99show only when there's data, so an older operator (p95/peak only) renders a clean table instead of a column of—.Memory is OOM-safe
Memory is the dangerous resource and a short window can miss the true peak. The memory headroom now scales inversely with data maturity: a young history keeps a large cushion (up to ~peak×2), tightening to peak+15% only once a representative window of samples backs it. CPU stays lean (it only throttles). New
Policy.MemoryImmatureSafety; maturity shared with the confidence model.Honest confidence
Data-maturity-gated score shown as a band + % (
▒ low 34%); a fresh operator reads low, not a false 85%. Thedata:tier reflects what actually backed the findings.Fast & quiet
⟦▰▰▰▰▰▱▱▱▱▱⟧ 51/102 · pricing the waste in dollars….Plumbing
model.PercentilesgainsAvg+P99;model.Recommendationcarries the observedUsage. Providers populate them — Prometheus (avg_over_time+quantile_over_time(0.99)), operator histogram (Mean()+ p99;UsageProfileCRDMetricHistorygains additiveavg/p99, regenerated — operator redeploy needed to populate), snapshot (single reading). Sub-hour windows render as12m.Docs & demo
README (rightsizing policy + trust),
CHANGELOG.md(v0.1.2),ARCHITECTURE.mdupdated. Demo workloads are now tiny load generators (real cpu+mem) and the README GIF is re-recorded against the new UX.Tests
report ~92%, rightsizer ~91%, scan ~85%, model 100%; race-tested concurrency; full
go build/vet/gofmt/test ./...+golangci-lint(0 issues) green.Builds on released v0.1.1 — candidate for v0.1.2.