You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+66-18Lines changed: 66 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,10 +20,11 @@ remote runner script that removes itself when tracing stops or cephlens exits;
20
20
## Features
21
21
22
22
- Live cluster health, quorum, OSD counts, and IO throughput over a single SSH stream.
23
-
- Per-node readiness: connection state, OSD ids, CPU and memory percent, and microceph version.
23
+
- Per-node readiness: connection state, OSD ids, CPU and memory percent, and Ceph version/deployment.
24
24
- osdtrace eBPF latency tracing with per-OSD and per-PG breakdown of queue, BlueStore, and KV-commit latency.
25
25
- No standing agent: no permanent daemon on the nodes; the osdtrace runner script removes itself on stop, quit, or TTL expiry. (The cephtrace tracer binaries you deploy do persist under `~/.cephlens/bin/`.)
26
26
- Edit hosts and trace settings live in the TUI; changes apply to open SSH streams immediately.
27
+
- Export recorded sessions as Markdown reports with the same diagnostic rules used by the TUI.
27
28
28
29
## Requirements
29
30
@@ -218,20 +219,32 @@ check for lab use only; do not use it for production clusters.
218
219
## Quick start
219
220
220
221
```sh
221
-
cargo run -- snapshot
222
-
cargo run -- probe
223
-
cargo run -- record --count 3 --interval-secs 2
222
+
cargo run -- init-config
223
+
cargo run -- doctor
224
224
cargo run -- tui --refresh-secs 1
225
-
cargo run -- bench --host ceph-node-1 --seconds 5
226
225
```
227
226
228
-
Create a fresh config template:
227
+
## Workflows
228
+
229
+
### Preflight
230
+
231
+
Run `doctor` before a live session or lab capture. It checks SSH, passwordless
232
+
sudo, admin Ceph CLI access, `rados`, osdtrace on OSD hosts, and
233
+
kfstrace/radostrace on `client_hosts`:
229
234
230
235
```sh
231
-
cargo run -- init-config
236
+
cargo run -- doctor
232
237
```
233
238
234
-
Useful keys in the TUI (the dashboard auto-refreshes every `refresh_secs`):
239
+
### Live TUI
240
+
241
+
Start the dashboard after `doctor` passes:
242
+
243
+
```sh
244
+
cargo run -- tui --refresh-secs 1
245
+
```
246
+
247
+
The dashboard auto-refreshes every `refresh_secs`. Useful keys:
235
248
236
249
```text
237
250
p run a probe readiness check
@@ -265,33 +278,68 @@ Esc/c return to live dashboard
265
278
Config edits are written to `cephlens.toml` and applied to the live SSH streams
266
279
immediately after the edit is confirmed.
267
280
268
-
Before installing, cephlens checks the remote kernel, architecture, and
269
-
`/etc/os-release`. The automatic install path only runs when the target is
270
-
Linux, x86_64/amd64, and in the Debian/Ubuntu family. Other platforms are shown
271
-
as unsupported instead of being blindly overwritten.
272
-
273
281
The integrated trace panel can show osdtrace, kfstrace, or radostrace data.
274
282
The osdtrace view observes Ceph OSD nodes. It streams `op_r`, `op_w`, and
275
283
`subop_w` lines into the live dashboard and summarizes total, queue, and
276
284
BlueStore latency. The kfstrace and radostrace views run on `client_hosts`.
277
285
On wide terminals the trace panel appears on the right; on tall terminals it
278
286
appears below the dashboard.
287
+
Live TUI mode keeps one SSH stream open for cluster status and one stream per
288
+
host for node readiness. Each stream emits data once per second by default and
289
+
the node table shows connection state (`live`, `dial`, `retry`, `error`), OSD
290
+
ids, CPU percentage, and memory percentage.
279
291
When `trace_auto_start` is true, cephlens starts osdtrace runners as soon as the
280
292
TUI opens. The default config keeps it false so an operator explicitly starts
281
293
and stops tracing with `t`, `f`, `r`, or `a`.
282
294
If no events appear, the cluster may be idle or all observed operations may be
283
295
below the configured 1ms latency threshold. Set `trace_latency_ms = 0` to trace
284
296
all observed osdtrace ops, or run Ceph IO while the trace runners are active.
285
297
298
+
Before installing osdtrace from the TUI, cephlens checks the remote kernel,
299
+
architecture, and `/etc/os-release`. The automatic install path only runs when
300
+
the target is Linux, x86_64/amd64, and in the Debian/Ubuntu family. Other
301
+
platforms are shown as unsupported instead of being blindly overwritten.
302
+
303
+
### Lab capture
304
+
305
+
Use `lab` for a short benchmark plus trace capture. It creates a session,
306
+
records before/after snapshots, writes `bench.log`, and writes `report.md`.
307
+
The trace mode can be `none`, `osd`, or `all`:
308
+
309
+
```sh
310
+
cargo run -- lab --host ceph-node-1 --seconds 30 --trace all
311
+
```
312
+
313
+
### One-shot commands
314
+
315
+
Use these for scripts or quick checks outside the TUI:
316
+
317
+
```sh
318
+
cargo run -- snapshot
319
+
cargo run -- probe
320
+
cargo run -- record --count 3 --interval-secs 2
321
+
cargo run -- bench --host ceph-node-1 --seconds 5
322
+
```
323
+
324
+
### Sessions and reports
325
+
286
326
Live TUI sessions are recorded under `.cephlens/sessions/<timestamp>/`.
287
327
Cluster snapshots are appended to `snapshots.jsonl`, and raw trace lines are
288
328
appended as plain text to `trace-osd.log`, `trace-kfs.log`, and
289
-
`trace-rados.log`.
329
+
`trace-rados.log`. When the TUI exits after recording at least one snapshot,
330
+
cephlens also writes `.cephlens/sessions/<timestamp>/report.md`.
290
331
291
-
Live TUI mode keeps one SSH stream open for cluster status and one stream per
292
-
host for node readiness. Each stream emits data once per second by default and
293
-
the node table shows connection state (`live`, `dial`, `retry`, `error`), OSD
294
-
ids, CPU percentage, and memory percentage.
332
+
Turn a recorded session into a Markdown investigation note:
333
+
334
+
```sh
335
+
cargo run -- report .cephlens/sessions/<timestamp> --out report.md
336
+
```
337
+
338
+
Replay a recorded snapshot sequence in the TUI:
339
+
340
+
```sh
341
+
cargo run -- replay .cephlens/sessions/<timestamp>
0 commit comments