Skip to content

Commit 002e11d

Browse files
committed
docs: document the Phase G ops suite
- Add docs/OPS.md covering audit log, 2FA/group gating, telemetry, schedule, and tunnel (config + behavior for each). - Update README roadmap row and command table; flip schedule/tunnel from "(Phase G)" placeholders to their real subcommands/usage. - Add the Phase G ops-suite entry to CHANGELOG.
1 parent 36ea455 commit 002e11d

3 files changed

Lines changed: 126 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- Phase G (ops) — **operational suite** (audit, 2FA gating, telemetry, schedule, tunnel):
13+
- **Audit log** (`internal/audit`): an append-only JSONL trail of destructive ops (backup/restore/sync/prune) recording who/what/when/outcome, via a single `guardedOp` interception seam wired into the verbs. Off by default; enable with the `audit:` config block. The same seam is reused by 2FA gating and telemetry (no per-feature re-wrapping).
14+
- **2FA / group gating** (`internal/twofactor`): a profile's group can require a typed-name confirmation (`confirm_destructive`) and/or a TOTP code (`require_2fa`) before a destructive op runs. Stdlib RFC 6238 TOTP (HMAC-SHA1, 30s, 6 digits, ±1-step skew), no new dependency; the group's `totp_secret` is a secret-ref. `require_2fa` with no resolvable secret fails closed.
15+
- **Telemetry** (`internal/telemetry`): opt-in aggregate per-op counts and error tallies, flushed as JSON. Records the op name and outcome only — never profile, actor, target, or data. Off by default (`telemetry:` config block); composed onto the audit seam via `audit.Multi`.
16+
- **`siphon schedule`**: manage cron-scheduled recurring backups. siphon maintains a delimited, siphon-owned block in the user's crontab that invokes `siphon backup <profile>` — the host cron runs the jobs (no daemon). `schedule add <profile> --cron`, `list`, `remove`.
17+
- **`siphon tunnel <profile>`**: open a foreground `ssh -L` local-forward to a profile's database through a configured `tunnel.bastion`, using the system ssh client; held open until Ctrl-C.
18+
1219
- Phase G (ops) — **retention & lifecycle** (second G cycle):
1320
- Chain-aware retention engine in `internal/dumps` (`GroupChains` + `Plan`, pure and DB/clock-free): the catalog is grouped into restorable **chains** (a base plus its incrementals), and retention keeps or prunes each chain as a unit, so an incremental is never orphaned from its base. A chain's age is its newest member, so an actively-appended chain is never pruned mid-life.
1421
- Three composable policy rules with **union** semantics (a chain is kept if it satisfies any active rule, so adding a rule only ever protects more): `keep_last: N`, `max_age: <duration>`, and grandfather-father-son `gfs: {daily, weekly, monthly}`. An all-zero/omitted policy keeps everything — prune is a no-op unless a rule is explicitly configured.

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
5555
| **D** — Driver hardening | Shared cross-driver test harness (`RunDriverSuite`), capability-gating helper (`RequireCapability`), Postgres connection-probe retry, and a `docs/DRIVERS.md` contributor guide | ✅ Complete |
5656
| **E** — MySQL + MariaDB | Both drivers via a shared `_mysqlcommon` package (shared `Conn`, `mysqldump`/`mariadb-dump` backup, client-pipe restore), exercised by the Phase D `RunDriverSuite` harness | ✅ Complete |
5757
| **F** — Advanced transfer | All four advanced-transfer modes work end-to-end: bounded-buffer streaming sync; **incremental** backup/restore (`backup --incremental --base <id>` captures a bounded change set via Postgres logical decoding / MySQL-MariaDB binlog, `restore` replays the base→incremental chain, Postgres orphan-slot sweep); **cross-engine** sync (`sync --cross-engine` — typed `SchemaInspector` introspection → canonical type-mapping, e.g. Postgres → MySQL); and **CDC** (`siphon cdc` / `sync --continuous` — unbounded change streaming with snapshot→stream handoff, resumable, same- and cross-engine). Live DB paths are integration-tested in CI — see [docs/INCREMENTAL.md](docs/INCREMENTAL.md), [docs/CROSS_ENGINE.md](docs/CROSS_ENGINE.md), [docs/CDC.md](docs/CDC.md) | ✅ Complete |
58-
| **G** — Ops features | **Cloud storage** (✅): the dump catalog can live in an S3 / S3-compatible bucket via a pluggable `storage.Store` backend (local + S3; GCS/Azure are a fast-follow), `storage:` config block, SHA-256 integrity end-to-end — see [docs/STORAGE.md](docs/STORAGE.md). **Retention** (✅): chain-aware pruning (`siphon prune`) with keep-last-N / max-age / GFS rules, per-profile `retention:` config, dry-run by default — never orphans an incremental's base; see [docs/RETENTION.md](docs/RETENTION.md). Secret backends, profile groups + 2FA, team mode, audit log, and telemetry follow in later G cycles. | 🟡 In progress |
58+
| **G** — Ops features | **Cloud storage** (✅): the dump catalog can live in an S3 / S3-compatible bucket via a pluggable `storage.Store` backend (local + S3; GCS/Azure are a fast-follow), `storage:` config block, SHA-256 integrity end-to-end — see [docs/STORAGE.md](docs/STORAGE.md). **Retention** (✅): chain-aware pruning (`siphon dumps prune`) with keep-last-N / max-age / GFS rules, per-profile `retention:` config; see [docs/RETENTION.md](docs/RETENTION.md). **Ops suite** (✅): an append-only **audit log** of destructive ops (`audit:` config), **2FA/group gating** (a profile group can require a typed confirmation and/or TOTP before destructive ops), opt-in aggregate **telemetry** (`telemetry:` config), **`siphon schedule`** (manages recurring backups in your crontab), and **`siphon tunnel`** (SSH local-forward to a DB via a bastion). Multi-backend secrets are the remaining G item; see [docs/OPS.md](docs/OPS.md). | 🟡 In progress |
5959
| **H** — Distribution | GoReleaser, Homebrew tap, Scoop bucket, install script, docs site | ⏳ Planned |
6060

6161
## Requirements
@@ -134,8 +134,8 @@ Exit codes follow a POSIX-friendly taxonomy (`0` ok, `1` user error, `2` system
134134
| `siphon dumps list\|inspect\|prune` | List, inspect, or prune saved dumps |
135135
| `siphon profile add\|list\|show\|rm` | Manage named connection profiles |
136136
| `siphon config path\|edit` | Show or edit the config file |
137-
| `siphon schedule` | Cron-managed recurring backups _(Phase G)_ |
138-
| `siphon tunnel` | SSH tunnel helper _(Phase G)_ |
137+
| `siphon schedule add\|list\|remove` | Manage cron-scheduled recurring backups |
138+
| `siphon tunnel <profile>` | Open an SSH tunnel to a DB via its bastion |
139139
| `siphon` _(bare)_ | Launch the interactive multi-panel dashboard |
140140

141141
Run `siphon <command> --help` for full flags.

docs/OPS.md

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Operational features
2+
3+
Phase G's ops suite adds five operational capabilities: an audit log, 2FA /
4+
group gating, opt-in telemetry, scheduled backups, and an SSH tunnel helper.
5+
The first three share one interception point in the app layer (`guardedOp`),
6+
which wraps the destructive verbs (backup, restore, sync, prune) exactly once.
7+
8+
## Table of contents
9+
10+
- [Audit log](#audit-log)
11+
- [2FA & group gating](#2fa--group-gating)
12+
- [Telemetry](#telemetry)
13+
- [Scheduled backups](#scheduled-backups)
14+
- [SSH tunnel](#ssh-tunnel)
15+
16+
## Audit log
17+
18+
An append-only JSONL trail of destructive operations — who ran what, against
19+
which profile, when, and the outcome. Off by default.
20+
21+
```yaml
22+
audit:
23+
enabled: true
24+
path: ~/.local/state/siphon/audit.log # optional; this is the default
25+
```
26+
27+
Each line records `time`, `op`, `profile`, `target`, `actor` (OS user),
28+
`outcome` (`ok`/`error`), `error`, and `duration_ms`. Audit writes are
29+
best-effort: a failure to write the log never fails the operation it records.
30+
31+
## 2FA & group gating
32+
33+
A profile belongs to a **group**; a group can require a second deliberate step
34+
before any destructive op on its profiles:
35+
36+
```yaml
37+
groups:
38+
critical:
39+
confirm_destructive: true # operator must retype the profile name
40+
require_2fa: true # operator must enter a current TOTP code
41+
totp_secret: env:SIPHON_PROD_TOTP # base32 RFC-6238 secret (a secret-ref)
42+
43+
profiles:
44+
prod:
45+
driver: postgres
46+
group: critical
47+
# ...
48+
```
49+
50+
`confirm_destructive` prompts the operator to retype the profile name;
51+
`require_2fa` prompts for a 6-digit TOTP verified (with ±1 step skew) against the
52+
group's `totp_secret` — the same code your authenticator app shows. The check
53+
runs **before** the operation, so a failed confirmation aborts before any
54+
destructive work. `require_2fa` with no resolvable secret fails closed. The TOTP
55+
secret is a secret-ref, so the plaintext never lives in config.
56+
57+
This is offline by design — siphon is a local CLI, so "2FA" means a TOTP
58+
(standard, no network) rather than a push notification.
59+
60+
## Telemetry
61+
62+
Opt-in aggregate operational metrics: per-op counts and error tallies, flushed
63+
as JSON. Off by default.
64+
65+
```yaml
66+
telemetry:
67+
enabled: true
68+
path: ~/.local/state/siphon/telemetry.json # optional; this is the default
69+
```
70+
71+
Telemetry records **only** the operation name and outcome — never profile names,
72+
hosts, dump IDs, the actor, or any data. It is composed onto the audit seam, so
73+
enabling it adds no new interception in the verbs.
74+
75+
## Scheduled backups
76+
77+
`siphon schedule` manages recurring backups by maintaining a delimited,
78+
siphon-owned block in your **crontab** — siphon does not run a scheduler daemon;
79+
your system's cron invokes `siphon backup <profile>` on the schedule.
80+
81+
```bash
82+
siphon schedule add prod --cron "0 2 * * *" # nightly at 02:00
83+
siphon schedule list
84+
siphon schedule remove prod
85+
```
86+
87+
Entries outside the siphon-managed block are preserved. Re-adding a profile
88+
updates its schedule in place; removing the last entry drops the managed block.
89+
Requires the `crontab` command.
90+
91+
## SSH tunnel
92+
93+
`siphon tunnel <profile>` opens an SSH local-forward to a profile's database
94+
through a configured bastion, using your **system ssh client** (your ssh config,
95+
keys, and agent all apply). It runs in the foreground and holds the tunnel open
96+
until you press Ctrl-C.
97+
98+
```yaml
99+
profiles:
100+
prod:
101+
driver: postgres
102+
host: db.internal
103+
port: 5432
104+
tunnel:
105+
bastion: jump@bastion.example.com
106+
local_port: 15432 # optional; defaults to the DB port
107+
```
108+
109+
```bash
110+
siphon tunnel prod
111+
# tunnel open: localhost:15432 → db.internal:5432 via jump@bastion.example.com (Ctrl-C to close)
112+
```
113+
114+
Run it in one terminal and point a client (or another siphon command) at the
115+
printed local address in another. siphon delegates to `ssh -L` rather than
116+
reimplementing SSH or holding a connection in a daemon.

0 commit comments

Comments
 (0)