Feat/implement phase f

CHANGELOG.md

@@ -22,3 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - `mysql` and `mariadb` drivers as thin wrappers that inject the fork-specific binary names and declare capabilities honestly (`Parallel: false` — the dump tools are single-threaded). Registered via side-effect import in `internal/app/drivers.go`.
   - Integration suites for both engines run on the Phase D `RunDriverSuite` harness via testcontainers (`mysql:8.0`, `mariadb:11`).
   - CI installs the Postgres/MySQL/MariaDB client tools and runs the full integration suite on the Ubuntu runner (where Docker is available); unit tests continue to run on Linux/macOS/Windows.
+- Phase F advanced-transfer machinery (some CLI paths gated pending follow-up wiring — see the `docs/INCREMENTAL.md`, `docs/CROSS_ENGINE.md`, `docs/CDC.md` Status sections):
+  - **Working today:** every dump is prefixed with a 4 KB `SIPH` JSON envelope (type, base/parent IDs, WAL/binlog positions, checksum); `restore` resolves the base→incremental chain and applies it in order, with `--up-to <id>` to stop early (cycle + broken-chain detection). Sync now streams through a bounded `jobs.Stream` (default 64×1 MB) instead of `io.Pipe`, exposing a `FillPercent` backpressure metric and propagating backup failures to the restore side via `CloseErr` (no truncated dump committed as clean).
+  - **Machinery in place, CLI gated (follow-up):** incremental backup capture (Postgres WAL replication slot + LSN; MySQL/MariaDB binlog position) — `backup --incremental` returns a clear "not yet wired" error; cross-engine type-mapping (canonical schema + JSONL emit/consume with per-engine identifier quoting and placeholders) — `sync --cross-engine` is capability-gated off until typed schema introspection exists; CDC continuous mode (state-file persistence + resume + capability gating) ships as a polling scaffold, not real logical-replication streaming.
+  - Known follow-up gates before incremental backup ships to users: Postgres orphan-replication-slot cleanup, validation of the `pg_receivewal`/`mysqlbinlog` streaming invocations against live servers (compile-checked only here).

README.md

@@ -47,16 +47,16 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
 
 ## Project status
 
-| Phase                             | What it delivers                                                                                                                                                                | Status      |
-| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
-| **A** — Skeleton                  | Go module, Cobra CLI, TUI placeholder, `Driver` interface + registry, `errs`/`config`/`secrets`/`profile` packages, golangci-lint + depguard, cross-platform CI                 | ✅ Complete |
-| **B** — Postgres walking skeleton | `backup`, `restore`, `sync`, `verify`, `inspect`, `dumps`, `config`, `profile` working end-to-end against PostgreSQL                                                            | ✅ Complete |
-| **C** — TUI dashboard             | Multi-panel Bubble Tea dashboard (profiles · dumps · jobs) with live job progress, backup/restore modal forms, and snapshot tests                                               | ✅ Complete |
-| **D** — Driver hardening          | Shared cross-driver test harness (`RunDriverSuite`), capability-gating helper (`RequireCapability`), Postgres connection-probe retry, and a `docs/DRIVERS.md` contributor guide | ✅ Complete |
-| **E** — MySQL + MariaDB           | Both drivers via a shared `_mysqlcommon` package (shared `Conn`, `mysqldump`/`mariadb-dump` backup, client-pipe restore), exercised by the Phase D `RunDriverSuite` harness       | ✅ Complete |
-| **F** — Advanced transfer         | Incremental backups, bounded-buffer streaming, cross-engine sync, CDC                                                                                                           | ⏳ Planned  |
-| **G** — Ops features              | Cloud storage, secret backends, profile groups + 2FA, team mode, audit log, retention, telemetry                                                                                | ⏳ Planned  |
-| **H** — Distribution              | GoReleaser, Homebrew tap, Scoop bucket, install script, docs site                                                                                                               | ⏳ Planned  |
+| Phase                             | What it delivers                                                                                                                                                                                                                                                                                                                                                                                                                        | Status      |
+| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
+| **A** — Skeleton                  | Go module, Cobra CLI, TUI placeholder, `Driver` interface + registry, `errs`/`config`/`secrets`/`profile` packages, golangci-lint + depguard, cross-platform CI                                                                                                                                                                                                                                                                         | ✅ Complete |
+| **B** — Postgres walking skeleton | `backup`, `restore`, `sync`, `verify`, `inspect`, `dumps`, `config`, `profile` working end-to-end against PostgreSQL                                                                                                                                                                                                                                                                                                                    | ✅ Complete |
+| **C** — TUI dashboard             | Multi-panel Bubble Tea dashboard (profiles · dumps · jobs) with live job progress, backup/restore modal forms, and snapshot tests                                                                                                                                                                                                                                                                                                       | ✅ Complete |
+| **D** — Driver hardening          | Shared cross-driver test harness (`RunDriverSuite`), capability-gating helper (`RequireCapability`), Postgres connection-probe retry, and a `docs/DRIVERS.md` contributor guide                                                                                                                                                                                                                                                         | ✅ Complete |
+| **E** — MySQL + MariaDB           | Both drivers via a shared `_mysqlcommon` package (shared `Conn`, `mysqldump`/`mariadb-dump` backup, client-pipe restore), exercised by the Phase D `RunDriverSuite` harness                                                                                                                                                                                                                                                             | ✅ Complete |
+| **F** — Advanced transfer         | Bounded-buffer streaming sync + chain-walking incremental restore work today; the dump envelope, incremental WAL/binlog capture, cross-engine type-mapping, and CDC state machinery are in place but their CLI paths (`backup --incremental`, `sync --cross-engine`, CDC) are gated pending follow-up wiring — see [docs/INCREMENTAL.md](docs/INCREMENTAL.md), [docs/CROSS_ENGINE.md](docs/CROSS_ENGINE.md), [docs/CDC.md](docs/CDC.md) | 🟡 Partial  |
+| **G** — Ops features              | Cloud storage, secret backends, profile groups + 2FA, team mode, audit log, retention, telemetry                                                                                                                                                                                                                                                                                                                                        | ⏳ Planned  |
+| **H** — Distribution              | GoReleaser, Homebrew tap, Scoop bucket, install script, docs site                                                                                                                                                                                                                                                                                                                                                                       | ⏳ Planned  |
 
 ## Requirements
 
@@ -66,12 +66,12 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
   - **MySQL** profiles need `mysqldump`, `mysql`.
   - **MariaDB** profiles need `mariadb-dump`, `mariadb` (the renamed binaries shipped by MariaDB 10.5+; older installs that only ship `mysqldump`/`mysql` are not yet supported).
 
-  | Platform      | PostgreSQL                                                                                                      | MySQL / MariaDB                                       |
-  | ------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
-  | macOS         | `brew install postgresql@16`                                                                                    | `brew install mysql-client` / `brew install mariadb`  |
-  | Debian/Ubuntu | `sudo apt install postgresql-client`                                                                            | `sudo apt install mysql-client mariadb-client`        |
-  | Fedora/RHEL   | `sudo dnf install postgresql`                                                                                   | `sudo dnf install mysql mariadb`                      |
-  | Windows       | Install the [EDB PostgreSQL](https://www.postgresql.org/download/windows/) package and add its `bin/` to `PATH` | Install the MySQL / MariaDB client and add to `PATH`  |
+  | Platform      | PostgreSQL                                                                                                      | MySQL / MariaDB                                      |
+  | ------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
+  | macOS         | `brew install postgresql@16`                                                                                    | `brew install mysql-client` / `brew install mariadb` |
+  | Debian/Ubuntu | `sudo apt install postgresql-client`                                                                            | `sudo apt install mysql-client mariadb-client`       |
+  | Fedora/RHEL   | `sudo dnf install postgresql`                                                                                   | `sudo dnf install mysql mariadb`                     |
+  | Windows       | Install the [EDB PostgreSQL](https://www.postgresql.org/download/windows/) package and add its `bin/` to `PATH` | Install the MySQL / MariaDB client and add to `PATH` |
 
 - **Docker** _(optional)_ — only needed to run the integration test suite (`make test-integration`).
 
@@ -222,6 +222,8 @@ Contributions are welcome. Please read [CONTRIBUTING.md](CONTRIBUTING.md), keep
 
 Adding a new database engine? See [docs/DRIVERS.md](docs/DRIVERS.md) for the driver contributor guide.
 
+Concept docs (honest as-built status — several paths are documented follow-ups): [docs/INCREMENTAL.md](docs/INCREMENTAL.md) (incremental restore works; the backup path is a follow-up), [docs/CROSS_ENGINE.md](docs/CROSS_ENGINE.md) (type-map matrix; `--cross-engine` is gated off pending typed introspection), and [docs/CDC.md](docs/CDC.md) (scaffold only; not yet runnable).
+
 ## License
 
 [MIT](LICENSE) © [Nikhil Rajput](https://github.com/nixrajput)

docs/CDC.md

@@ -0,0 +1,88 @@
+# Change data capture (CDC)
+
+CDC ("continuous sync" / follow mode) keeps a target database in step with a
+source by streaming changes as they happen, rather than re-running a full
+backup/restore. In siphon this is **scaffolding today** — the resume-state
+plumbing and capability gating are in place, but continuous streaming is a
+future deliverable and does not run.
+
+## Table of contents
+
+- [What exists](#what-exists)
+- [The CLI surface](#the-cli-surface)
+- [Status](#status)
+- [Resume state](#resume-state)
+- [Limitations](#limitations)
+
+## What exists
+
+`internal/app/cdc.go` holds:
+
+- **`CDCState`** — the persisted resume record (`job_id`, source/target
+  profiles, `last_applied_lsn` for Postgres, `last_binlog_file` +
+  `last_binlog_pos` for MySQL/MariaDB, `updated_at`).
+- **`saveCDCState` / `loadCDCState`** — JSON state files under a per-user
+  directory resolved from `SIPHON_STATE_HOME`, then `XDG_STATE_HOME`, then
+  `$HOME/.local/state/siphon/cdc`.
+- **`RunCDC`** — a capability-gated entry point. It resolves the source and
+  target profiles, then requires both drivers to advertise `CapCDC`. Its body is
+  a **polling scaffold**: a 10-second ticker that persists state each tick and
+  resumes from a prior run's state on restart. It is **not** real change
+  streaming.
+
+`CDCState` save/load round-trip, the state-dir resolution, and the
+no-capability rejection are unit-tested (`internal/app/cdc_test.go`).
+
+## The CLI surface
+
+There is **no `siphon cdc` subcommand wired today.** `RunCDC` exists as an
+application function but is not registered as a Cobra command
+(`internal/cli/root.go` wires `backup`, `restore`, `sync`, `verify`, `inspect`,
+`profile`, `dumps`, `config`, `schedule`, `tunnel` — no `cdc`).
+
+`sync --continuous` exposes the flag but does not run CDC; it returns a clear
+error pointing at the (not-yet-wired) `siphon cdc` follow-up:
+
+```bash
+siphon sync --from pg-prod --to pg-replica --continuous
+# Error: continuous CDC sync is not wired here; use `siphon cdc` (Phase F Task 10)
+```
+
+## Status
+
+| Capability | Status |
+| --- | --- |
+| `CDCState` persistence (save/load) | ✅ Works (unit-tested) |
+| State-dir resolution (`SIPHON_STATE_HOME` / `XDG_STATE_HOME`) | ✅ Works (unit-tested) |
+| Resume from prior state on restart | ✅ Works (in the scaffold loop) |
+| Capability gating on `CapCDC` | ✅ Works |
+| `siphon cdc` CLI subcommand | ❌ Not wired (no Cobra command) |
+| Continuous change streaming | ⚠️ Scaffold only (polling tick; not real CDC) |
+
+CDC does **not run today**. No driver declares `CapCDC` true, so `RunCDC` is
+rejected with `ErrDriverUnsupported`. Even if a driver enabled it, the loop is a
+polling scaffold, not logical-replication streaming.
+
+## Resume state
+
+When CDC streaming is built, `RunCDC` resumes from the last persisted position.
+State files live at:
+
+```text
+$SIPHON_STATE_HOME/cdc/<job-id>.state        # if SIPHON_STATE_HOME is set
+$XDG_STATE_HOME/siphon/cdc/<job-id>.state     # else if XDG_STATE_HOME is set
+$HOME/.local/state/siphon/cdc/<job-id>.state  # default
+```
+
+Each file is JSON: source/target profiles plus the last applied position (LSN
+for Postgres, binlog file + offset for MySQL/MariaDB).
+
+## Limitations
+
+- **Not enabled on any driver** — `CapCDC` is `false` everywhere, so CDC is
+  rejected up front.
+- **No real streaming** — the scaffold polls on a ticker. Real logical-
+  replication streaming (`pglogrepl` for Postgres, binlog tailing for
+  MySQL/MariaDB) is deferred to a future revision.
+- **No CLI entry point** — `RunCDC` is internal application scaffolding, not a
+  user-facing command yet.