Feat/implement phase e (#4)

Author: nixrajput

.github/workflows/ci.yml

@@ -38,5 +38,28 @@ jobs:
           cache: true
       - name: go vet
         run: go vet ./...
+      # `./...` skips underscore-prefixed dirs, so name the _mysqlcommon package
+      # explicitly by import path to run its unit tests. Use the import path (not
+      # a `_*` shell glob) so it works on PowerShell (Windows) as well as bash.
       - name: go test
-        run: go test ./...
+        run: go test ./... github.com/nixrajput/siphon/internal/driver/_mysqlcommon
+      # Integration tests use testcontainers, which needs a Docker daemon.
+      # Only ubuntu-latest runners have Docker available by default, so the
+      # DB client install and integration suite are scoped to Ubuntu.
+      # macOS/Windows run unit tests only (the integration-tagged files that
+      # shell out to the client binaries are tagged out of the unit build).
+      - name: Install DB client tools (integration, Ubuntu only)
+        if: matrix.os == 'ubuntu-latest'
+        run: |
+          sudo apt-get update
+          # postgresql-client -> pg_dump/pg_restore ; mysql-client -> mysqldump/mysql
+          sudo apt-get install -y postgresql-client mysql-client
+          # Ubuntu's mariadb-client Conflicts with mysql-client-core, so it can't
+          # co-install. Pull mariadb-dump/mariadb from MariaDB's official APT repo
+          # (the repo's mariadb-client is built to coexist with MySQL's client).
+          curl -fsSL https://r.mariadb.com/downloads/mariadb_repo_setup | sudo bash
+          sudo apt-get update
+          sudo apt-get install -y mariadb-client
+      - name: Integration tests (Ubuntu only — needs Docker)
+        if: matrix.os == 'ubuntu-latest'
+        run: go test -tags=integration ./... github.com/nixrajput/siphon/internal/driver/_mysqlcommon

CHANGELOG.md

@@ -17,3 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Capability gating helper: `app.RequireCapability` resolves a profile to its driver and rejects unsupported affordances with a `CodeUser` error. The helper and the `Capabilities` flags are in place; verbs are wired to it only where the gated feature is implemented. Streaming (`NativeStream`) and parallel backup (`Parallel`) are deferred to Phase F, so `sync --stream` / `backup --jobs` are not yet gated — the wiring lands with those features.
   - Connection-probe retry on Postgres `Connect` (3 attempts, exponential backoff) per spec §4.3; the generic `Retry` helper moved to `internal/jobs` to keep the driver layer free of an app-layer import.
   - `docs/DRIVERS.md` contributor guide documenting the driver contract, registration, the test harness, capability flags, and error mapping.
+- Phase E MySQL + MariaDB drivers:
+  - Shared `internal/driver/_mysqlcommon` package: DSN builder, `mysqldump`/`mariadb-dump` arg builder, fork detection, and a shared `Conn` implementing the full `driver.Conn` contract (Inspect via `information_schema`, Backup by shelling to the dump tool, Restore by piping SQL into the client, sha256 Verify, bounded connect-probe retry). Backup/Restore pass the password via `MYSQL_PWD` and stream through stdout/stdin.
+  - `mysql` and `mariadb` drivers as thin wrappers that inject the fork-specific binary names and declare capabilities honestly (`Parallel: false` — the dump tools are single-threaded). Registered via side-effect import in `internal/app/drivers.go`.
+  - Integration suites for both engines run on the Phase D `RunDriverSuite` harness via testcontainers (`mysql:8.0`, `mariadb:11`).
+  - CI installs the Postgres/MySQL/MariaDB client tools and runs the full integration suite on the Ubuntu runner (where Docker is available); unit tests continue to run on Linux/macOS/Windows.

Makefile

@@ -9,14 +9,21 @@ help: ## Show this help
 lint: ## Run golangci-lint
 	golangci-lint run
 
+# NOTE: `go test ./...` skips directories whose names start with `_` (Go tool
+# convention), so underscore-prefixed packages with real tests (e.g.
+# internal/driver/_mysqlcommon) are silently excluded. Name them explicitly by
+# import path so their unit tests run. Use the import path (not a `_*` shell
+# glob) so this is identical to what CI runs and works regardless of shell.
+UNDERSCORE_PKGS := github.com/nixrajput/siphon/internal/driver/_mysqlcommon
+
 test: ## Run unit tests
-	go test ./...
+	go test ./... $(UNDERSCORE_PKGS)
 
 test-verbose: ## Run unit tests verbosely
-	go test -v ./...
+	go test -v ./... $(UNDERSCORE_PKGS)
 
 test-integration: ## Run integration tests (build tag: integration)
-	go test -tags=integration ./...
+	go test -tags=integration ./... $(UNDERSCORE_PKGS)
 
 build: ## Build the siphon binary into ./bin
 	@mkdir -p bin

README.md

@@ -17,7 +17,7 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
 ---
 
 > [!WARNING]
-> **Pre-1.0 — active development.** Postgres backup/restore/sync/verify/inspect work end-to-end today (Phase B), and bare `siphon` opens an interactive multi-panel dashboard (Phase C). The driver layer is hardened with a shared cross-driver test harness, capability gating, and connection retry (Phase D), so MySQL/MariaDB can land mechanically next. MySQL/MariaDB, incremental backups, and ops features are on the [roadmap](#roadmap). APIs, flags, and the on-disk dump format may change before 1.0. Track progress via the milestone tags (`phase-a`, `phase-b`, `phase-c`, `phase-d`, …).
+> **Pre-1.0 — active development.** Postgres, MySQL, and MariaDB backup/restore/sync/verify/inspect work end-to-end today (Phases B + E), and bare `siphon` opens an interactive multi-panel dashboard (Phase C). The driver layer is hardened with a shared cross-driver test harness, capability gating, and connection retry (Phase D). Incremental backups, cross-engine sync, and ops features are on the [roadmap](#roadmap). APIs, flags, and the on-disk dump format may change before 1.0. Track progress via the milestone tags (`phase-a`, `phase-b`, `phase-c`, `phase-d`, `phase-e`, …).
 
 ## Table of contents
 
@@ -38,8 +38,8 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
 
 ## Why siphon
 
-- **One CLI, many databases.** Postgres works today; MySQL and MariaDB land in v1.0 (sharing a common backend). The driver interface is engine-agnostic, so SQLite, MongoDB, SQL Server, and ClickHouse can follow.
-- **Native, not reimplemented.** siphon shells out to `pg_dump`/`pg_restore` (and `mysqldump`/`mariadb-dump` in v1.0) for the actual data movement — you inherit 20+ years of correctness from the official tools, wrapped in a consistent UX.
+- **One CLI, many databases.** Postgres, MySQL, and MariaDB all work today (MySQL and MariaDB share a common `_mysqlcommon` backend). The driver interface is engine-agnostic, so SQLite, MongoDB, SQL Server, and ClickHouse can follow.
+- **Native, not reimplemented.** siphon shells out to `pg_dump`/`pg_restore`, `mysqldump`/`mysql`, and `mariadb-dump`/`mariadb` for the actual data movement — you inherit 20+ years of correctness from the official tools, wrapped in a consistent UX.
 - **Integrity by default.** Every dump is checksummed (SHA-256) and recorded in a sidecar metadata file. `siphon verify` re-hashes the dump and flags corruption or tampering — and fails with a distinct exit code so CI can catch it.
 - **Built for scripts and humans.** A Cobra command tree with predictable flags and POSIX exit codes for automation; an interactive Bubble Tea dashboard when you invoke `siphon` bare.
 - **Named profiles + secret refs.** Store connection details once; reference secrets as `env:VAR` today, with OS keychain / Vault / 1Password / AWS Secrets Manager backends on the roadmap. Plaintext passwords never have to live in your config.
@@ -53,22 +53,25 @@ A single binary that turns the painful, error-prone sprawl of `pg_dump` → `pg_
 | **B** — Postgres walking skeleton | `backup`, `restore`, `sync`, `verify`, `inspect`, `dumps`, `config`, `profile` working end-to-end against PostgreSQL                                                            | ✅ Complete |
 | **C** — TUI dashboard             | Multi-panel Bubble Tea dashboard (profiles · dumps · jobs) with live job progress, backup/restore modal forms, and snapshot tests                                               | ✅ Complete |
 | **D** — Driver hardening          | Shared cross-driver test harness (`RunDriverSuite`), capability-gating helper (`RequireCapability`), Postgres connection-probe retry, and a `docs/DRIVERS.md` contributor guide | ✅ Complete |
-| **E** — MySQL + MariaDB           | Both drivers via a shared `_mysqlcommon` package                                                                                                                                | ⏳ Planned  |
+| **E** — MySQL + MariaDB           | Both drivers via a shared `_mysqlcommon` package (shared `Conn`, `mysqldump`/`mariadb-dump` backup, client-pipe restore), exercised by the Phase D `RunDriverSuite` harness       | ✅ Complete |
 | **F** — Advanced transfer         | Incremental backups, bounded-buffer streaming, cross-engine sync, CDC                                                                                                           | ⏳ Planned  |
 | **G** — Ops features              | Cloud storage, secret backends, profile groups + 2FA, team mode, audit log, retention, telemetry                                                                                | ⏳ Planned  |
 | **H** — Distribution              | GoReleaser, Homebrew tap, Scoop bucket, install script, docs site                                                                                                               | ⏳ Planned  |
 
 ## Requirements
 
 - **[Go](https://go.dev/dl/) 1.26 or newer** — to build from source (the only install method until Phase H).
-- **PostgreSQL client tools** — `pg_dump`, `pg_restore`, and `psql` must be on your `PATH`. siphon shells out to them; it does not embed a Postgres client.
-
-  | Platform      | Install                                                                                                         |
-  | ------------- | --------------------------------------------------------------------------------------------------------------- |
-  | macOS         | `brew install postgresql@16`                                                                                    |
-  | Debian/Ubuntu | `sudo apt install postgresql-client`                                                                            |
-  | Fedora/RHEL   | `sudo dnf install postgresql`                                                                                   |
-  | Windows       | Install the [EDB PostgreSQL](https://www.postgresql.org/download/windows/) package and add its `bin/` to `PATH` |
+- **Database client tools** — siphon shells out to the native dump/restore tools; it does not embed a client. You only need the tools for the engines you actually use:
+  - **PostgreSQL** profiles need `pg_dump`, `pg_restore`, `psql`.
+  - **MySQL** profiles need `mysqldump`, `mysql`.
+  - **MariaDB** profiles need `mariadb-dump`, `mariadb` (the renamed binaries shipped by MariaDB 10.5+; older installs that only ship `mysqldump`/`mysql` are not yet supported).
+
+  | Platform      | PostgreSQL                                                                                                      | MySQL / MariaDB                                       |
+  | ------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
+  | macOS         | `brew install postgresql@16`                                                                                    | `brew install mysql-client` / `brew install mariadb`  |
+  | Debian/Ubuntu | `sudo apt install postgresql-client`                                                                            | `sudo apt install mysql-client mariadb-client`        |
+  | Fedora/RHEL   | `sudo dnf install postgresql`                                                                                   | `sudo dnf install mysql mariadb`                      |
+  | Windows       | Install the [EDB PostgreSQL](https://www.postgresql.org/download/windows/) package and add its `bin/` to `PATH` | Install the MySQL / MariaDB client and add to `PATH`  |
 
 - **Docker** _(optional)_ — only needed to run the integration test suite (`make test-integration`).
 

docs/DRIVERS.md

@@ -216,6 +216,12 @@ The `Fixtures` struct (in `internal/driver/_testing/fixtures.go`) you populate:
 
 The worked example is `internal/driver/postgres/integration_test.go` — copy its
 shape. The suite runs behind the `integration` build tag (`make test-integration`).
+`internal/driver/mysql/` and `internal/driver/mariadb/` are a second worked example:
+two engines that share almost everything live in `internal/driver/_mysqlcommon/`
+(a shared `Conn` plus arg/DSN builders), leaving each driver a ~30-line wrapper
+that only injects the fork-specific binary names and capabilities. If you're
+adding an engine that's a fork of an existing one, follow that shared-helper
+pattern rather than copying a whole driver.
 
 ## Error mapping
 

go.mod

@@ -8,16 +8,20 @@ require (
 	github.com/charmbracelet/huh v1.0.0
 	github.com/charmbracelet/lipgloss v1.1.0
 	github.com/charmbracelet/x/exp/teatest v0.0.0-20260527151214-009e6338d40d
+	github.com/go-sql-driver/mysql v1.10.0
 	github.com/jackc/pgx/v5 v5.9.2
 	github.com/muesli/termenv v0.16.0
 	github.com/oklog/ulid/v2 v2.1.1
 	github.com/spf13/cobra v1.10.2
+	github.com/testcontainers/testcontainers-go/modules/mariadb v0.42.0
+	github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0
 	github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0
 	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
 	dario.cat/mergo v1.0.2 // indirect
+	filippo.io/edwards25519 v1.2.0 // indirect
 	github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
 	github.com/Microsoft/go-winio v0.6.2 // indirect
 	github.com/atotto/clipboard v0.1.4 // indirect

go.sum

@@ -1,5 +1,7 @@
 dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
 dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
+filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
+filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
 github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
 github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
@@ -95,6 +97,8 @@ github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
 github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
+github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
+github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
 github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
 github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
@@ -195,6 +199,10 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
 github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
 github.com/testcontainers/testcontainers-go v0.42.0 h1:He3IhTzTZOygSXLJPMX7n44XtK+qhjat1nI9cneBbUY=
 github.com/testcontainers/testcontainers-go v0.42.0/go.mod h1:vZjdY1YmUA1qEForxOIOazfsrdyORJAbhi0bp8plN30=
+github.com/testcontainers/testcontainers-go/modules/mariadb v0.42.0 h1:ZfWUJSIDnbNgoLAXMV1fc7lqcxBIX3zdnhwjaVUo7N0=
+github.com/testcontainers/testcontainers-go/modules/mariadb v0.42.0/go.mod h1:0kV+yHee7zAgp0yccydxjNnHvlC1EOavTLCeg/lnRBY=
+github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0 h1:Yhv1k7vDpyzZePntg5R5Oj4ZMCyWpAfpJeRu1ROsgiU=
+github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0/go.mod h1:Z7SCTuiZlghAdRjkv3Ir0iXJKC2T2avbtxLR0DRe+ng=
 github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0 h1:GCbb1ndrF7OTDiIvxXyItaDab4qkzTFJ48LKFdM7EIo=
 github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0/go.mod h1:IRPBaI8jXdrNfD0e4Zm7Fbcgaz5shKxOQv4axiL09xs=
 github.com/tklauser/go-sysconf v0.3.16 h1:frioLaCQSsF5Cy1jgRBrzr6t502KIIwQ0MArYICU0nA=

internal/app/drivers.go

@@ -2,6 +2,8 @@ package app
 
 import (
 	"github.com/nixrajput/siphon/internal/driver"
+	_ "github.com/nixrajput/siphon/internal/driver/mariadb"  // register the mariadb driver
+	_ "github.com/nixrajput/siphon/internal/driver/mysql"    // register the mysql driver
 	_ "github.com/nixrajput/siphon/internal/driver/postgres" // register the postgres driver
 )
 

internal/driver/_mysqlcommon/args.go

@@ -0,0 +1,56 @@
+package mysqlcommon
+
+import (
+	"strconv"
+
+	"github.com/nixrajput/siphon/internal/driver"
+)
+
+// BuildDumpArgs assembles the mysqldump argument vector for a backup. The
+// binary name itself is supplied by the caller (mysqldump vs mariadb-dump).
+func BuildDumpArgs(p driver.Profile, opt driver.BackupOpts) []string {
+	args := []string{
+		"-h", p.Host,
+		"-P", strconv.Itoa(p.Port),
+		"-u", p.User,
+		"--single-transaction",
+		"--routines",
+		"--triggers",
+		"--events",
+		"--no-tablespaces",
+		"--skip-comments",
+		p.Database,
+	}
+	if opt.SchemaOnly {
+		args = append(args, "--no-data")
+	}
+	if opt.DataOnly {
+		args = append(args, "--no-create-info")
+	}
+	args = append(args, opt.IncludeTables...)
+	for _, t := range opt.ExcludeTables {
+		args = append(args, "--ignore-table="+p.Database+"."+t)
+	}
+	return args
+}
+
+// BuildRestoreArgs assembles the mysql client argument vector for a restore.
+// The dump file is authoritative for shape; the restore client just pipes it
+// in. Clean is a no-op here because mysqldump output already emits
+// DROP TABLE IF EXISTS / CREATE TABLE, making the restore idempotent.
+func BuildRestoreArgs(p driver.Profile, _ driver.RestoreOpts) []string {
+	return []string{
+		"-h", p.Host,
+		"-P", strconv.Itoa(p.Port),
+		"-u", p.User,
+		"--default-character-set=utf8mb4",
+		p.Database,
+	}
+}
+
+// NOTE: Profile.SSLMode is honored by the connect DSN (see dsn.go) but is NOT
+// yet propagated to the dump/restore CLI tools. The flag differs across forks —
+// MySQL's mysqldump uses --ssl-mode=<DISABLED|REQUIRED|VERIFY_CA|VERIFY_IDENTITY>
+// while MariaDB's mariadb-dump uses --ssl / --skip-ssl — so it needs per-fork
+// handling (and the dump tools' stderr surfaced so a rejected flag isn't opaque).
+// Tracked as a follow-up; see the PR #4 discussion.