Skip to content

Commit 18c2ea6

Browse files
julianknutsenclaude
andcommitted
Add remote Provider abstraction and parameterized offline integration tests
Extract DoltHub-specific code behind a Provider interface so the same federation logic works with DoltHub, file:// dolt remotes, and bare git repos. Wire up --remote-base and --git-remote flags on "wl join" so tests (and users) can operate entirely offline. All 12 offline integration tests are parameterized over both file and git backends (24 subtests), plus 9 Provider conformance tests covering both implementations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 448b1ef commit 18c2ea6

File tree

18 files changed

+1941
-321
lines changed

18 files changed

+1941
-321
lines changed

.github/workflows/ci.yml

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,30 @@ jobs:
4141
token: ${{ secrets.CODECOV_TOKEN }}
4242
verbose: true
4343

44+
integration-offline:
45+
name: Integration (offline)
46+
needs: check
47+
runs-on: ubuntu-latest
48+
steps:
49+
- uses: actions/checkout@v6
50+
51+
- uses: actions/setup-go@v6
52+
with:
53+
go-version-file: go.mod
54+
55+
- name: Install dolt
56+
run: sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash'
57+
58+
- name: Configure dolt
59+
run: |
60+
dolt config --global --add user.email "ci@wasteland.dev"
61+
dolt config --global --add user.name "CI"
62+
63+
- name: Offline integration tests
64+
run: make test-integration-offline
65+
4466
integration:
45-
name: Integration
67+
name: Integration (DoltHub)
4668
needs: check
4769
if: github.event_name == 'push'
4870
runs-on: ubuntu-latest

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@ CLAUDE.local.md
22
coverage.txt
33
/bin/
44
/wl
5+
.envrc

Makefile

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ LDFLAGS := -X main.version=$(VERSION) \
2020
-X main.commit=$(COMMIT) \
2121
-X main.date=$(BUILD_TIME)
2222

23-
.PHONY: build check check-all lint fmt-check fmt vet test test-integration test-cover cover install install-tools setup clean
23+
.PHONY: build check check-all lint fmt-check fmt vet test test-integration test-integration-offline test-cover cover install install-tools setup clean
2424

2525
## build: compile wl binary with version metadata
2626
build:
@@ -67,6 +67,10 @@ test:
6767
test-integration:
6868
go test -tags integration ./...
6969

70+
## test-integration-offline: run offline integration tests only (no network, requires dolt)
71+
test-integration-offline:
72+
go test -tags integration -v ./internal/remote/ ./test/integration/offline/
73+
7074
## test-cover: run tests with coverage output
7175
test-cover:
7276
go test -coverprofile=coverage.txt ./...

TESTING.md

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,26 @@ injection. These are fast and run everywhere.
1717
When to use: corrupted data, concurrent writes, specific error types,
1818
double-claim conflicts, rollback behavior, boundary conditions.
1919

20-
### 2. Integration tests (`//go:build integration`)
20+
### 2a. Offline integration tests (`test/integration/offline/`)
2121

22-
Test that real pieces fit together. Need real dolt CLI, real DoltHub,
23-
real filesystem. Run separately — not in CI by default.
22+
Test the wl binary end-to-end against real dolt databases using `file://`
23+
remotes. No network access required. Runs in CI on every PR.
24+
25+
Each test gets its own isolated `testEnv` with temp XDG dirs — zero
26+
cross-contamination. The `wl` binary is built once in `TestMain`.
27+
28+
- **Lifecycle tests**: post -> claim -> done full cycle, error cases
29+
- **Sync tests**: file:// upstream remotes, `wl sync` and `--dry-run`
30+
31+
Run with: `make test-integration-offline`
32+
33+
When to use: verifying CLI behavior end-to-end, testing dolt interactions
34+
without network, validating the full post/claim/done lifecycle.
35+
36+
### 2b. DoltHub integration tests (`test/integration/`)
37+
38+
Test that real pieces fit together with DoltHub. Need real dolt CLI,
39+
real DoltHub, real filesystem. Runs only on push to main.
2440

2541
```go
2642
//go:build integration
@@ -31,9 +47,9 @@ func TestRealDoltClone(t *testing.T) {
3147
```
3248

3349
When to use: proving the fakes are honest, smoke testing the real infra,
34-
testing dolt CLI interactions with real databases.
50+
testing dolt CLI interactions with real DoltHub databases.
3551

36-
Run with: `go test -tags integration ./test/`
52+
Run with: `go test -tags integration ./test/integration/`
3753

3854
## Decision guide
3955

@@ -44,9 +60,12 @@ Run with: `go test -tags integration ./test/`
4460
| Does CSV parsing handle quoted fields? | Unit test |
4561
| Does SQL escaping prevent injection? | Unit test |
4662
| Does the federation join workflow call steps in order? | Unit test |
47-
| Does a real dolt clone succeed from DoltHub? | Integration |
48-
| Does `hop/wl-commons` schema match expected tables/columns? | Integration |
49-
| Are all `wanted` statuses/priorities/types valid? | Integration |
63+
| Does `wl post` create a valid database row? | Offline integration |
64+
| Does `wl claim` on an already-claimed item fail? | Offline integration |
65+
| Does `wl sync` pull from an upstream file:// remote? | Offline integration |
66+
| Does a real dolt clone succeed from DoltHub? | DoltHub integration |
67+
| Does `hop/wl-commons` schema match expected tables/columns? | DoltHub integration |
68+
| Are all `wanted` statuses/priorities/types valid? | DoltHub integration |
5069

5170
## Test doubles
5271

cmd/wl/cmd_join.go

Lines changed: 64 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99
"github.com/spf13/cobra"
1010
"github.com/steveyegge/wasteland/internal/commons"
1111
"github.com/steveyegge/wasteland/internal/federation"
12+
"github.com/steveyegge/wasteland/internal/remote"
1213
"github.com/steveyegge/wasteland/internal/style"
1314
)
1415

@@ -17,6 +18,9 @@ func newJoinCmd(stdout, stderr io.Writer) *cobra.Command {
1718
handle string
1819
displayName string
1920
email string
21+
forkOrg string
22+
remoteBase string
23+
gitRemote string
2024
)
2125

2226
cmd := &cobra.Command{
@@ -25,53 +29,55 @@ func newJoinCmd(stdout, stderr io.Writer) *cobra.Command {
2529
Long: `Join a wasteland community by forking its shared commons database.
2630
2731
This command:
28-
1. Forks the upstream commons to your DoltHub org
32+
1. Forks the upstream commons to your org
2933
2. Clones the fork locally
3034
3. Registers your rig in the rigs table
3135
4. Pushes the registration to your fork
3236
5. Saves wasteland configuration locally
3337
34-
The upstream argument is a DoltHub path like 'steveyegge/wl-commons'.
38+
The upstream argument is an org/database path like 'steveyegge/wl-commons'.
3539
36-
Required environment variables:
37-
DOLTHUB_TOKEN - Your DoltHub API token
38-
DOLTHUB_ORG - Your DoltHub organization name
40+
DoltHub mode (default):
41+
Requires DOLTHUB_TOKEN and DOLTHUB_ORG (or --fork-org).
42+
Forks and clones via DoltHub.
43+
44+
Offline file mode (--remote-base):
45+
Uses file:// dolt remotes. No DoltHub credentials needed.
46+
Requires --fork-org (or DOLTHUB_ORG).
47+
48+
Git remote mode (--git-remote):
49+
Uses bare git repos as dolt remotes. No DoltHub credentials needed.
50+
Requires --fork-org (or DOLTHUB_ORG).
3951
4052
Examples:
4153
wl join steveyegge/wl-commons
4254
wl join steveyegge/wl-commons --handle my-rig
43-
wl join steveyegge/wl-commons --display-name "Alice's Workshop"`,
55+
wl join test-org/wl-commons --remote-base /tmp/remotes --fork-org my-fork
56+
wl join test-org/wl-commons --git-remote /tmp/git-remotes --fork-org my-fork`,
4457
Args: cobra.ExactArgs(1),
4558
RunE: func(cmd *cobra.Command, args []string) error {
46-
return runJoin(stdout, stderr, args[0], handle, displayName, email)
59+
return runJoin(stdout, stderr, args[0], handle, displayName, email, forkOrg, remoteBase, gitRemote)
4760
},
4861
}
4962

50-
cmd.Flags().StringVar(&handle, "handle", "", "Rig handle for registration (default: DoltHub org)")
63+
cmd.Flags().StringVar(&handle, "handle", "", "Rig handle for registration (default: fork org)")
5164
cmd.Flags().StringVar(&displayName, "display-name", "", "Display name for the rig registry")
5265
cmd.Flags().StringVar(&email, "email", "", "Registration email (default: git config user.email)")
66+
cmd.Flags().StringVar(&forkOrg, "fork-org", "", "Fork organization (default: DOLTHUB_ORG)")
67+
cmd.Flags().StringVar(&remoteBase, "remote-base", "", "Base directory for file:// remotes (offline mode)")
68+
cmd.Flags().StringVar(&gitRemote, "git-remote", "", "Base directory for bare git remotes")
69+
cmd.MarkFlagsMutuallyExclusive("remote-base", "git-remote")
5370

5471
return cmd
5572
}
5673

57-
func runJoin(stdout, stderr io.Writer, upstream, handle, displayName, email string) error {
74+
func runJoin(stdout, stderr io.Writer, upstream, handle, displayName, email, forkOrg, remoteBase, gitRemote string) error {
5875
// Parse upstream path (validate early)
5976
_, _, err := federation.ParseUpstream(upstream)
6077
if err != nil {
6178
return err
6279
}
6380

64-
// Require DoltHub credentials
65-
token := commons.DoltHubToken()
66-
if token == "" {
67-
return fmt.Errorf("DOLTHUB_TOKEN environment variable is required\n\nGet your token from https://www.dolthub.com/settings/tokens")
68-
}
69-
70-
forkOrg := commons.DoltHubOrg()
71-
if forkOrg == "" {
72-
return fmt.Errorf("DOLTHUB_ORG environment variable is required\n\nSet this to your DoltHub organization name")
73-
}
74-
7581
// Fast path: check if already joined
7682
if existing, loadErr := federation.LoadConfig(); loadErr == nil {
7783
if existing.Upstream == upstream {
@@ -84,6 +90,40 @@ func runJoin(stdout, stderr io.Writer, upstream, handle, displayName, email stri
8490
return fmt.Errorf("already joined to %s; run wl leave first", existing.Upstream)
8591
}
8692

93+
// Resolve fork org: flag > env var
94+
if forkOrg == "" {
95+
forkOrg = commons.DoltHubOrg()
96+
}
97+
98+
var provider remote.Provider
99+
100+
switch {
101+
case remoteBase != "":
102+
// Offline file mode — file:// dolt remotes, no DoltHub credentials needed.
103+
if forkOrg == "" {
104+
return fmt.Errorf("--fork-org is required in offline mode (or set DOLTHUB_ORG)")
105+
}
106+
provider = remote.NewFileProvider(remoteBase)
107+
108+
case gitRemote != "":
109+
// Git remote mode — bare git repos as dolt remotes, no DoltHub credentials needed.
110+
if forkOrg == "" {
111+
return fmt.Errorf("--fork-org is required in git remote mode (or set DOLTHUB_ORG)")
112+
}
113+
provider = remote.NewGitProvider(gitRemote)
114+
115+
default:
116+
// DoltHub mode — requires token and org.
117+
token := commons.DoltHubToken()
118+
if token == "" {
119+
return fmt.Errorf("DOLTHUB_TOKEN environment variable is required\n\nGet your token from https://www.dolthub.com/settings/tokens")
120+
}
121+
if forkOrg == "" {
122+
return fmt.Errorf("DOLTHUB_ORG environment variable is required\n\nSet this to your DoltHub organization name")
123+
}
124+
provider = remote.NewDoltHubProvider(token)
125+
}
126+
87127
// Determine handle
88128
if handle == "" {
89129
handle = forkOrg
@@ -101,13 +141,14 @@ func runJoin(stdout, stderr io.Writer, upstream, handle, displayName, email stri
101141

102142
wlVersion := "dev"
103143

104-
svc := federation.NewService()
144+
svc := federation.NewService(provider)
105145
svc.OnProgress = func(step string) {
106146
fmt.Fprintf(stdout, " %s\n", step)
107147
}
108148

109-
fmt.Fprintf(stdout, "Joining wasteland %s (fork to %s/%s)...\n", upstream, forkOrg, upstream[strings.Index(upstream, "/")+1:])
110-
cfg, err := svc.Join(upstream, forkOrg, token, handle, displayName, email, wlVersion)
149+
dbName := upstream[strings.Index(upstream, "/")+1:]
150+
fmt.Fprintf(stdout, "Joining wasteland %s (fork to %s/%s)...\n", upstream, forkOrg, dbName)
151+
cfg, err := svc.Join(upstream, forkOrg, handle, displayName, email, wlVersion)
111152
if err != nil {
112153
return err
113154
}

0 commit comments

Comments
 (0)