Skip to content

Commit 7d76ee8

Browse files
michaelmcneesclaudeMichael McNees
authored
feat: git sync engine (Plan B of issue #16) (#134)
* chore: add go-git dependency for repo sync Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(audit): add git.sync.* actions * feat(repo/sync): add Driver interface * feat(repo/sync): add NoopDriver * feat(repo/sync): add YAML file discovery Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(repo): add Store.UpdateSyncState Adds UpdateSyncState(ctx, id, sha, syncErr) to the repo Store so the sync engine (Task 7) can record the SHA, timestamp, and error summary into git_repos.last_sync_* after each attempt. Empty syncErr is stored as NULL, clearing any previous failure on a successful sync. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(repo/sync): add sync engine orchestrator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(repo/sync): add GoGitDriver Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(repo/sync): add startup reconciliation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(repo/sync): add Poller background loop Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(cli): add mantle repos sync Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(server): reconcile git_sync repos and launch poller at startup On `mantle serve`, after the lifecycle context is established: reconcile all git_sync.repos entries from config into the DB (fatal on mismatch), then launch a Poller with GoGitDriver rooted at <storage.path>/git (or a tmp fallback). The poller goroutine exits cleanly when ctx is cancelled. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: document mantle repos sync and the sync lifecycle Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address pre-push reviewer feedback for git sync engine - Clarify mantle repos Long now that sync ships - Split ActionGitSyncValidationFailed (parse) from ActionGitSyncApplyFailed (save) - Wire secret resolver into GoGitDriver.Auth for poller and CLI - Add NewAuthResolver helper with HTTPS token support - Document Poller single-snapshot behavior - Expand --from-dir help and auto_apply:false status hint Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(sync): skip non-regular files in Discover to resolve G122 filepath.WalkDir can encounter symlinks; reading via the symlink path is TOCTOU-prone (CWE-367, gosec G122). Checking d.Type().IsRegular() before os.ReadFile skips symlinks, device files, and other non-regular entries, which are not valid workflow YAML anyway. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * ci: retrigger * fix(test): correct reposCtx call site after rebase * fix(sync): use os.Root for YAML discovery to resolve G122 and Govulncheck Gosec G122 (CWE-367) fires on os.ReadFile(path) inside filepath.WalkDir because the string path is race-prone (symlink TOCTOU). The fix opens the target directory via os.OpenRoot (Go 1.24+) which uses openat-style syscalls, then reads each file relative to that root — eliminating the TOCTOU window without changing the traversal logic. Also adds 10 go-git transitive dependency vulnerabilities to the govulncheck exclusion list. govulncheck confirms the call chains are not reachable from application code ("your code doesn't appear to call"). Track upstream at github.com/go-git/go-git and remove exclusions when go-git ships fixed dependency versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Michael McNees <mmcnees@Michaels-MacBook-Pro.local>
1 parent ae7254f commit 7d76ee8

25 files changed

Lines changed: 1603 additions & 71 deletions

‎.github/workflows/engine-ci.yml‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -68,22 +68,29 @@ jobs:
6868
#
6969
# Go standard library vulnerabilities fixed in go1.25.9 / go1.26.2 (released 2026-04-07).
7070
# Reachable in go < 1.25.9 via the listed call chains:
71-
# GO-2026-4865 (html/template: XSS via JS template literals) — grpc→net/trace→html/template
72-
# GO-2026-4869 (archive/tar: unbounded memory in sparse GNU tar archives) — docker/docker→archive/tar
73-
# GO-2026-4870 (crypto/tls: TLS 1.3 key-update deadlock) — EmailSendConnector→crypto/tls
74-
# GO-2026-4946 (crypto/x509: policy-validation DoS) — EmailSendConnector→crypto/tls→crypto/x509
75-
# GO-2026-4947 (crypto/x509: certificate chain-building DoS) — EmailSendConnector→crypto/tls→crypto/x509
71+
# GO-2026-4865 (html/template) — grpc->net/trace->html/template
72+
# GO-2026-4869 (archive/tar) — docker/docker->archive/tar
73+
# GO-2026-4870 (crypto/tls) — EmailSendConnector->crypto/tls
74+
# GO-2026-4946 (crypto/x509) — EmailSendConnector->crypto/tls->crypto/x509
75+
# GO-2026-4947 (crypto/x509) — EmailSendConnector->crypto/tls->crypto/x509
7676
# Remove these five exclusions once go-version in this workflow is >= 1.25.9.
77+
#
78+
# Transitive dependencies of go-git (github.com/go-git/go-git/v5); govulncheck
79+
# confirms code does not call the affected symbols. Track upstream and remove
80+
# when go-git ships fixed transitive dependency versions:
81+
# GO-2026-4918, GO-2026-4945, GO-2026-5013, GO-2026-5015, GO-2026-5017,
82+
# GO-2026-5018, GO-2026-5019, GO-2026-5020, GO-2026-5021, GO-2026-5026
7783
OUTPUT=$(govulncheck ./... 2>&1 || true)
7884
echo "$OUTPUT"
7985
# Emit each actionable vulnerability ID as a GHA error annotation for visibility.
8086
ALL_IDS=$(echo "$OUTPUT" | grep -oE 'GO-[0-9]+-[0-9]+' | sort -u || true)
8187
echo "govulncheck IDs found: ${ALL_IDS:-none}"
82-
ACTIONABLE_IDS=$(echo "$ALL_IDS" | grep -vE '^(GO-2026-4887|GO-2026-4883|GO-2026-4865|GO-2026-4869|GO-2026-4870|GO-2026-4946|GO-2026-4947)$' || true)
88+
KNOWN='GO-2026-4887|GO-2026-4883|GO-2026-4865|GO-2026-4869|GO-2026-4870|GO-2026-4946|GO-2026-4947|GO-2026-4918|GO-2026-4945|GO-2026-5013|GO-2026-5015|GO-2026-5017|GO-2026-5018|GO-2026-5019|GO-2026-5020|GO-2026-5021|GO-2026-5026'
89+
ACTIONABLE_IDS=$(echo "$ALL_IDS" | grep -vE "^($KNOWN)$" || true)
8390
for ID in $ACTIONABLE_IDS; do
8491
echo "::error file=go.mod,line=1,title=Unfixed vulnerability::$ID — add to exclusion list or upgrade the affected dependency"
8592
done
86-
ACTIONABLE=$(echo "$OUTPUT" | grep -E '^Vulnerability #[0-9]+:' | grep -vE 'GO-2026-4887|GO-2026-4883|GO-2026-4865|GO-2026-4869|GO-2026-4870|GO-2026-4946|GO-2026-4947' || true)
93+
ACTIONABLE=$(echo "$OUTPUT" | grep -E '^Vulnerability #[0-9]+:' | grep -vE "$KNOWN" || true)
8794
if [ -n "$ACTIONABLE" ]; then
8895
exit 1
8996
fi

‎CLAUDE.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ mantle repos add <name> --url <url> --credential <cred> # Register a GitOps sou
8484
mantle repos list # List registered repos with last-sync status
8585
mantle repos status <name> # Show detailed repo status
8686
mantle repos remove <name> -y # Unregister a repo (requires --yes)
87+
mantle repos sync <name> # Force an immediate sync of a registered repo
8788
mantle run <wf> --values f.yaml # Run with a values file (inputs + env overrides)
8889
mantle run <wf> --env <name> # Run against a stored named environment
8990
mantle plan <wf> --env <name> # Plan; appends resolved inputs/env with source
@@ -114,7 +115,7 @@ git_sync:
114115
115116
Credentials of type `git` accept `token` (for HTTPS), `ssh_key` (for SSH), and optional `username`. At least one of `token` or `ssh_key` is required.
116117

117-
> Plan A ships the repo registry and CLI only. The sync engine that consumes this block — pulling from the sidecar, validating, and applying workflow YAML — ships in the next milestone. Registering a repo today has no runtime effect yet beyond persistence.
118+
> Plan A shipped the repo registry and CLI. Plan B ships the sync engine — when `mantle serve` runs, it reconciles this block into `git_repos` rows and then polls each enabled `auto_apply: true` repo every `poll_interval`, applying any changed workflow YAML. Force an immediate sync with `mantle repos sync <name>`.
118119

119120
## License
120121

‎packages/engine/go.mod‎

Lines changed: 17 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ require (
1414
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.41.4
1515
github.com/coreos/go-oidc/v3 v3.17.0
1616
github.com/docker/docker v28.5.2+incompatible
17+
github.com/emersion/go-imap/v2 v2.0.0-beta.8
18+
github.com/go-git/go-git/v5 v5.18.0
1719
github.com/golang-jwt/jwt/v5 v5.3.0
1820
github.com/google/cel-go v0.27.0
1921
github.com/googleapis/gax-go/v2 v2.15.0
@@ -26,6 +28,7 @@ require (
2628
github.com/testcontainers/testcontainers-go v0.41.0
2729
github.com/testcontainers/testcontainers-go/modules/postgres v0.41.0
2830
golang.org/x/oauth2 v0.35.0
31+
golang.org/x/term v0.41.0
2932
golang.org/x/time v0.15.0
3033
gopkg.in/yaml.v3 v3.0.1
3134
)
@@ -42,10 +45,8 @@ require (
4245
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.2.0 // indirect
4346
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
4447
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
45-
github.com/KyleBanks/depth v1.2.1 // indirect
4648
github.com/Microsoft/go-winio v0.6.2 // indirect
47-
github.com/PuerkitoBio/purell v1.1.1 // indirect
48-
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect
49+
github.com/ProtonMail/go-crypto v1.1.6 // indirect
4950
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
5051
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7 // indirect
5152
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 // indirect
@@ -65,30 +66,31 @@ require (
6566
github.com/beorn7/perks v1.0.1 // indirect
6667
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
6768
github.com/cespare/xxhash/v2 v2.3.0 // indirect
69+
github.com/cloudflare/circl v1.6.3 // indirect
6870
github.com/containerd/errdefs v1.0.0 // indirect
6971
github.com/containerd/errdefs/pkg v0.3.0 // indirect
7072
github.com/containerd/log v0.1.0 // indirect
7173
github.com/containerd/platforms v0.2.1 // indirect
7274
github.com/cpuguy83/dockercfg v0.3.2 // indirect
75+
github.com/cyphar/filepath-securejoin v0.4.1 // indirect
7376
github.com/davecgh/go-spew v1.1.1 // indirect
7477
github.com/distribution/reference v0.6.0 // indirect
7578
github.com/docker/go-connections v0.6.0 // indirect
7679
github.com/docker/go-units v0.5.0 // indirect
7780
github.com/ebitengine/purego v0.10.0 // indirect
78-
github.com/emersion/go-imap/v2 v2.0.0-beta.8 // indirect
7981
github.com/emersion/go-message v0.18.2 // indirect
8082
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
83+
github.com/emirpasic/gods v1.18.1 // indirect
8184
github.com/felixge/httpsnoop v1.0.4 // indirect
8285
github.com/fsnotify/fsnotify v1.9.0 // indirect
86+
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
87+
github.com/go-git/go-billy/v5 v5.8.0 // indirect
8388
github.com/go-jose/go-jose/v4 v4.1.3 // indirect
8489
github.com/go-logr/logr v1.4.3 // indirect
8590
github.com/go-logr/stdr v1.2.2 // indirect
8691
github.com/go-ole/go-ole v1.2.6 // indirect
87-
github.com/go-openapi/jsonpointer v0.19.5 // indirect
88-
github.com/go-openapi/jsonreference v0.19.6 // indirect
89-
github.com/go-openapi/spec v0.20.4 // indirect
90-
github.com/go-openapi/swag v0.19.15 // indirect
9192
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
93+
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
9294
github.com/google/s2a-go v0.1.9 // indirect
9395
github.com/google/uuid v1.6.0 // indirect
9496
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
@@ -97,12 +99,12 @@ require (
9799
github.com/jackc/pgpassfile v1.0.0 // indirect
98100
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
99101
github.com/jackc/puddle/v2 v2.2.2 // indirect
100-
github.com/josharian/intern v1.0.0 // indirect
102+
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
103+
github.com/kevinburke/ssh_config v1.2.0 // indirect
101104
github.com/klauspost/compress v1.18.4 // indirect
102105
github.com/kylelemons/godebug v1.1.0 // indirect
103106
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
104107
github.com/magiconair/properties v1.8.10 // indirect
105-
github.com/mailru/easyjson v0.7.6 // indirect
106108
github.com/mfridman/interpolate v0.0.2 // indirect
107109
github.com/moby/docker-image-spec v1.3.1 // indirect
108110
github.com/moby/go-archive v0.2.0 // indirect
@@ -116,6 +118,7 @@ require (
116118
github.com/opencontainers/go-digest v1.0.0 // indirect
117119
github.com/opencontainers/image-spec v1.1.1 // indirect
118120
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
121+
github.com/pjbgf/sha1cd v0.3.2 // indirect
119122
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
120123
github.com/pkg/errors v0.9.1 // indirect
121124
github.com/pmezard/go-difflib v1.0.0 // indirect
@@ -124,17 +127,19 @@ require (
124127
github.com/prometheus/common v0.66.1 // indirect
125128
github.com/prometheus/procfs v0.19.2 // indirect
126129
github.com/sagikazarmark/locafero v0.11.0 // indirect
130+
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
127131
github.com/sethvargo/go-retry v0.3.0 // indirect
128132
github.com/shirou/gopsutil/v4 v4.26.2 // indirect
129133
github.com/sirupsen/logrus v1.9.3 // indirect
134+
github.com/skeema/knownhosts v1.3.1 // indirect
130135
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
131136
github.com/spf13/afero v1.15.0 // indirect
132137
github.com/spf13/cast v1.10.0 // indirect
133138
github.com/spf13/pflag v1.0.10 // indirect
134139
github.com/subosito/gotenv v1.6.0 // indirect
135-
github.com/swaggo/swag v1.16.6 // indirect
136140
github.com/tklauser/go-sysconf v0.3.16 // indirect
137141
github.com/tklauser/numcpus v0.11.0 // indirect
142+
github.com/xanzy/ssh-agent v0.3.3 // indirect
138143
github.com/yusufpapurcu/wmi v1.2.4 // indirect
139144
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
140145
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
@@ -147,18 +152,15 @@ require (
147152
go.yaml.in/yaml/v3 v3.0.4 // indirect
148153
golang.org/x/crypto v0.49.0 // indirect
149154
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa // indirect
150-
golang.org/x/mod v0.33.0 // indirect
151155
golang.org/x/net v0.52.0 // indirect
152156
golang.org/x/sync v0.20.0 // indirect
153157
golang.org/x/sys v0.42.0 // indirect
154-
golang.org/x/term v0.41.0 // indirect
155158
golang.org/x/text v0.35.0 // indirect
156-
golang.org/x/tools v0.42.0 // indirect
157159
google.golang.org/api v0.247.0 // indirect
158160
google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect
159161
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 // indirect
160162
google.golang.org/genproto/googleapis/rpc v0.0.0-20260217215200-42d3e9bedb6d // indirect
161163
google.golang.org/grpc v1.79.3 // indirect
162164
google.golang.org/protobuf v1.36.11 // indirect
163-
gopkg.in/yaml.v2 v2.4.0 // indirect
165+
gopkg.in/warnings.v0 v0.1.2 // indirect
164166
)

0 commit comments

Comments
 (0)