Skip to content

Commit cb8152d

Browse files
Share Linux binary acquisition between run and vendoring.
Extract internal/binary for run and --vendor-fullsend-binary resolution, wire --fullsend-binary on install/setup, and use title+body commit messages for vendored binary upload and stale cleanup. Signed-off-by: Barak Korren <bkorren@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 93be032 commit cb8152d

22 files changed

Lines changed: 1320 additions & 641 deletions

docs/guides/dev/cli-internals.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,19 @@ Install: process 1→7 (forward)
216216
Uninstall: process 7→1 (reverse)
217217
```
218218

219-
Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline rather than through `VendorBinaryLayer`.
219+
Per-repo mode does not use the layer stack — it runs the same phases inline in `runPerRepoInstall()` and `runGitHubSetupPerRepo()` since there's no need for composable uninstall ordering with a single repo. Binary vendoring (when `--vendor-fullsend-binary` is set) and stale binary cleanup are handled inline or via shared helpers; per-org mode uses `VendorBinaryLayer`.
220+
221+
### Binary acquisition (`internal/binary`)
222+
223+
Linux binary resolution for `fullsend run` and vendoring lives in `internal/binary`:
224+
225+
| Function | Policy |
226+
|----------|--------|
227+
| `ResolveForRun` | Release download (if released CLI) → cross-compile → latest release |
228+
| `ResolveForVendor` | Cross-compile → matching release (released CLI only) → fail (no latest) |
229+
| `ResolveExplicit` | Validate linux/{arch} ELF for `--fullsend-binary` |
230+
231+
Vendoring commit messages use title + body (upload and stale delete). `admin analyze` reports stale vendored binaries at `bin/fullsend` or `.fullsend/bin/fullsend` without install-intent flags.
220232

221233
---
222234

docs/guides/getting-started/github-setup.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,9 +118,16 @@ fullsend github setup acme-corp \
118118
| `--app-set` | No | `fullsend-ai` | App set name prefix for GitHub Apps |
119119
| `--enroll-all` | No | `false` | Enroll all repositories without prompting (per-org only) |
120120
| `--enroll-none` | No | `false` | Skip enrollment without prompting (per-org only) |
121-
| `--vendor-fullsend-binary` | No | `false` | Build and upload the fullsend binary to the config repo for local dev testing (e.g., macOS with a Podman Linux VM) |
121+
| `--vendor-fullsend-binary` | No | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) |
122+
| `--fullsend-binary` | No | | Path to a Linux fullsend binary when vendoring (skips auto-resolution) |
122123
| `--dry-run` | No | `false` | Preview changes without making them |
123124

125+
### Vendoring the CLI binary
126+
127+
Same policy as [admin install](installation.md#vendoring-the-cli-binary): `--fullsend-binary` → checkout cross-compile → matching release (released CLI only) → fail. Per-repo setup now wires vendoring and stale-binary cleanup when the flag is off.
128+
129+
`fullsend admin analyze <org>` reports when a stale vendored binary is present (no install-intent flags on analyze).
130+
124131
## Per-repo setup
125132

126133
Per-repo mode bootstraps a single repository with a `.fullsend/` directory, shim workflow, and repo-level secrets:

docs/guides/getting-started/installation.md

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -256,7 +256,8 @@ The installer automatically provisions [Workload Identity Federation (WIF)](http
256256
| `--skip-mint-check` | `false` | Skip mint validation, GCP provisioning, and app setup; requires `--mint-url` |
257257
| `--enroll-all` | `false` | Enroll all repositories without prompting (per-org only) |
258258
| `--enroll-none` | `false` | Skip repository enrollment without prompting (per-org only) |
259-
| `--vendor-fullsend-binary` | `false` | Cross-compile and vendor the fullsend binary for development iteration |
259+
| `--vendor-fullsend-binary` | `false` | Resolve and upload a linux/amd64 fullsend binary for CI (see [Vendoring the CLI binary](#vendoring-the-cli-binary)) |
260+
| `--fullsend-binary` | | Path to a Linux fullsend binary to upload when `--vendor-fullsend-binary` is set (skips auto-resolution) |
260261

261262
The `--skip-mint-check` flag bypasses all mint validation, GCP provisioning, and app setup. It requires `--mint-url` to be set and only validates that the URL uses HTTPS. This is useful when the mint infrastructure is managed externally or you want to skip GCP API calls entirely.
262263

@@ -266,6 +267,25 @@ The installer automatically detects when the deployed mint function is up-to-dat
266267

267268
A single token mint can serve multiple GitHub organizations. See [Mint service administration — Multi-org setup](../infrastructure/mint-administration.md#multi-org-setup) for the complete multi-org workflow.
268269

270+
### Vendoring the CLI binary
271+
272+
Use `--vendor-fullsend-binary` to upload a linux/amd64 `fullsend` binary into the config repo (`bin/fullsend`) or per-repo path (`.fullsend/bin/fullsend`). CI workflows prefer this file over downloading from GitHub releases.
273+
274+
When the flag is set, the binary is resolved in this order:
275+
276+
1. **`--fullsend-binary <path>`** — upload that file (validated as linux/amd64 ELF)
277+
2. **Checkout build** — cross-compile from the fullsend module root (`go env GOMOD`), stamped `{version}-vendored`
278+
3. **Release fetch** — only if step 2 is unavailable **and** the running CLI is a released version (e.g. `0.4.0`); downloads the matching GitHub release (no `-vendored` suffix)
279+
4. **Fail** — dev CLI outside a checkout fails with a clear error (no “latest release” fallback)
280+
281+
When the flag is **off**, any existing vendored binary is removed so CI uses released versions.
282+
283+
**Notes:**
284+
285+
- Vendoring the CLI alone does not air-gap the full pipeline (OpenShell, gateway, sandbox image, upstream scaffold still download at runtime).
286+
- Release fallback requires network access at install time; CI consumes the uploaded file.
287+
- Works from any directory inside the module checkout (module root discovery via `GOMOD`).
288+
269289
### Merge enrollment PRs
270290

271291
If you chose to enroll repositories during install, the installer dispatches a workflow that creates an enrollment PR in each enrolled repo. These PRs add a shim workflow (`.github/workflows/fullsend.yaml`) that wires events to the agent pipeline.

e2e/admin/admin_test.go

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import (
2424
"github.com/fullsend-ai/fullsend/internal/config"
2525
"github.com/fullsend-ai/fullsend/internal/forge"
2626
gh "github.com/fullsend-ai/fullsend/internal/forge/github"
27+
"github.com/fullsend-ai/fullsend/internal/layers"
2728
)
2829

2930
// e2eEnv holds the shared state for an e2e test run.
@@ -617,3 +618,33 @@ func runUnenrollmentTest(t *testing.T, env *e2eEnv) {
617618
require.True(t, forge.IsNotFound(err), "shim should be removed from %s after unenrollment", testRepo)
618619
t.Log("Verified shim is gone")
619620
}
621+
622+
// TestVendorFromSubdirectory verifies that --vendor-fullsend-binary cross-compiles
623+
// when the CLI is run from a subdirectory inside the module (GOMOD discovery).
624+
func TestVendorFromSubdirectory(t *testing.T) {
625+
env := setupE2ETest(t)
626+
ctx := context.Background()
627+
628+
subdir := filepath.Join(moduleRoot(t), "internal", "cli")
629+
installArgs := []string{
630+
"admin", "install", env.org,
631+
"--skip-app-setup",
632+
"--skip-mint-check",
633+
"--mint-url", env.cfg.mintURL,
634+
"--app-set", e2eAppSet,
635+
"--enroll-none",
636+
"--vendor-fullsend-binary",
637+
}
638+
runCLIFromDir(t, env.binary, env.token, subdir, installArgs...)
639+
640+
_, err := env.client.GetFileContent(ctx, env.org, forge.ConfigRepoName, layers.VendoredBinaryPath)
641+
require.NoError(t, err, "vendored binary should exist at %s", layers.VendoredBinaryPath)
642+
643+
registerRepoCleanup(t, env.client, env.org, forge.ConfigRepoName)
644+
645+
runCLI(t, env.binary, env.token,
646+
"admin", "uninstall", env.org,
647+
"--yolo",
648+
"--app-set", e2eAppSet,
649+
)
650+
}

e2e/admin/testutil.go

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -260,19 +260,19 @@ func buildCLIBinary(t *testing.T) string {
260260
}
261261

262262
// runCLI executes the fullsend CLI with the given args, passing GITHUB_TOKEN.
263-
// The working directory is set to the module root so that --vendor-fullsend-binary
264-
// can find ./cmd/fullsend/ (same as a user running from the repo root).
263+
// By default the working directory is the module root. Use runCLIFromDir to
264+
// run from a subdirectory (GOMOD discovery makes this work for vendoring).
265265
func runCLI(t *testing.T, binary, token string, args ...string) string {
266-
t.Helper()
267-
t.Logf("[cli] fullsend %s", strings.Join(args, " "))
266+
return runCLIFromDir(t, binary, token, moduleRoot(t), args...)
267+
}
268268

269-
modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output()
270-
if err != nil {
271-
t.Fatalf("finding module root for runCLI: %v", err)
272-
}
269+
// runCLIFromDir runs the CLI with cwd set to dir.
270+
func runCLIFromDir(t *testing.T, binary, token, dir string, args ...string) string {
271+
t.Helper()
272+
t.Logf("[cli] fullsend %s (cwd=%s)", strings.Join(args, " "), dir)
273273

274274
cmd := exec.Command(binary, args...)
275-
cmd.Dir = strings.TrimSpace(string(modRoot))
275+
cmd.Dir = dir
276276
cmd.Env = append(os.Environ(), "GITHUB_TOKEN="+token, "CI=true")
277277
out, runErr := cmd.CombinedOutput()
278278
output := string(out)
@@ -283,6 +283,15 @@ func runCLI(t *testing.T, binary, token string, args ...string) string {
283283
return output
284284
}
285285

286+
func moduleRoot(t *testing.T) string {
287+
t.Helper()
288+
modRoot, err := exec.Command("go", "list", "-m", "-f", "{{.Dir}}").Output()
289+
if err != nil {
290+
t.Fatalf("finding module root: %v", err)
291+
}
292+
return strings.TrimSpace(string(modRoot))
293+
}
294+
286295
// retryOnNotFound retries an operation up to maxAttempts times with linear
287296
// backoff when it returns a not-found error (GitHub eventual consistency).
288297
func retryOnNotFound(ctx context.Context, maxAttempts int, fn func() error) error {

internal/binary/acquire.go

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
package binary
2+
3+
import (
4+
"fmt"
5+
"os"
6+
"path/filepath"
7+
)
8+
9+
// Source identifies how a Linux fullsend binary was obtained.
10+
type Source int
11+
12+
const (
13+
SourceExplicitPath Source = iota
14+
SourceCheckoutBuild
15+
SourceReleaseDownload
16+
)
17+
18+
// AcquireResult holds the path to an acquired binary and metadata for callers.
19+
type AcquireResult struct {
20+
TmpDir string // caller must RemoveAll when non-empty
21+
Path string
22+
Source Source
23+
}
24+
25+
// ResolveExplicit validates that path is a Linux ELF for arch.
26+
func ResolveExplicit(path, arch string) error {
27+
return ValidateLinuxBinary(path, arch)
28+
}
29+
30+
// ResolveForRun obtains a Linux binary using the run policy:
31+
// release download (if released) → cross-compile → latest release.
32+
func ResolveForRun(version, arch string) (AcquireResult, error) {
33+
tmpDir, err := os.MkdirTemp("", "fullsend-linux-*")
34+
if err != nil {
35+
return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err)
36+
}
37+
binaryPath := filepath.Join(tmpDir, "fullsend")
38+
39+
// 1. Released version → download matching release asset.
40+
if IsReleasedVersion(version) {
41+
fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch)
42+
if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil {
43+
fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch)
44+
return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil
45+
} else {
46+
fmt.Fprintf(os.Stderr, "WARNING: release download failed: %v\n", dlErr)
47+
}
48+
}
49+
50+
// 2. Try cross-compilation (requires Go toolchain + module checkout).
51+
fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch)
52+
if ccErr := CrossCompile(CrossCompileOpts{
53+
Version: version,
54+
Arch: arch,
55+
DestPath: binaryPath,
56+
VersionStamp: "-crosscompiled",
57+
}); ccErr == nil {
58+
fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch)
59+
return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil
60+
} else {
61+
fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr)
62+
}
63+
64+
// 3. Last resort → download latest release.
65+
fmt.Fprintf(os.Stderr, "Downloading latest fullsend release for linux/%s...\n", arch)
66+
latestErr := DownloadLatestRelease(arch, binaryPath)
67+
if latestErr == nil {
68+
fmt.Fprintf(os.Stderr, "Downloaded latest fullsend for linux/%s\n", arch)
69+
return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil
70+
}
71+
fmt.Fprintf(os.Stderr, "WARNING: latest release download failed: %v\n", latestErr)
72+
73+
os.RemoveAll(tmpDir)
74+
return AcquireResult{}, fmt.Errorf("all strategies failed for linux/%s: provide --fullsend-binary or install Go toolchain", arch)
75+
}
76+
77+
// ResolveForVendor obtains a Linux binary using the vendoring policy:
78+
// cross-compile from checkout → matching release (released CLI only) → fail.
79+
// No latest-release fallback.
80+
func ResolveForVendor(version, arch string) (AcquireResult, error) {
81+
tmpDir, err := os.MkdirTemp("", "fullsend-linux-*")
82+
if err != nil {
83+
return AcquireResult{}, fmt.Errorf("creating temp dir: %w", err)
84+
}
85+
binaryPath := filepath.Join(tmpDir, "fullsend")
86+
87+
// 1. Cross-compile from checkout.
88+
fmt.Fprintf(os.Stderr, "Cross-compiling fullsend for linux/%s...\n", arch)
89+
if ccErr := CrossCompile(CrossCompileOpts{
90+
Version: version,
91+
Arch: arch,
92+
DestPath: binaryPath,
93+
VersionStamp: "-vendored",
94+
}); ccErr == nil {
95+
fmt.Fprintf(os.Stderr, "Cross-compiled fullsend for linux/%s\n", arch)
96+
return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceCheckoutBuild}, nil
97+
} else {
98+
fmt.Fprintf(os.Stderr, "WARNING: cross-compilation failed: %v\n", ccErr)
99+
}
100+
101+
// 2. Release fetch only for released CLI versions.
102+
if IsReleasedVersion(version) {
103+
fmt.Fprintf(os.Stderr, "Downloading fullsend %s for linux/%s from GitHub Release...\n", version, arch)
104+
if dlErr := DownloadRelease(version, arch, binaryPath); dlErr == nil {
105+
fmt.Fprintf(os.Stderr, "Downloaded fullsend for linux/%s\n", arch)
106+
return AcquireResult{TmpDir: tmpDir, Path: binaryPath, Source: SourceReleaseDownload}, nil
107+
} else {
108+
os.RemoveAll(tmpDir)
109+
return AcquireResult{}, fmt.Errorf("cross-compilation unavailable and release download failed for v%s: %w", version, dlErr)
110+
}
111+
}
112+
113+
os.RemoveAll(tmpDir)
114+
return AcquireResult{}, fmt.Errorf("cannot vendor binary: not in fullsend source tree and CLI version %s is a dev build — use --fullsend-binary, run from a checkout, or use a released CLI", version)
115+
}

internal/binary/crosscompile.go

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
package binary
2+
3+
import (
4+
"fmt"
5+
"os"
6+
"os/exec"
7+
"path/filepath"
8+
"strings"
9+
)
10+
11+
// CrossCompileOpts configures a cross-compilation build.
12+
type CrossCompileOpts struct {
13+
Version string // CLI version to embed (before stamp suffix)
14+
Arch string
15+
DestPath string
16+
VersionStamp string // e.g. "-vendored", "-crosscompiled", or ""
17+
}
18+
19+
// ModuleRoot returns the fullsend module root directory, or an error if not
20+
// inside a Go module checkout.
21+
func ModuleRoot() (string, error) {
22+
goPath, lookErr := exec.LookPath("go")
23+
if lookErr != nil {
24+
return "", fmt.Errorf("Go toolchain not found: %w", lookErr)
25+
}
26+
modRootCmd := exec.Command(goPath, "env", "GOMOD")
27+
modOutput, err := modRootCmd.Output()
28+
if err != nil {
29+
return "", fmt.Errorf("finding module root: %w", err)
30+
}
31+
modPath := strings.TrimSpace(string(modOutput))
32+
if modPath == "" || modPath == os.DevNull {
33+
return "", fmt.Errorf("not in a Go module")
34+
}
35+
return filepath.Dir(modPath), nil
36+
}
37+
38+
// CrossCompile builds a Linux fullsend binary and writes it to DestPath.
39+
// Requires the Go toolchain and a fullsend module checkout (go env GOMOD).
40+
func CrossCompile(opts CrossCompileOpts) error {
41+
goPath, lookErr := exec.LookPath("go")
42+
if lookErr != nil {
43+
return fmt.Errorf("Go toolchain not found — install Go or use a released version of fullsend: %w", lookErr)
44+
}
45+
46+
modRoot, err := ModuleRoot()
47+
if err != nil {
48+
return fmt.Errorf("not in a Go module — run from the fullsend source tree or use a released version: %w", err)
49+
}
50+
51+
versionLD := opts.Version + opts.VersionStamp
52+
buildCmd := exec.Command(goPath, "build",
53+
"-ldflags", fmt.Sprintf("-X github.com/fullsend-ai/fullsend/internal/cli.version=%s", versionLD),
54+
"-o", opts.DestPath,
55+
"./cmd/fullsend/",
56+
)
57+
buildCmd.Dir = modRoot
58+
buildCmd.Env = append(os.Environ(), "GOTOOLCHAIN=auto", "GOOS=linux", "GOARCH="+opts.Arch, "CGO_ENABLED=0")
59+
buildCmd.Stderr = os.Stderr
60+
if err := buildCmd.Run(); err != nil {
61+
return fmt.Errorf("cross-compiling for linux/%s: %w", opts.Arch, err)
62+
}
63+
return nil
64+
}

0 commit comments

Comments
 (0)