Add proxy git client for repository cloning via Semaphore server

[Copilot]

This PR implements a new "proxy git client" feature that allows Semaphore runners to clone repositories via the Semaphore server instead of directly from git servers. This addresses scenarios where runners cannot access git servers directly due to network restrictions, firewalls, or security policies.

How it works

1. Server-side cloning: When git_client: "proxy_git" is configured, the Semaphore server clones repositories using standard git clients (cmd_git or go_git)
2. Archive creation: The server creates a compressed tar.gz archive of the cloned repository
3. API transfer: The archive is sent to runners via a new internal API endpoint (/api/internal/repositories/archive)
4. Local extraction: Runners extract the archive to their local filesystem

Configuration

Enable proxy mode by setting the git client type in your Semaphore configuration:

{
  "git_client": "proxy_git"
}

Or via environment variable:

SEMAPHORE_GIT_CLIENT=proxy_git

No runner configuration changes are required - they automatically use proxy mode when the server is configured for it.

Key features

• Transparent operation: Existing workflows continue to work without modification
• Backward compatible: Current cmd_git and go_git configurations remain unchanged
• Centralized access: Git authentication and access control handled server-side
• Network isolation: Runners only need HTTP access to the Semaphore server

Use cases

• Runners deployed in isolated networks without direct git access
• Corporate environments with restrictive firewall policies
• Ephemeral runners that need minimal network dependencies
• Centralized git access control and auditing

Implementation details

• Added ProxyGitClientId constant and validation rules
• Implemented ProxyGitClient with full GitClient interface compatibility
• Created /api/internal/repositories/archive endpoint with runner authentication
• Updated GitClientFactory to support the new proxy mode
• Added comprehensive test suite and documentation

The implementation maintains full backward compatibility while providing a robust solution for restricted network environments.

Fixes #3259.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

The best way to fix this problem is to ensure that any data containing sensitive fields—such as a Git URL that may have username:password@hostname—is either omitted, obfuscated, or carefully redacted before being logged. Specifically, when logging errors related to repository operations, we should avoid logging sensitive values, or sanitize the error and associated fields before logging.

In this case, the problematic log line is:

log.WithError(err).WithFields(log.Fields{
	"git_url":    req.GitURL,
	"git_branch": req.GitBranch,
}).Error("Failed to clone repository")

The err itself may include the sensitive password, and req.GitURL is known-safe (doesn't include credentials), but downstream errors (originating from e.g., constructed clone URLs with credentials) could inject sensitive data.

To fix:

• Sanitize err.Error() before logging:
  • Redact credentials from any URLs embedded in the error string prior to logging.
• Alternatively, log only a generic error message without the actual error details, or explicitly check and redact known patterns.

Implementation:

• Introduce a helper to sanitize sensitive data from error messages.
• Before logging, pass the error through this helper, and log only the sanitized error message.
• This should be applied to the affected log statement in api/runners/runners.go.

---

To prevent leaking passwords in log messages, especially when logging file paths that are derived from URLs containing credentials, we should sanitize such inputs before logging. Specifically, in the context provided, whenever logging a file path (e.g., a requirements file path) that was constructed from a potentially sensitive repository URL, we need to remove any embedded credentials before writing to the log.

The best way to achieve this, without altering existing functionality, is to introduce a helper function that strips credentials from URLs. For example, we can use Go's net/url package (standard library, well-known) to parse the URL and drop the User field, then reconstruct the sanitized URL. We'll apply this sanitization right before logging the filename/path.

Concrete steps:

• In db_lib/AnsibleApp.go, modify logging statements that log a requirements file path, specifically the call to t.Log(requirementsFilePath + " has no changes. Skip galaxy install process.\n") in function installGalaxyRequirementsFile. Sanitize the path/URL for any embedded credentials.
• (Optional/future-proofing) If there are other logging statements in the flow where URLs with potential credentials are logged, apply similar sanitization.
• Define a helper method to sanitize embedded credentials from URLs/paths. Use Go's net/url package for parsing and reconstructing.
• Add the net/url import if not already present.

No changes are needed in the generic logger (simpleLogger), as the fix should be applied at the source of the potentially problematic log call.

---

The logging call in func (l *simpleLogger) LogCmd(cmd *exec.Cmd) in api/runners/runners.go currently logs the entire cmd object. Since command arguments may contain sensitive credentials (e.g., in URLs), we should redact credentials before logging, or log only safe, non-sensitive portions of the command. The best fix is to sanitize command arguments before logging.

Approach:

• Write a helper function in api/runners/runners.go to redact any authentication information in URLs, such as user:password@host constructs.
• In LogCmd, prepare a string version of the command and apply the sanitization function to all arguments before building/logging.
• Only make changes inside the shown runners.go code.
• Add helper function for redacting passwords from URLs using regular expressions.
• Change LogCmd so that, instead of logging cmd verbatim, it logs the executable and the sanitized arguments.

To remediate this vulnerability, strictly validate and constrain all filesystem path manipulations derived from user input:

• Before using targetPath, ensure that it is normalized (with filepath.Abs) and that it remains contained within a designated safe base directory (e.g., a project temp dir, as already present in parts of the codebase).
• Only allow repository locations inside intended directories.
• Implement a function (isPathWithinBase(base, target)) to verify that the absolute targetPath resides under the allowed base directory.
• Reject or error if the check fails, and ideally log this incident for audit.

Required Changes:

• In db_lib/ProxyGitClient.go, in the Clone function:
  • Before creating targetPath, resolve and validate it.
  • Use a helper to compute and enforce containment of targetPath in a safe base.
  • If invalid, return an error.
  • Add the helper within the same file for context, and add necessary imports.
• Use filepath.Abs and string prefix checks to guard traversal or absolute path attacks.

---

The core issue is that user-controlled input can result in arbitrary file paths. The correct solution is to ensure that any file or directory accessed based on user input is verified to be within an allowed base directory (the "safe folder"). This can be done by resolving the resulting path to its absolute form and ensuring that it starts with the base directory path (similarly to the example in the background).

There are two main avenues:

• For repository types that treat GitURL as a pathname (e.g., local repositories), ensure that only expected subdirectories of a trusted base directory are allowed.
• For other repository types, ensure any derived paths are validated before any file operation is performed.

Thus, in the methods that interact with paths constructed from user input, insert a path validation routine that:

• Resolves the path to an absolute canonical path.
• Ensures this canonical path starts with the allowed directory root.
• If the validation fails, return an error or handle accordingly.

Specifically:

• In ProxyGitClient.GetLastCommitMessage and ProxyGitClient.GetLastCommitHash, add validation on gitDir (the path to .git).
• Validation logic should be extracted into a helper function for clarity and reuse.
• Because only common packages can be used, use filepath.Abs and filepath.Clean for canonicalization.
• Determine the base safe directory (could be the result of util.Config.GetProjectTmpDir()), and ensure all accessed paths are inside this directory.

You only need to modify the shown code in db_lib/ProxyGitClient.go, so insert the helper and use it where needed.

---

To fix the problem, all file paths derived from user-controlled data must be validated before they are used in filesystem operations. Specifically, anything returned by GetFullPath() for a GitRepository must be checked to confirm it resides within a safe base directory. This can be performed by resolving the absolute path, and ensuring it is a subdirectory of a known safe location (for example, util.Config.GetProjectTmpDir(r.Repository.ProjectID)).

The best approach is to add a validation step before any filesystem access using these paths. Since the code constructs .git subpaths using filepath.Join(r.GetFullPath(), ".git"), the checks should be performed immediately prior to filesystem operations in affected methods such as GetLastCommitHash, GetLastCommitMessage, and Pull methods in ProxyGitClient (db_lib/ProxyGitClient.go). For more robust defense, the GetFullPath() function itself should avoid returning unsafe paths, but given the code shown, per-use validation provides defences closest to the vulnerability.

To implement the changes, import "path/filepath" and "strings" if needed, and before any call to os.Stat, os.RemoveAll, etc. with derived paths, ensure:

• The resolved absolute path starts with the literal absolute path of the corresponding safe working directory.
• If not, return an error and avoid performing the operation.

This does not change existing functionality, except to reject unsafe requests.

---

In general terms:
You must ensure that all file system paths derived from untrusted data, especially during extraction of archives (tar, zip, etc.), are validated to prevent path traversal. This means, for every extracted entry, verifying that its target path is actually within the intended extraction directory after all symlink and relative resolution.

Best fix for the given code:
In extractArchive, after constructing targetFilePath, resolve its absolute path and ensure it starts with the absolute path of targetPath plus a path separator (to prevent /some/dir_traversal_is_here/evil matching /some/dir). Alternatively, use Go's filepath.Rel to compute the path relative to targetPath and reject any that resolve to a path outside (filepath.Rel returns a path starting with .. in such cases).

Precisely, in db_lib/ProxyGitClient.go, in the extractArchive method:

• After calculating targetFilePath, get its absolute path and verify it's within the target directory.
• If it's not, skip extraction.
• You will need to add logic for this check (and possibly import "path/filepath" and/or "strings" for compatibility).
• Only filter entries after joining with targetPath, and do not rely only on presence of ".." in the cleaned header name.

To robustly fix uncontrolled path usage during tar archive extraction, we should validate that every output path (after joining the cleaned header name with the destination directory) stays strictly within the intended base directory. This is achieved by:

• Resolving the absolute path of both the target base directory (targetPath) and the destination file (targetFilePath).
• Ensuring that the absolute path of the destination file starts with the absolute base directory. If not, skip extraction for that entry.
• Additionally, skip header names that are absolute paths.

Implement these checks inside the extraction loop in extractArchive (db_lib/ProxyGitClient.go).
No additional imports are needed, as filepath.Abs, filepath.Join, and strings.HasPrefix are already imported.

Regions to change:

• extractArchive method in db_lib/ProxyGitClient.go, specifically after computing targetFilePath, before attempting to create the directory or file.

---

To fix this vulnerability, we must ensure that files extracted from the archive are only written within the intended extraction directory, targetPath. The robust and common approach for this is as follows:

• After joining targetPath and cleanPath to get targetFilePath, compute the absolute path for both targetPath and targetFilePath.
• Then check (e.g., with strings.HasPrefix or filepath.Rel) that the absolute targetFilePath is within the absolute, canonicalized targetPath.
• Only allow file operations to proceed if this condition is satisfied; otherwise, skip the file, and ideally, log a warning.
• Additionally, reject files whose archive path (header.Name) is absolute (i.e., starts with / or drive letter).
• These checks are best performed in the file: db_lib/ProxyGitClient.go, function extractArchive (lines 173–236), in the extraction loop.

Only code within the scope of this function will be changed, following the shown context.

Required changes:

• Compute absolute path for targetPath at the start of extractArchive().
• In the extraction loop, after calculating targetFilePath, compute its absolute path.
• Ensure targetFilePathAbs is within targetPathAbs. If not, skip extraction for this file, and possibly log the event.
• Additionally, skip extraction for archive entries whose names are absolute paths.
• The required imports are already present in the file. No new dependencies are needed.