Add reusable TruffleHog secret scanning workflow #73

isaiah-grafana · 2025-10-01T01:02:17Z

This pull request introduces a comprehensive overhaul of secret scanning infrastructure for the repository, focusing on both CI/CD and developer workflows. The main improvements is the addition of a reusable GitHub Actions for TruffleHog-based secret scanning, and Renovate configuration to keep TruffleHog dependencies up to date.

Secret scanning infrastructure:

Added .github/workflows/org-required-trufflehog.yml to enforce TruffleHog secret scanning on all pull requests and pushes to main, using a reusable workflow and sensible defaults for organization-wide adoption.
Introduced .github/workflows/reusable-trufflehog.yml, a parameterized, reusable workflow for TruffleHog scanning that supports custom runner selection, configurable failure policies, PR commenting, artifact upload, and detailed reporting.

Dependency management:

Updated renovate.json to add a custom manager for tracking TruffleHog versions and digests in GitHub Actions workflows.

- Migrated from shared-workflows repo for better organization - Supports filesystem and git history scanning - Configurable scan scope (full-repo, changed-files, both) - Optimized with entropy filtering and exclusions - Posts PR comments and creates GitHub status checks - Ready for org rulesets integration

- Allows organizations to specify custom runners - Maintains backward compatibility with ubuntu-latest default - Enhances workflow flexibility for org-wide deployment

- Creates org-required-trufflehog.yml that calls the reusable workflow - Optimized settings for org-wide deployment (changed-files, lenient) - Designed to be enforced via GitHub organization rulesets - Maintains separation: reusable workflow for flexibility, required workflow for enforcement

- Add JSON validation before processing with jq - Handle invalid JSON gracefully with fallback values - Add error handling for all jq commands - Filter empty lines from NDJSON before conversion - Prevent 'Cannot index array with string' errors - Ensure workflow continues even with malformed TruffleHog output

- Changed from 'Organization Required - TruffleHog Secret Scan' - More user-friendly name for GitHub Actions UI - Cleaner appearance in org rulesets and status checks

Removed: - Debug/test code checking for test-secrets.txt - Excessive verbose logging and debug output - Unused verify-secrets input parameter - Redundant echo statements and file size checks - Overly verbose comments Benefits: - Cleaner, more professional output - Faster execution (less logging overhead) - Easier to read and maintain - Reduced noise in GitHub Actions logs

- Add scan scope and type to initial log message - Helps with debugging and understanding workflow execution - Minor improvement for better visibility into scan parameters

- Change comment identifier from workflow/job specific to static identifier - Use 'trufflehog-secret-scan-comment' instead of dynamic workflow names - Ensures all TruffleHog workflows can minimize each other's comments - Fixes issue where multiple TruffleHog workflows create duplicate comments

- Org rulesets are already enforcing TruffleHog scans - Prevents duplicate workflow executions - Keeps only the reusable workflow for flexibility - Eliminates duplicate status checks in PRs

- Org rulesets weren't set up yet, so removing this broke TruffleHog scans - Restore the workflow to maintain security scanning - Will remove this once org rulesets are properly configured - Keeps TruffleHog protection active in the meantime

- Eliminates duplicate check runs that cause 'No check run found' errors - Workflow job status already provides all necessary information - Prevents confusion between automatic job checks and custom checks - Fixes check run ID mismatch issues when clicking on status checks

- Contains fake/invalid secrets with common patterns - Designed to trigger TruffleHog detectors for testing - Safe for testing - no real credentials - Includes AWS, GitHub, Slack, API keys, and other common patterns

- Add comment to ensure file shows as changed in PR - Should trigger TruffleHog scan on changed files - Contains multiple fake secret patterns for testing

- Temporarily change from changed-files to full-repo - Ensures TruffleHog scans the entire repository including test secrets - Should detect the fake secrets in test-secrets-sample.txt

- Lower entropy filtering from 3.0 to 2.0 for better detection - Remove --filter-unverified flag to catch more potential secrets - Update test secrets with more realistic patterns and higher entropy - Add JWT token and improve existing secret patterns - Should now detect the test secrets properly

- Show .txt files found in directory - Display filesystem scan results file size - Help diagnose why test-secrets-sample.txt isn't being detected - Temporary debugging to understand scan behavior

- Remove entropy filtering that was blocking detection - Add 'unknown' results to catch all potential secrets - Remove --filter-unverified flag completely - Simplified scan commands to be more aggressive - Should now detect test secrets in .txt files

- Updated TRUFFLEHOG_VERSION to use ratchet format - Follows same pattern as zizmor workflow - Renovate will automatically update TruffleHog to latest versions - Format: # ratchet:trufflesecurity/[email protected]

- Changed from ratchet format to proper Renovate datasource format - Uses 'datasource=github-releases depName=trufflesecurity/trufflehog' - Matches the pattern used in shared-workflows/reusable-zizmor.yml - Renovate will now properly track TruffleHog GitHub releases

- Added regex matcher to detect '# renovate: datasource=...' patterns - Matches the same pattern used in shared-workflows - Will automatically update TRUFFLEHOG_VERSION when new releases are available - Enables automated dependency management for security tools

Copilot

Pull Request Overview

Introduces organization-wide and reusable TruffleHog secret scanning workflows plus supporting pre-commit script and Renovate configuration. Key changes center on adding a parameterized reusable workflow, an org-enforced wrapper workflow, an enhanced local/pre-commit scanning script, and a Renovate custom manager for maintaining TruffleHog versions.

Added reusable GitHub Actions workflow for flexible TruffleHog scanning (with reporting and failure policies)
Added org-required wrapper workflow invoking the reusable workflow with opinionated defaults
Enhanced pre-commit TruffleHog script and added Renovate regex manager to track versions/digests

Reviewed Changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.

File	Description
renovate.json	Adds custom regex manager to let Renovate track TruffleHog image/tool versions in workflow files.
pre-commit/trufflehog.sh	Replaces simple invocation with configurable, JSON-processing scanner including custom detectors and exclusion logic.
.github/workflows/reusable-trufflehog.yml	New reusable workflow implementing scoped or full-repo scans, PR commenting, artifact generation, and failure policy controls.
.github/workflows/org-required-trufflehog.yml	Org-level wrapper invoking the reusable workflow with standard org-wide defaults.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

pre-commit/trufflehog.sh

.github/workflows/reusable-trufflehog.yml

pre-commit/trufflehog.sh

- Removed all the complex Grafana-specific patterns and configuration - Removed custom canary token detection and alerts - Removed colored output and verbose logging - Back to the clean, simple Docker-based script - Focuses on core TruffleHog functionality without extras

- Updated from 3.82.13 to 3.89.2 (latest available) - Includes several months of security fixes and improvements - Renovate will handle future updates automatically

Copilot

Pull Request Overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 7 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

.github/workflows/reusable-trufflehog.yml

pre-commit/trufflehog.sh

.github/workflows/reusable-trufflehog.yml

Co-authored-by: Copilot <[email protected]>

- Replace unpinned install script with direct binary download from GitHub releases - Eliminates risk of mutable main branch script being compromised - Downloads specific versioned binary directly from trusted GitHub releases - More secure and deterministic installation process - Addresses supply chain security best practices

- Replace invalid hash with correct commit hash for v4.4.0 - Fixes workflow execution error where action could not be found - Uses proper SHA commit hash for security pinning

Co-authored-by: Copilot <[email protected]>

jamesc-grafana

LGTM, just a couple of nits

.github/workflows/org-required-trufflehog.yml

renovate.json

Co-authored-by: James C <[email protected]>

- Include pre-commit/*.sh files in Renovate file matching - Will now automatically update TruffleHog versions in shell scripts - Ensures consistent version management across all TruffleHog usage

github-actions · 2025-10-01T13:45:34Z

TruffleHog Scan Results

No secrets detected in this PR.

isaiah-grafana added 3 commits September 28, 2025 17:11

Add runs-on input parameter for org ruleset flexibility

84722fd

- Allows organizations to specify custom runners - Maintains backward compatibility with ubuntu-latest default - Enhances workflow flexibility for org-wide deployment