Skip to content

Commit bfc5a16

Browse files
Feature/ferret scan plugin (#233)
* feat: add ferret-scan community plugin for sensitive data detection Adds a new community scanner plugin that integrates ferret-scan for detecting sensitive data (PII, credentials, secrets) in source code and documents. Features: - Wraps ferret-scan CLI with SARIF output for ASH integration - Supports configurable confidence levels and check filtering - Bundled profiles: quick, debug, ci, security-audit, comprehensive - Version compatibility validation (>=1.0.0, <2.0.0) - Graceful handling of empty directories and missing dependencies Includes: - Plugin module with scanner, config, and documentation - 64 unit tests with full coverage - Test data fixtures - Community plugin registration in .ash_community_plugins.yaml - CI workflow updates for ferret-scan installation - Hugo documentation page for the plugin * remived ferret from .ash.yaml * updated dev readme * fixed logging options for plugin so as not to be confused with ash options. updated test data. updated docs. * suppressed false positives triggered by test data * fixed windows path separator issue in ferret-scan test * fixed false positive suppression. fixed ferret-scan config to catch only high confidence matches * updated documentation to reflect ferret-scan testing changes * changed test var to prevent triggering false positive * added a test/validation/hygeine script to improve build success * replaced false positives in docs * suppressed false positives in docs * communit plugins suppression did not work. Re-adding to ash.yaml * communit plugins suppression did not work. Re-adding to ash.yaml * fix: remove PyYAML dependency from validate_ferret_plugin.py Replace yaml.safe_load with stdlib-only line parser for reading suppressions from .ash_community_plugins.yaml. PyYAML is not available in the CI test environment where the validation script runs via subprocess (not uv run). Fixes TestFerretPluginValidation::test_validate_ferret_plugin_passes failing on all CI platforms with ModuleNotFoundError: No module named 'yaml'. * fix: eliminate SECRET-SECRET-KEYWORD trigger from DEVELOPMENT.md line 501 Split API_KEY from the assignment operator in the documentation example so the keyword + '=' pattern no longer triggers the SECRET-SECRET-KEYWORD rule. Removed the now-unnecessary suppression entry from the community plugins config. * Revert "fix: eliminate SECRET-SECRET-KEYWORD trigger from DEVELOPMENT.md line 501" This reverts commit 802d496. * fix: add ferret-scan suppressions to both .ash.yaml and community plugins config All ferret-related suppressions (DEVELOPMENT.md SECRET-SECRET-KEYWORD, ferret_scanner.py B404/B603/B607, generate_test_docx.py B311/HEX, conftest.py SECRET-SECRET-KEYWORD) now appear in both config files with line numbers for precision. * docs: update suppression guidance to reflect both config files Suppressions must be in both .ash.yaml and .ash_community_plugins.yaml. * fix: eliminate SECRET-SECRET-KEYWORD trigger from DEVELOPMENT.md:501 GitHub code scanning does not honor SARIF suppressions field — alerts appear regardless. Split API_KEY from assignment operator into separate inline code spans to break the detection pattern while keeping the documentation readable. Suppressions remain in both configs as defense in depth for ASH's own suppression system. * Bump version to 3.2.0 --------- Co-authored-by: Nikhil Prabhakar <> Co-authored-by: Rafael Pereyra <31078199+rafaelpereyra@users.noreply.github.com>
1 parent e4dc506 commit bfc5a16

28 files changed

Lines changed: 6536 additions & 47 deletions

File tree

.ash/.ash.yaml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,41 @@ global_settings:
3333
- path: tests/unit/plugin_modules/ash_aws_plugins/test_cloudwatch_logs_reporter*
3434
rule_id: SECRET-SECRET-KEYWORD
3535
reason: False positive, dummy test data and not an actual secret.
36+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/DEVELOPMENT.md
37+
rule_id: SECRET-SECRET-KEYWORD
38+
line_start: 501
39+
line_end: 501
40+
reason: Documentation example showing the SECRET-SECRET-KEYWORD detection pattern.
41+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
42+
rule_id: B404
43+
line_start: 9
44+
line_end: 9
45+
reason: Subprocess module required — ferret-scan is an external CLI tool invoked via subprocess with list args (no shell=True).
46+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
47+
rule_id: B603
48+
line_start: 410
49+
line_end: 435
50+
reason: Subprocess calls use list args (no shell=True) with validated inputs. ferret-scan binary path comes from shutil.which().
51+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
52+
rule_id: B607
53+
line_start: 410
54+
line_end: 435
55+
reason: Partial executable path resolved via shutil.which() in validate_plugin_dependencies(). Standard ASH scanner pattern.
56+
- path: scripts/generate_test_docx.py
57+
rule_id: B311
58+
line_start: 1
59+
line_end: 203
60+
reason: Standard random is intentional — generates synthetic test data with random.seed(42) for reproducibility, not used for security/crypto.
61+
- path: tests/unit/plugin_modules/ash_ferret_plugins/conftest.py
62+
rule_id: SECRET-SECRET-KEYWORD
63+
line_start: 146
64+
line_end: 146
65+
reason: False positive — SETTING = "placeholder" uses a neutral variable name.
66+
- path: scripts/generate_test_docx.py
67+
rule_id: SECRET-HEX-HIGH-ENTROPY-STRING
68+
line_start: 175
69+
line_end: 177
70+
reason: False positive — runtime-generated synthetic credit card numbers from seeded random for test data.
3671
- path: .github/workflows/run-ash-security-scan.yml
3772
rule_id: yaml.github-actions.security.run-shell-injection.run-shell-injection
3873
reason: Troubleshooting swapping from direct injection to evaluating interpolated env vars

.ash/.ash_community_plugins.yaml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,46 @@ fail_on_findings: true
55
ash_plugin_modules:
66
- automated_security_helper.plugin_modules.ash_snyk_plugins
77
- automated_security_helper.plugin_modules.ash_trivy_plugins
8+
- automated_security_helper.plugin_modules.ash_ferret_plugins
89
external_reports_to_include: []
910
global_settings:
1011
severity_threshold: MEDIUM
1112
suppressions:
13+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/DEVELOPMENT.md
14+
rule_id: SECRET-SECRET-KEYWORD
15+
line_start: 501
16+
line_end: 501
17+
reason: Documentation example showing the SECRET-SECRET-KEYWORD detection pattern.
18+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
19+
rule_id: B404
20+
line_start: 9
21+
line_end: 9
22+
reason: Subprocess module required — ferret-scan is an external CLI tool invoked via subprocess with list args (no shell=True).
23+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
24+
rule_id: B603
25+
line_start: 410
26+
line_end: 435
27+
reason: Subprocess calls use list args (no shell=True) with validated inputs. ferret-scan binary path comes from shutil.which().
28+
- path: automated_security_helper/plugin_modules/ash_ferret_plugins/ferret_scanner.py
29+
rule_id: B607
30+
line_start: 410
31+
line_end: 435
32+
reason: Partial executable path resolved via shutil.which() in validate_plugin_dependencies(). Standard ASH scanner pattern.
33+
- path: scripts/generate_test_docx.py
34+
rule_id: B311
35+
line_start: 1
36+
line_end: 203
37+
reason: Standard random is intentional — generates synthetic test data with random.seed(42) for reproducibility, not used for security/crypto.
38+
- path: tests/unit/plugin_modules/ash_ferret_plugins/conftest.py
39+
rule_id: SECRET-SECRET-KEYWORD
40+
line_start: 146
41+
line_end: 146
42+
reason: False positive — SETTING = "placeholder" uses a neutral variable name.
43+
- path: scripts/generate_test_docx.py
44+
rule_id: SECRET-HEX-HIGH-ENTROPY-STRING
45+
line_start: 175
46+
line_end: 177
47+
reason: False positive — runtime-generated synthetic credit card numbers from seeded random for test data.
1248
- path: tests/integration/cli/test_mcp_file_tracking_integration.py
1349
rule_id: SECRET-SECRET-KEYWORD
1450
reason: False positive.
@@ -172,6 +208,20 @@ scanners:
172208
severity_threshold: "MEDIUM"
173209
syft:
174210
enabled: false
211+
ferret-scan:
212+
enabled: true
213+
options:
214+
confidence_levels: "high"
215+
checks: "CREDIT_CARD,SECRETS,SSN,PASSPORT"
216+
recursive: true
217+
use_default_config: false
218+
exclude_patterns:
219+
- ".venv"
220+
- "tests/test_data"
221+
- ".ash/ash_output"
222+
- ".git"
223+
- "__pycache__"
224+
- "*.pyc"
175225
trivy-repo:
176226
enabled: true
177227
options:

.github/workflows/ash-repo-scan-validation.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,14 @@ jobs:
202202
exit 1
203203
fi
204204
205+
# Install ferret-scan (cross-platform via pip)
206+
echo "Installing ferret-scan..."
207+
pip install ferret-scan
208+
209+
# Verify ferret-scan installation
210+
ferret-scan --version
211+
echo "ferret-scan installation completed!"
212+
205213
echo "Community plugin tools installation completed successfully!"
206214
207215
# Call the appropriate workflow based on matrix.method

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -91,15 +91,15 @@ ASH v3 integrates multiple open-source security tools as scanners. Tools like Ba
9191
curl -sSfL https://astral.sh/uv/install.sh | sh
9292

9393
# Create an alias for ASH
94-
alias ash="uvx git+https://github.com/awslabs/automated-security-helper.git@v3.1.13"
94+
alias ash="uvx git+https://github.com/awslabs/automated-security-helper.git@v3.2.0"
9595
```
9696

9797
```powershell
9898
# Install uv on Windows with PowerShell if it isn't installed already
9999
irm https://astral.sh/uv/install.ps1 | iex
100100
101101
# Create a function for ASH
102-
function ash { uvx git+https://github.com/awslabs/automated-security-helper.git@v3.1.13 $args }
102+
function ash { uvx git+https://github.com/awslabs/automated-security-helper.git@v3.2.0 $args }
103103
```
104104

105105
### Other Installation Methods
@@ -120,13 +120,13 @@ ash --help
120120
#### Using `pip`
121121

122122
```bash
123-
pip install git+https://github.com/awslabs/automated-security-helper.git@v3.1.13
123+
pip install git+https://github.com/awslabs/automated-security-helper.git@v3.2.0
124124
```
125125

126126
#### Clone the Repository
127127

128128
```bash
129-
git clone https://github.com/awslabs/automated-security-helper.git --branch v3.1.13
129+
git clone https://github.com/awslabs/automated-security-helper.git --branch v3.2.0
130130
cd automated-security-helper
131131
pip install .
132132
```

0 commit comments

Comments
 (0)