New option: ignore matches based on regex patterns (`--ignore-match-pattern '^ENC\[AES256_GCM,data:'`)

[beavuck]

Is your feature request related to a problem? Please describe.

Secrets encrypted at rest can trigger detection on the ciphertext: a sops value like ENC[AES256_GCM,data:...] is often
reported as a Generic Password. ignored_matches needs each secret listed and goes stale on re-encryption,
ignored_paths excludes whole files, and --banlist-detector drops the detector everywhere.

Describe the solution you'd like

A repeatable --ignore-match-pattern PATTERN option on ggshield secret scan. A finding is ignored when one of its
match values is matched by a pattern:

ggshield secret scan path secrets.yaml --ignore-match-pattern '^ENC\[AES256_GCM,data:'

There is a matching secret.ignored_match_patterns setting in .gitguardian.yaml

secret:
    #...
  ignored_match_patterns:
    - '^ENC\[AES256_GCM,data:'

Ignored findings are reported as ignored and shown with --all-secrets, like other ignore reasons.

Describe alternatives you've considered

• ignored_matches -> per-secret, breaks on re-encryption.
• ignored_paths / --exclude -> excludes whole files, potentially hiding real secrets alongside.
• --banlist-detector -> loses the detector everywhere it would catch a real leak.

Additional context

I have a working implementation and can open a PR soon if this feature request is approved by maintainers.

Here is a before/after on a sops-encrypted file encrypted with AES256_GCM:

Before:

$ ggshield secret scan path secrets.yaml --json
... "incidents": [{"policy": "Secrets detection", "occurrences": [{"match": "ENC[AES256_GCM,data****...", "type": "password", ...}], "type": "Generic Password",...}], "total_incidents": 1, "total_occurrences": 1 ...

After:

$ ggshield secret scan path secrets.yaml --json --ignore-match-pattern "^ENC\[AES256_GCM,data:"
... "incidents": [], "total_incidents": 0, "total_occurrences": 0 ...

[beavuck]

It would help more projets adopt GitGuardian / ggshield -- there's at least one that I can think of, at my company, where this feature would be decisive in helping me getting the team to chose this tool.

I don't know how to say it better than it's already been said on GitGuardian's own website (talking about filepath exclusions):

The ability to reduce the number of incidents and concentrate solely on those that matter most is critical to scaling your secrets detection and remediation program.

This feature request goes further, definitely in the same correct direction, by proposing filtering out false positives based on regex patterns -- ignoring more noise to let the real signals get noticed better.

And like I said, I already have a fully-tested working implementation ready, in the hope this gets approved

[pierrelalanne]

Hi @beavuck,
I'm back from PTO and didn't follow this ticket creation: thanks a lot for this.

Two things I would like to mention:

1. The original example you are describing looks like a detection issue on our side, I'll forward to our detection team. For consistency, I'd rather have this fixed in the engine than handled downstream with specific exclusion rules.

2. That being said, your remark is still relevant in other situations, but did you have a look at the secret pattern exclusion rules in the dashboard? This allows you to specify regexes that wouldn't create any incidents. You can also restrict them to some of your sources. Pretty handy, and seems in line with what you are trying to do.

To elaborate a bit on our philosophy here:

• We provide you several ways to ignore some findings: based on the filepath, based on the secret value itself, and maybe in the future based on the source directly. These are managed centrally in your dashboard and they can always be restricted to certain sources. They are then honored and propagated to all ggshield users accordingly: this is typically pretty powerful in an enterprise context.

• Note that ggshield also honors the status of the incident in the dashboard, and the activation status of the detectors in the dashboard: incident ignored in the dashboard => ignored by ggshield. Detector deactivated in the dashboard => deactivated for ggshield.

• I know that you can still perform some customization at ggshield level directly with options: ignored_matches , ignored_paths, --banlist-detector. I'm just not sure we should push this even further. It kind of maintain a possible drift between the dashboard and ggshield.

[pierrelalanne]

I'd like to loop @mathieubellon in this discussion. I know you will be working on exclusion (again) shortly, and I know you have strong opinions on the way things should be when it comes to "central" vs. "local" settings. WDYT? 🙂

[beavuck]

Hi @pierrelalanne , thanks for the messages -- and the awesome DevSecOps tool

The original example you are describing looks like a detection issue on our side, I'll forward to our detection team. For consistency, I'd rather have this fixed in the engine than handled downstream with specific exclusion rules.

Thanks, yes that would be great actually, if it can be addressed while keeping GitGuardian's stated preference for false positives over false negatives. Such an interesting challenge btw

did you have a look at the secret pattern exclusion rules in the dashboard? This allows you to specify regexes that wouldn't create any incidents. You can also restrict them to some of your sources. Pretty handy, and seems in line with what you are trying to do.

Great question, sorry for not mentioning it when I opened the issue.

I can't use it because my use case happens to be dashboard-less: I'm on self-hosted GitLab, CE, and premium is needed if I want to monitor my repos on a GitGuardian dashboard. I rely on ggshield in CI (and as pre-commit hook on dev PCs) for secret detection...

I know that you can still perform some customization at ggshield level directly with options: ignored_matches , ignored_paths, --banlist-detector. I'm just not sure we should push this even further. It kind of maintain a possible drift between the dashboard and ggshield.

...so for what it's worth, I depend on such ggshield-level options pretty hard.

---

Right now my alternative has been to make ggshield ignore the entire SOPS secret.yaml files, and then in CI add a little shell script that inspects those to ensure that indeed the secret values are all encrypted.

It's workable, but I do think the new option I propose would be cleaner for my case, and maybe could serve a lot of folks in other such situations.

[pierrelalanne]

Do you have a more specific example to share on the kind of pattern that was picked up by our detectors? We already have some encryption pattern that we ignore, having more details and some examples would help to understand where we are falling short.

I can't use it because my use case happens to be dashboard-less: I'm on self-hosted GitLab, CE, and premium is needed if I want to monitor my repos on a GitGuardian dashboard. I rely on ggshield in CI (and as pre-commit hook on dev PCs) for secret detection...

I think my answer was not clear enough. This should not be related to your GitLab instance. It's not a problem if you are not monitoring your repos with the dashboard integration. The dashboard still defines how your PAT behaves (the API key you are using with ggshield). You should be able to go in your settings This is purely defined in your GitGuardian dashboard under "Settings/Secrets/Pattern Exclusions", and you can set up exclusion rules that will be honored by your PAT, and therefore by ggshield.

LMK if this helps.

[beavuck]

Do you have a more specific example to share on the kind of pattern that was picked up by our detectors?

Here's one way I've found, to reliably reproduce it:

I have SOPS and AGE set up as a pre-requisite, I can go deeper into the setup if that's necessary.

I decrypt one of my secrets.yaml files in a FluxCD repo

sops -i -d path/to/some/secrets.yaml

I add a RabbitMQ password KVP and I re-encrypt the file

sops -i -e path/to/some/secrets.yaml

Upon committing, I will trigger a false positive in ggshield (I'm redacting some stuff with underscores):

git commit -am "try to trigger ggshield"

Scanning... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 2 / 2

> commit://staged/path/to/some/secrets.yaml: 1 secret detected
>                                            1 secret ignored - use the `--all-secrets` option to display it

>> Secret detected: Generic Password
   Validity: No Checker
   Occurrences: 1
   Known by GitGuardian dashboard: NO
   Incident URL: N/A
   Secret SHA: 94feb82080987d5eb3d21bcfe316d6d6cbd9d136941c1041d801e183035d4a1c
   Secret found in vault: No
   Detector documentation: https://docs.gitguardian.com/secrets-detection/secrets-detection-engine/detectors/generics/generic_password_yaml

   21 | +    _________________: _____________________
   22 | +    _________________: _____________________
   23 | +    RABBITMQ_PASSWORD: ENC[AES256_GCM,data:wr**************************************************+********************+*************Tp8SVxkYwQTJHJxIvbFw==,type:str]
   24 | +    _________________: _____________________

...

Since it's encrypted and it's a secret I made up for this test I don't mind sharing it in plaintext here:

ENC[AES256_GCM,data:wr9xCTwjMCsCgTlLtzzMyCpPWn8LMWh9,iv:/KWKtKx5qjjzjx02+nLCzNUTiLzclrrw8CuGC+1mZT8=,tag:SpTp8SVxkYwQTJHJxIvbFw==,type:str]

I'm realizing after playing around with it that this got triggered because of the key RABBITMQ_PASSWORD, not because of the value. Same value + different key => no trigger.

But I've seen triggers on keys such as MAILER_URL and others, in the past few weeks. Besides, the values are encrypted and safe to commit even when the key is RABBITMQ_PASSWORD.

---

This is purely defined in your GitGuardian dashboard under "Settings/Secrets/Pattern Exclusions", and you can set up exclusion rules that will be honored by your PAT, and therefore by ggshield.

Oh wow, then yes that would do it, thanks for the extra info.

I have gone to Settings > Secrets > Pattern Exclusions and added .*ENC\[AES256_GCM,data:.* as a pattern, and it works like a charm.