Skip to content

Commit 849436a

Browse files
fproulx-boostsecurityclaude
fproulx-boostsecurity
and
claude
authored
Add structured metadata fields to findings for programmatic access (#385)
* Add structured metadata fields to findings for programmatic access Adds new fields to FindingMeta for library users who need machine-readable access to security-relevant data without parsing human-readable strings: - injection_sources: Array of specific expressions being injected - lotp_tool: Living Off The Pipeline build tool (npm, pip, make, etc.) - lotp_action: LOTP GitHub Action identifier - referenced_secrets: Secrets referenced in the job (excludes GITHUB_TOKEN) The referenced_secrets field is automatically extracted when rules pass the _job field, supporting dot notation (secrets.FOO) and bracket notation (secrets['FOO'] and secrets["FOO"]). Benchmark (Apple M4 Pro): | Version | ns/op | B/op | allocs/op | |---------|-----------|-----------|-----------| | Before | 10971339 | 7149084 | 132787 | | After | 12059356 | 7858242 | 148422 | | Delta | +9.9% | +9.9% | +11.8% | Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Use hex.EncodeToString for faster fingerprint encoding Fixes perfsprint linter warning by replacing fmt.Sprintf("%x", ...) with hex.EncodeToString which is more performant. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Use require.NoError for error assertions in test Fixes testifylint warning by using require.NoError instead of assert.NoError for error checking in TestStructuredFindingFields. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent a6900f5 commit 849436a

9 files changed

Lines changed: 513 additions & 102 deletions

File tree

‎docs/content/en/structured-finding-metadata.md‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
---
2+
title: "Structured Finding Metadata"
3+
weight: 10
4+
---
5+
6+
# Structured Finding Metadata
7+
8+
Poutine findings now include structured metadata fields that provide programmatic access to security-relevant information. These fields enable library users to build automated triage workflows, correlate findings with secrets exposure, and integrate with downstream security tooling without parsing human-readable text.
9+
10+
## New Finding Fields
11+
12+
### `injection_sources`
13+
14+
**Type:** `[]string`
15+
16+
A sorted array of the specific expression sources that are being injected into a sink (shell script, JavaScript, etc.).
17+
18+
**Example:**
19+
```json
20+
{
21+
"rule_id": "injection",
22+
"meta": {
23+
"details": "Sources: github.event.issue.title github.head_ref",
24+
"injection_sources": ["github.event.issue.title", "github.head_ref"]
25+
}
26+
}
27+
```
28+
29+
**Use case:** Programmatically identify which untrusted inputs are exploitable without parsing the `details` string.
30+
31+
---
32+
33+
### `lotp_tool`
34+
35+
**Type:** `string`
36+
37+
The "Living Off The Pipeline" build tool detected after an untrusted checkout. Common values include `npm`, `pip`, `make`, `bash`, `cargo`, `gradle`, etc.
38+
39+
**Example:**
40+
```json
41+
{
42+
"rule_id": "untrusted_checkout_exec",
43+
"meta": {
44+
"details": "Detected usage of `npm`",
45+
"lotp_tool": "npm"
46+
}
47+
}
48+
```
49+
50+
**Use case:** Filter findings by tool type, prioritize based on tool risk, or build tool-specific remediation guidance.
51+
52+
---
53+
54+
### `lotp_action`
55+
56+
**Type:** `string`
57+
58+
The GitHub Action identified as a "Living Off The Pipeline" vector (e.g., actions that execute code from the checked-out repository).
59+
60+
**Example:**
61+
```json
62+
{
63+
"rule_id": "untrusted_checkout_exec",
64+
"meta": {
65+
"details": "Detected usage the GitHub Action `bridgecrewio/checkov-action`",
66+
"lotp_action": "bridgecrewio/checkov-action"
67+
}
68+
}
69+
```
70+
71+
**Use case:** Track which third-party actions introduce code execution risks, build action allowlists.
72+
73+
---
74+
75+
### `referenced_secrets`
76+
77+
**Type:** `[]string`
78+
79+
A sorted array of secret names referenced in the job where the vulnerability was found. The `GITHUB_TOKEN` is excluded since it's always available.
80+
81+
Supports both dot notation (`secrets.MY_SECRET`) and bracket notation (`secrets['MY_SECRET']`).
82+
83+
**Example:**
84+
```json
85+
{
86+
"rule_id": "untrusted_checkout_exec",
87+
"meta": {
88+
"lotp_tool": "npm",
89+
"referenced_secrets": ["API_KEY", "DATABASE_PASSWORD", "DEPLOY_TOKEN"]
90+
}
91+
}
92+
```
93+
94+
**Use case:** Assess blast radius of a vulnerability - if a job with an injection vulnerability also references `PROD_DEPLOY_KEY`, the finding is more critical than one with no secrets.
95+
96+
---
97+
98+
## Usage Examples
99+
100+
### Prioritize by Secrets Exposure
101+
102+
```python
103+
def calculate_priority(finding):
104+
secrets = finding.get("meta", {}).get("referenced_secrets", [])
105+
high_value = ["DEPLOY", "PROD", "AWS", "GCP", "AZURE", "NPM_TOKEN"]
106+
107+
if any(s for s in secrets if any(h in s for h in high_value)):
108+
return "critical"
109+
elif secrets:
110+
return "high"
111+
return "medium"
112+
```
113+
114+
### Filter Injection Sources
115+
116+
```python
117+
def is_pr_body_injection(finding):
118+
sources = finding.get("meta", {}).get("injection_sources", [])
119+
pr_body_patterns = ["pull_request.body", "issue.body", "comment.body"]
120+
return any(p in s for s in sources for p in pr_body_patterns)
121+
```
122+
123+
### Group by LOTP Tool
124+
125+
```python
126+
from collections import defaultdict
127+
128+
def group_by_tool(findings):
129+
by_tool = defaultdict(list)
130+
for f in findings:
131+
if f["rule_id"] == "untrusted_checkout_exec":
132+
tool = f["meta"].get("lotp_tool") or f["meta"].get("lotp_action", "unknown")
133+
by_tool[tool].append(f)
134+
return dict(by_tool)
135+
```
136+
137+
---
138+
139+
## Backward Compatibility
140+
141+
These fields are additive - the existing `details` field continues to provide human-readable descriptions. Tools parsing `details` will continue to work, but new integrations should prefer the structured fields for reliability.
142+
143+
**JSON behavior:**
144+
- `injection_sources`, `lotp_tool`, `lotp_action`: Omitted when not applicable
145+
- `referenced_secrets`: Present as `[]` (empty array) for GitHub Actions findings even when no secrets are found; omitted for other CI systems
146+
147+
---
148+
149+
## Supported Rules
150+
151+
| Rule | `injection_sources` | `lotp_tool` | `lotp_action` | `referenced_secrets` |
152+
|------|---------------------|-------------|---------------|----------------------|
153+
| `injection` (GitHub Actions) | Yes | - | - | Yes |
154+
| `injection` (GitLab CI) | Yes | - | - | - |
155+
| `injection` (Azure Pipelines) | Yes | - | - | - |
156+
| `injection` (Tekton) | Yes | - | - | - |
157+
| `untrusted_checkout_exec` (GitHub Actions) | - | Yes | Yes | Yes |
158+
| `untrusted_checkout_exec` (Azure DevOps) | - | Yes | - | - |
159+
| `untrusted_checkout_exec` (Tekton) | - | Yes | - | - |

‎opa/rego/poutine.rego‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
package poutine
22

3+
import data.poutine.utils
34
import rego.v1
45

56
rule(chain) = {
@@ -24,10 +25,25 @@ rule(chain) = {
2425
)
2526
}
2627

28+
# finding with _job field - extracts referenced_secrets automatically
29+
finding(rule, pkg_purl, meta) = {
30+
"rule_id": rule.id,
31+
"purl": pkg_purl,
32+
"meta": object.union(
33+
object.remove(meta, ["_job"]),
34+
{"referenced_secrets": utils.job_referenced_secrets(meta._job)},
35+
),
36+
} if {
37+
meta._job
38+
}
39+
40+
# finding without _job field - no automatic secrets extraction
2741
finding(rule, pkg_purl, meta) = {
2842
"rule_id": rule.id,
2943
"purl": pkg_purl,
3044
"meta": meta,
45+
} if {
46+
not meta._job
3147
}
3248

3349
_rule_config(rule_id, meta) = object.union(rule_config, config_values) if {

‎opa/rego/poutine/utils.rego‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,4 +103,42 @@ find_first_uses_in_job(job, uses) := xs if {
103103
s := job.steps[i]
104104
startswith(s.uses, sprintf("%v@", [uses[_]]))
105105
}
106-
}
106+
}
107+
108+
########################################################################
109+
# extract_referenced_secrets
110+
# Extracts all secrets.* references from GitHub Actions expressions (${{ }})
111+
# Excludes GITHUB_TOKEN. Handles dot and bracket notation.
112+
########################################################################
113+
114+
# Dot notation: ${{ secrets.FOO }} or ${{ format(secrets.FOO) }}
115+
_secrets_dot_notation(str) := {m[1] |
116+
matches := regex.find_all_string_submatch_n("\\$\\{\\{[^}]*?secrets\\.([a-zA-Z_][a-zA-Z0-9_]*)", str, -1)
117+
m := matches[_]
118+
m[1] != "GITHUB_TOKEN"
119+
}
120+
121+
# Bracket notation with single quotes: ${{ secrets['FOO'] }}
122+
_secrets_bracket_single(str) := {m[1] |
123+
matches := regex.find_all_string_submatch_n("\\$\\{\\{[^}]*?secrets\\['([a-zA-Z_][a-zA-Z0-9_]*)'\\]", str, -1)
124+
m := matches[_]
125+
m[1] != "GITHUB_TOKEN"
126+
}
127+
128+
# Bracket notation with double quotes: ${{ secrets["FOO"] }}
129+
# Also handles JSON-escaped quotes: secrets[\"FOO\"] (after json.marshal)
130+
_secrets_bracket_double(str) := {m[1] |
131+
matches := regex.find_all_string_submatch_n("\\$\\{\\{[^}]*?secrets\\[\\\\?\"([a-zA-Z_][a-zA-Z0-9_]*)\\\\?\"\\]", str, -1)
132+
m := matches[_]
133+
m[1] != "GITHUB_TOKEN"
134+
}
135+
136+
extract_referenced_secrets(str) := sort(secrets) if {
137+
secrets := _secrets_dot_notation(str) | _secrets_bracket_single(str) | _secrets_bracket_double(str)
138+
}
139+
140+
# Extract secrets from a job by marshaling to JSON and searching
141+
job_referenced_secrets(job) := secrets if {
142+
job_json := json.marshal(job)
143+
secrets := extract_referenced_secrets(job_json)
144+
}

‎opa/rego/rules/injection.rego‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ results contains poutine.finding(rule, pkg.purl, {
3333
"job": job.id,
3434
"step": i,
3535
"details": sprintf("Sources: %s", [concat(" ", exprs)]),
36+
"injection_sources": sort(exprs),
37+
"_job": job,
3638
"event_triggers": [event | event := workflow.events[j].name],
3739
}) if {
3840
pkg = input.packages[_]
@@ -48,6 +50,7 @@ results contains poutine.finding(rule, pkg.purl, {
4850
"line": line,
4951
"step": i,
5052
"details": sprintf("Sources: %s", [concat(" ", exprs)]),
53+
"injection_sources": sort(exprs),
5154
"event_triggers": [event | event := action.events[j].name],
5255
}) if {
5356
pkg = input.packages[_]
@@ -69,6 +72,7 @@ results contains poutine.finding(rule, pkg.purl, {
6972
"path": config.path,
7073
"job": sprintf("%s.%s[%d]", [job.name, attr, i]),
7174
"details": sprintf("Sources: %s", [concat(" ", exprs)]),
75+
"injection_sources": sort(exprs),
7276
"line": job[attr][i].line,
7377
}) if {
7478
pkg = input.packages[_]
@@ -94,6 +98,7 @@ results contains poutine.finding(rule, pkg.purl, {
9498
"step": step_id,
9599
"line": step.lines[attr],
96100
"details": sprintf("Sources: %s", [concat(" ", exprs)]),
101+
"injection_sources": sort(exprs),
97102
}) if {
98103
some attr in {"script", "powershell", "pwsh", "bash"}
99104
pkg := input.packages[_]
@@ -117,6 +122,7 @@ results contains poutine.finding(rule, pkg.purl, {
117122
"step": step_idx,
118123
"line": step.lines.start,
119124
"details": sprintf("Sources: %s", [concat(" ", exprs)]),
125+
"injection_sources": sort(exprs),
120126
}) if {
121127
pkg := input.packages[_]
122128
pipeline := pkg.pipeline_as_code_tekton[_]

‎opa/rego/rules/untrusted_checkout_exec.rego‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -99,10 +99,13 @@ build_commands[cmd] = {
9999
results contains poutine.finding(rule, pkg_purl, {
100100
"path": workflow_path,
101101
"line": step.lines.run,
102+
"job": job_id,
103+
"lotp_tool": cmd,
104+
"_job": job_obj,
102105
"details": sprintf("Detected usage of `%s`", [cmd]),
103106
"event_triggers": workflow_events,
104107
}) if {
105-
[pkg_purl, workflow_path, workflow_events, step] := _steps_after_untrusted_checkout[_]
108+
[pkg_purl, workflow_path, workflow_events, step, job_id, job_obj] := _steps_after_untrusted_checkout[_]
106109
regex.match(
107110
sprintf("([^a-z]|^)(%v)", [concat("|", build_commands[cmd])]),
108111
step.run,
@@ -112,10 +115,13 @@ results contains poutine.finding(rule, pkg_purl, {
112115
results contains poutine.finding(rule, pkg_purl, {
113116
"path": workflow_path,
114117
"line": step.lines.uses,
118+
"job": job_id,
119+
"lotp_action": step.action,
120+
"_job": job_obj,
115121
"details": sprintf("Detected usage the GitHub Action `%s`", [step.action]),
116122
"event_triggers": workflow_events,
117123
}) if {
118-
[pkg_purl, workflow_path, workflow_events, step] := _steps_after_untrusted_checkout[_]
124+
[pkg_purl, workflow_path, workflow_events, step, job_id, job_obj] := _steps_after_untrusted_checkout[_]
119125
regex.match(
120126
sprintf("([^a-z]|^)(%v)@", [concat("|", build_github_actions[_])]),
121127
step.uses,
@@ -126,17 +132,20 @@ results contains poutine.finding(rule, pkg_purl, {
126132
results contains poutine.finding(rule, pkg_purl, {
127133
"path": workflow_path,
128134
"line": step.lines.uses,
135+
"job": job_id,
136+
"lotp_action": step.action,
137+
"_job": job_obj,
129138
"details": sprintf("Detected usage of a Local GitHub Action at path: `%s`", [step.action]),
130139
"event_triggers": workflow_events,
131140
}) if {
132-
[pkg_purl, workflow_path, workflow_events, step] := _steps_after_untrusted_checkout[_]
141+
[pkg_purl, workflow_path, workflow_events, step, job_id, job_obj] := _steps_after_untrusted_checkout[_]
133142
regex.match(
134143
`^\./`,
135144
step.action,
136145
)
137146
}
138147

139-
_steps_after_untrusted_checkout contains [pkg.purl, workflow.path, events, s.step] if {
148+
_steps_after_untrusted_checkout contains [pkg.purl, workflow.path, events, s.step, workflow.jobs[s.job_idx].id, workflow.jobs[s.job_idx]] if {
140149
pkg := input.packages[_]
141150
workflow := pkg.github_actions_workflows[_]
142151

@@ -147,7 +156,7 @@ _steps_after_untrusted_checkout contains [pkg.purl, workflow.path, events, s.ste
147156
s := utils.workflow_steps_after(pr_checkout)[_]
148157
}
149158

150-
_steps_after_untrusted_checkout contains [pkg_purl, workflow.path, events, s.step] if {
159+
_steps_after_untrusted_checkout contains [pkg_purl, workflow.path, events, s.step, workflow.jobs[s.job_idx].id, workflow.jobs[s.job_idx]] if {
151160
[pkg_purl, workflow] := _workflows_runs_from_pr[_]
152161

153162
events := [event | event := workflow.events[i].name]
@@ -170,6 +179,7 @@ results contains poutine.finding(rule, pkg_purl, {
170179
"job": job,
171180
"step": s.step_idx,
172181
"line": s.step.lines[attr],
182+
"lotp_tool": cmd,
173183
"details": sprintf("Detected usage of `%s`", [cmd]),
174184
}) if {
175185
[pkg_purl, pipeline_path, s, job] := _steps_after_untrusted_checkout_ado[_]
@@ -213,6 +223,7 @@ results contains poutine.finding(rule, pkg.purl, {
213223
"job": task.name,
214224
"step": step_idx,
215225
"line": step.lines.script,
226+
"lotp_tool": cmd,
216227
"details": sprintf("Detected usage of `%s`", [cmd]),
217228
}) if {
218229
pkg := input.packages[_]

‎results/results.go‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ package results
22

33
import (
44
"crypto/sha256"
5+
"encoding/hex"
56
"encoding/json"
6-
"fmt"
77
"strconv"
88

99
"github.com/rs/zerolog/log"
@@ -23,6 +23,12 @@ type FindingMeta struct {
2323
Details string `json:"details,omitempty"`
2424
EventTriggers []string `json:"event_triggers,omitempty"`
2525
BlobSHA string `json:"blobsha,omitempty"`
26+
27+
// Structured fields for programmatic access
28+
InjectionSources []string `json:"injection_sources,omitempty"` // Sources confirmed as injected into a sink
29+
LOTPTool string `json:"lotp_tool,omitempty"` // Living Off The Pipeline tool (e.g., npm, pip)
30+
LOTPAction string `json:"lotp_action,omitempty"` // Living Off The Pipeline GitHub Action
31+
ReferencedSecrets []string `json:"referenced_secrets,omitempty"` // Secrets referenced in workflow (excludes GITHUB_TOKEN)
2632
}
2733

2834
type Finding struct {
@@ -36,7 +42,7 @@ func (f *Finding) GenerateFindingFingerprint() string {
3642
h := sha256.New()
3743
h.Write([]byte(fingerprintString))
3844
fingerprint := h.Sum(nil)
39-
return fmt.Sprintf("%x", fingerprint)
45+
return hex.EncodeToString(fingerprint)
4046
}
4147

4248
func (m *FindingMeta) UnmarshalJSON(data []byte) error {

‎scanner/inventory_scanner_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ func TestGithubWorkflows(t *testing.T) {
3535
".github/workflows/anchors_job.yml",
3636
".github/workflows/anchors_multiple.yml",
3737
".github/workflows/anchors_with_vulnerability.yml",
38+
".github/workflows/test_new_fields.yml",
3839
})
3940
}
4041

0 commit comments

Comments
 (0)