Skip to content

Commit 7780a65

Browse files
lforstlforst
authored
feat: Add braintrust monorepo security scan (#38)
1 parent bceda18 commit 7780a65

1 file changed

Lines changed: 152 additions & 0 deletions

File tree

‎.github/workflows/braintrust-security-audit.yml‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
name: Braintrust Security Audit
2+
3+
on:
4+
schedule:
5+
- cron: "0 7 * * *"
6+
workflow_dispatch:
7+
8+
permissions: {}
9+
10+
concurrency:
11+
group: braintrust-security-audit-scheduled-${{ github.ref }}
12+
cancel-in-progress: false
13+
14+
jobs:
15+
audit-braintrust-security:
16+
name: Audit braintrust security
17+
runs-on: ubuntu-latest
18+
timeout-minutes: 90
19+
steps:
20+
- name: Generate source repository token
21+
id: source-token
22+
uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
23+
with:
24+
app-id: ${{ secrets.BRAINTRUST_BOT_APP_ID }}
25+
private-key: ${{ secrets.BRAINTRUST_BOT_PRIVATE_KEY }}
26+
owner: braintrustdata
27+
repositories: |
28+
braintrust
29+
permission-contents: read
30+
31+
- name: Generate issue repository token
32+
id: issue-token
33+
uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
34+
with:
35+
app-id: ${{ secrets.BRAINTRUST_BOT_APP_ID }}
36+
private-key: ${{ secrets.BRAINTRUST_BOT_PRIVATE_KEY }}
37+
owner: braintrustdata
38+
repositories: |
39+
braintrust-security-scan-issues
40+
permission-issues: write
41+
42+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
43+
with:
44+
repository: braintrustdata/braintrust
45+
token: ${{ steps.source-token.outputs.token }}
46+
persist-credentials: false
47+
48+
- name: Run Claude security audit
49+
uses: anthropics/claude-code-action@df37d2f0760a4b5683a6e617c9325bc1a36443f6 # v1.0.75
50+
with:
51+
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
52+
github_token: ${{ steps.issue-token.outputs.token }}
53+
show_full_output: "true"
54+
display_report: "true"
55+
settings: |
56+
{
57+
"env": {
58+
"SOURCE_REPO_OWNER": "braintrustdata",
59+
"SOURCE_REPO_NAME": "braintrust",
60+
"ISSUE_REPO_OWNER": "braintrustdata",
61+
"ISSUE_REPO_NAME": "braintrust-security-scan-issues"
62+
}
63+
}
64+
prompt: |
65+
# Goal
66+
67+
Find high-confidence, actionable security vulnerabilities in the checked-out `SOURCE_REPO_OWNER/SOURCE_REPO_NAME` repository and create concise issue reports only in `ISSUE_REPO_OWNER/ISSUE_REPO_NAME`.
68+
69+
# Scope
70+
71+
- Inspect only the checked-out source repository for source-code, configuration, dependency, infrastructure-as-code, authentication, authorization, data exposure, injection, tenant isolation, cryptography, secret handling, webhook, API, and deployment security issues.
72+
- Focus on vulnerabilities with a concrete affected code path, configuration, or dependency and a plausible exploit or abuse path.
73+
- Treat tests, docs, examples, generated files, and scripts as supporting evidence, but do not report findings that only affect non-production examples unless they create a realistic production risk.
74+
- Exclude speculative hardening ideas, broad best-practice suggestions, style issues, missing tests without a vulnerability, and purely theoretical issues without repo evidence.
75+
- Do not report secret-scanning alerts or raw leaked secrets. If you encounter a secret-looking value in source, describe only the file/path and risk class without including the secret value.
76+
77+
# Process
78+
79+
1. Inspect local code, configs, tests, docs, examples, CI, deployment, and infrastructure files to understand security-relevant behavior.
80+
2. Follow security-sensitive flows across trust boundaries before deciding a finding is real.
81+
3. When external behavior matters, prefer official docs, official release notes, or authoritative project sources.
82+
4. Search existing GitHub issues for duplicates before creating anything.
83+
- Only search in `ISSUE_REPO_OWNER/ISSUE_REPO_NAME`.
84+
- For every GitHub MCP tool call (`mcp__github__search_issues`, `mcp__github__list_issues`, `mcp__github__get_issue`, `mcp__github__get_issue_comments`, `mcp__github__create_issue`), explicitly pass owner=`ISSUE_REPO_OWNER` and repo=`ISSUE_REPO_NAME`. `ISSUE_REPO_OWNER` and `ISSUE_REPO_NAME` are environment variables.
85+
- Never query, create, update, or comment on issues in `SOURCE_REPO_OWNER/SOURCE_REPO_NAME`.
86+
- Never query, create, update, or comment on issues in the repository running this workflow.
87+
5. Only create issues for high-confidence, non-duplicate vulnerabilities that a maintainer could act on from the report.
88+
89+
# Examples
90+
91+
## Good
92+
93+
- An authorization check is missing on a server-side route and the inspected code shows a tenant or project boundary can be crossed.
94+
- A user-controlled value reaches a shell, SQL, template, path, SSRF, or deserialization sink without adequate validation or escaping.
95+
- A production deployment or infrastructure configuration grants materially broader access than the code path needs, with an exact affected file and impact.
96+
- A dependency or framework behavior creates a vulnerability in this repository's concrete usage, with official or authoritative source evidence.
97+
98+
## Bad
99+
100+
- "Consider adding rate limiting" without showing an abuse path tied to a concrete endpoint or security control.
101+
- "This might be vulnerable" based only on a suspicious function name.
102+
- A finding that depends on ignoring an existing validation, authorization, or environment guard.
103+
- A raw secret value pasted into the issue body.
104+
- Any issue created in `SOURCE_REPO_OWNER/SOURCE_REPO_NAME`.
105+
106+
# If You Find Actionable Non-Duplicate Vulnerabilities
107+
108+
- Create at most 3 issues in this run.
109+
- Create one issue per distinct vulnerability.
110+
- Keep each issue concise, concrete, and source-backed.
111+
- Prefix every created issue title with `[bot] `.
112+
- If the repository supports issue types, choose whichever type fits best (`Bug`, `Feature`, or `Task`). If issue types are unavailable, create the issue without type.
113+
- Include a hidden marker comment near the top of the issue body in this exact form:
114+
115+
```html
116+
<!-- braintrust-security-audit: <finding_id> -->
117+
```
118+
119+
Each issue should clearly include:
120+
121+
- affected source repository: `SOURCE_REPO_OWNER/SOURCE_REPO_NAME`
122+
- severity: `critical`, `high`, `medium`, or `low`
123+
- affected source files and code paths inspected
124+
- vulnerability summary
125+
- concrete impact
126+
- evidence and reasoning
127+
- reproduction, exploit sketch, or abuse path when possible without harmful detail
128+
- suggested fix direction
129+
- exact external sources inspected, if any
130+
131+
# Duplicate Handling
132+
133+
- Do not create an issue if an open issue in `ISSUE_REPO_OWNER/ISSUE_REPO_NAME` already covers the same vulnerability.
134+
- Treat a matching hidden marker comment or a clearly equivalent open issue as a duplicate.
135+
- If duplicate checking is inconclusive, do not create the issue.
136+
137+
# Constraints
138+
139+
- Create issues only in `ISSUE_REPO_OWNER/ISSUE_REPO_NAME`.
140+
- Do not create comments.
141+
- Do not update, close, label, assign, or milestone existing issues.
142+
- Do not create pull requests.
143+
- Do not modify files.
144+
- Do not run shell commands.
145+
- Do not include raw secrets, private keys, session tokens, API keys, passwords, or exploit payloads that would materially increase risk.
146+
- Prefer local repo evidence plus authoritative external sources over inference.
147+
- If there are no high-confidence non-duplicate vulnerabilities, do nothing.
148+
claude_args: |
149+
--model claude-opus-4-6
150+
--max-turns 250
151+
--allowedTools "Read,Glob,Grep,LS,WebSearch,WebFetch,mcp__github__get_issue,mcp__github__get_issue_comments,mcp__github__search_issues,mcp__github__list_issues,mcp__github__create_issue"
152+
--disallowedTools "Bash,Edit,MultiEdit,Write,Replace,NotebookEditCell,mcp__github__create_issue_comment,mcp__github__update_issue,mcp__github__create_pr,mcp__github__create_or_update_file,mcp__github__delete_file,mcp__github_file_ops__commit_files,mcp__github_file_ops__delete_files"

0 commit comments

Comments
 (0)