-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathexample_workflow_gemini.yml
More file actions
216 lines (172 loc) · 7.49 KB
/
Copy pathexample_workflow_gemini.yml
File metadata and controls
216 lines (172 loc) · 7.49 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
name: AI Slop Gate LLM GEMINI Analysis
on:
pull_request:
branches: [ main ]
push:
branches: [ main ]
workflow_dispatch:
permissions:
pull-requests: write
contents: read
jobs:
llm-analysis:
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Cache ai-slop-gate cache directory
uses: actions/cache@v4
with:
path: ~/.cache/ai-slop-gate
key: ai-slop-gate-cache-${{ runner.os }}-${{ hashFiles('**/*.py', '**/*.yml', '**/*.yaml') }}
restore-keys: |
ai-slop-gate-cache-${{ runner.os }}-
# Run LLM analysis
- name: LLM Analysis (ai-slop-gate)
id: llm_gate
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
continue-on-error: true
run: |
mkdir -p ~/.cache/ai-slop-gate
# Check if API key is available
if [ -z "$GEMINI_API_KEY" ]; then
echo "⚠️ GEMINI_API_KEY not set, skipping LLM analysis"
echo "verdict=SKIPPED" >> $GITHUB_OUTPUT
echo "findings=0" >> $GITHUB_OUTPUT
exit 0
fi
# Check if policy.yml exists, otherwise use default
POLICY_FLAG=""
if [ -f "${{ github.workspace }}/policy.yml" ]; then
echo "📋 Using custom policy.yml"
POLICY_FLAG="--policy /data/policy.yml"
else
echo "📋 Using default policy"
fi
# Run LLM analysis and capture output
set +e # Disable exit on error temporarily
docker run --rm \
-v "${{ github.workspace }}:/data" \
-v ~/.cache/ai-slop-gate:/root/.cache/ai-slop-gate \
-e GITHUB_TOKEN \
-e GEMINI_API_KEY \
ghcr.io/sergudo/ai-slop-gate:latest \
run --provider gemini --llm-local $POLICY_FLAG --path /data > raw_report.txt 2>&1
EXIT_CODE=$?
set -e # Re-enable exit on error
# Always show report
cat raw_report.txt
# Save exit code
echo "exit_code=$EXIT_CODE" >> $GITHUB_OUTPUT
# Extract verdict (default to UNKNOWN if not found)
VERDICT=$(grep "Policy Verdict:" raw_report.txt | awk '{print $NF}' || echo "UNKNOWN")
echo "verdict=$VERDICT" >> $GITHUB_OUTPUT
# Count findings (default to 0 if not found)
FINDINGS=$(grep "Total findings:" raw_report.txt | awk '{print $NF}' || echo "0")
echo "findings=$FINDINGS" >> $GITHUB_OUTPUT
# Log extracted values
echo "📊 Extracted values:"
echo " Exit code: $EXIT_CODE"
echo " Verdict: $VERDICT"
echo " Findings: $FINDINGS"
# Don't fail here
exit 0
# Post comment on PR (always)
- name: Post LLM Analysis Report to PR
if: github.event_name == 'pull_request' && always()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Check if analysis was skipped
if [ "${{ steps.llm_gate.outputs.verdict }}" = "SKIPPED" ]; then
cat > final_comment.md << EOF
## ⏭️ AI Slop Gate LLM Analysis
**Status:** SKIPPED
**Reason:** GEMINI_API_KEY not configured
To enable LLM analysis, add \`GEMINI_API_KEY\` to repository secrets.
<sub>🤖 Powered by [AI Slop Gate](https://github.com/SergUdo/ai-slop-gate) | Run: \`${{ github.run_id }}\`</sub>
EOF
gh pr comment ${{ github.event.pull_request.number }} \
--body-file final_comment.md \
--repo ${{ github.repository }}
exit 0
fi
# Extract clean report
sed -n '/=== AI SLOP GATE REPORT ===/,/=== END OF REPORT ===/p' raw_report.txt > clean_report.md
# Check if report was extracted
if [ ! -s clean_report.md ]; then
echo "⚠️ Warning: Could not extract report from raw_report.txt"
echo "=== NO REPORT GENERATED ===" > clean_report.md
echo "The LLM analysis may have failed to run properly." >> clean_report.md
fi
# Get values with defaults
VERDICT="${{ steps.llm_gate.outputs.verdict }}"
FINDINGS="${{ steps.llm_gate.outputs.findings }}"
# Set defaults if empty
VERDICT="${VERDICT:-UNKNOWN}"
FINDINGS="${FINDINGS:-0}"
echo "📊 Report values:"
echo " Verdict: $VERDICT"
echo " Findings: $FINDINGS"
# Determine emoji and status
if [ "$VERDICT" = "BLOCKING" ]; then
EMOJI="🚨"
STATUS="**BLOCKING** - Action Required"
elif [ "$VERDICT" = "ADVISORY" ]; then
EMOJI="⚠️"
STATUS="**ADVISORY** - Review Recommended"
elif [ "$VERDICT" = "ALLOW" ]; then
EMOJI="✅"
STATUS="**PASSED** - No Issues Found"
else
EMOJI="❓"
STATUS="**UNKNOWN** - Check logs"
fi
# Create professional comment
cat > final_comment.md << EOF
## $EMOJI AI Slop Gate LLM Analysis (Gemini)
**Status:** $STATUS
**Findings:** $FINDINGS issue(s) detected
> 🤖 **Deep Analysis:** This report uses AI to detect architectural issues, anti-patterns, and logic flaws that static analysis might miss.
---
EOF
# Append the clean report
cat clean_report.md >> final_comment.md
# Add footer with fix guide ONLY if there are violations
if [ "$FINDINGS" != "0" ] && [ "$VERDICT" != "ALLOW" ]; then
cat >> final_comment.md << EOF
---
<details>
<summary>📚 Understanding LLM Findings</summary>
### What LLM Analysis Detects
- Architectural anti-patterns and code smells
- Logic inconsistencies and contradictions
- Misleading naming or documentation
- Potential design flaws
- Security vulnerabilities in business logic
### How to Interpret Findings
- **High Confidence (0.8-1.0):** Strong evidence of an issue
- **Medium Confidence (0.5-0.8):** Worth investigating
- **Low Confidence (<0.5):** Consider in context
### False Positives
LLM analysis may flag intentional design decisions. Review findings critically and validate against your requirements.
</details>
EOF
fi
# Always add footer
cat >> final_comment.md << EOF
<sub>🤖 Powered by [AI Slop Gate](https://github.com/SergUdo/ai-slop-gate) + Gemini | Run: \`${{ github.run_id }}\`</sub>
EOF
# Post comment
gh pr comment ${{ github.event.pull_request.number }} \
--body-file final_comment.md \
--repo ${{ github.repository }}
# Set job status based on verdict
- name: Check LLM Analysis Result
if: steps.llm_gate.outputs.verdict == 'BLOCKING'
run: |
echo "❌ LLM analysis found blocking violations"
exit 1