Skip to content

Commit 7736adf

Browse files
CopilotMossaka
Copilot
and
Mossaka
authored
feat: add debugging-workflows skill for debugging GitHub Actions (#181)
* Initial plan * feat: add debugging-workflows skill for debugging GitHub Actions Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com> * fix: address security issues in debugging-workflows scripts Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com> * refactor: improve input validation per parameter type Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
1 parent f873364 commit 7736adf

3 files changed

Lines changed: 1197 additions & 0 deletions

File tree

‎.github/skills/debugging-workflows/SKILL.md‎

Lines changed: 363 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,363 @@
1+
---
2+
name: debugging-workflows
3+
description: Debug GitHub Actions workflows by downloading logs, analyzing summaries, and understanding how agentic workflows and the AWF firewall work together.
4+
allowed-tools: Bash(gh:*), Bash(curl:*), Bash(npx:*), Bash(node:*), Bash(cat:*), Bash(ls:*), Bash(grep:*), Bash(jq:*), Read
5+
---
6+
7+
# Debugging Workflows Skill
8+
9+
Use this skill when you need to debug GitHub Actions workflows, download workflow logs or summaries, or understand how agentic workflows and the AWF firewall work together.
10+
11+
## Quick Start
12+
13+
### Download Workflow Logs
14+
15+
Use the `download-workflow-logs.ts` script to download logs from a workflow run:
16+
17+
```bash
18+
# Download logs from the latest workflow run
19+
npx tsx .github/skills/debugging-workflows/download-workflow-logs.ts
20+
21+
# Download logs from a specific run ID
22+
npx tsx .github/skills/debugging-workflows/download-workflow-logs.ts --run-id 1234567890
23+
24+
# Download logs from a specific workflow
25+
npx tsx .github/skills/debugging-workflows/download-workflow-logs.ts --workflow test-integration.yml
26+
27+
# Save logs to a specific directory
28+
npx tsx .github/skills/debugging-workflows/download-workflow-logs.ts --output ./my-logs
29+
```
30+
31+
### Download Workflow Summary
32+
33+
Use the `download-workflow-summary.ts` script to get a summary of workflow runs:
34+
35+
```bash
36+
# Get summary of latest workflow runs
37+
npx tsx .github/skills/debugging-workflows/download-workflow-summary.ts
38+
39+
# Get summary for a specific workflow run
40+
npx tsx .github/skills/debugging-workflows/download-workflow-summary.ts --run-id 1234567890
41+
42+
# Get summary for a specific workflow file
43+
npx tsx .github/skills/debugging-workflows/download-workflow-summary.ts --workflow test-integration.yml
44+
45+
# Get summary as JSON
46+
npx tsx .github/skills/debugging-workflows/download-workflow-summary.ts --format json
47+
```
48+
49+
## GitHub CLI Commands
50+
51+
The `gh` CLI is essential for debugging workflows. Here are the most useful commands:
52+
53+
### List Workflow Runs
54+
55+
```bash
56+
# List recent workflow runs
57+
gh run list --limit 10
58+
59+
# List runs for a specific workflow
60+
gh run list --workflow test-integration.yml --limit 10
61+
62+
# List only failed runs
63+
gh run list --status failure --limit 10
64+
65+
# List runs in JSON format for parsing
66+
gh run list --json databaseId,name,status,conclusion,createdAt --limit 10
67+
```
68+
69+
### View Workflow Run Details
70+
71+
```bash
72+
# View a specific run
73+
gh run view <run-id>
74+
75+
# View run with job details
76+
gh run view <run-id> --verbose
77+
78+
# View run as JSON
79+
gh run view <run-id> --json jobs,conclusion,status
80+
```
81+
82+
### Download Run Logs
83+
84+
```bash
85+
# Download all logs for a run
86+
gh run download <run-id>
87+
88+
# Download specific artifact
89+
gh run download <run-id> --name <artifact-name>
90+
91+
# Download to specific directory
92+
gh run download <run-id> --dir ./logs
93+
```
94+
95+
### Watch a Running Workflow
96+
97+
```bash
98+
# Watch a workflow run in real-time
99+
gh run watch <run-id>
100+
101+
# Watch with exit code (useful for CI)
102+
gh run watch <run-id> --exit-status
103+
```
104+
105+
### Re-run Failed Jobs
106+
107+
```bash
108+
# Re-run failed jobs only
109+
gh run rerun <run-id> --failed
110+
111+
# Re-run all jobs
112+
gh run rerun <run-id>
113+
```
114+
115+
## Understanding Agentic Workflows
116+
117+
### What are Agentic Workflows?
118+
119+
Agentic workflows are GitHub Actions workflows that use AI agents (like GitHub Copilot or Claude) to perform tasks. They are defined using **markdown + YAML frontmatter** format in `.github/workflows/*.md` files and compiled to GitHub Actions YAML (`.lock.yml` files).
120+
121+
### Key Components
122+
123+
1. **Workflow File Format**: `.github/workflows/<name>.md`
124+
- YAML frontmatter for configuration
125+
- Markdown body for AI instructions
126+
- Compiles to `.github/workflows/<name>.lock.yml`
127+
128+
2. **Triggers** (`on:` field):
129+
- Standard GitHub events: `issues`, `pull_request`, `push`, `schedule`
130+
- Command triggers: `/mention` in issues/comments
131+
- `workflow_dispatch` for manual triggers
132+
133+
3. **Safe Outputs**: Controlled way for AI to create GitHub entities
134+
- `create-issue:` - Create GitHub issues
135+
- `create-pull-request:` - Create PRs with git patches
136+
- `add-comment:` - Add comments to issues/PRs
137+
- `add-labels:` - Add labels to issues/PRs
138+
- `create-discussion:` - Create GitHub discussions
139+
140+
4. **Tools Configuration** (`tools:` field):
141+
- `github:` - GitHub API tools
142+
- `agentic-workflows:` - Workflow introspection tools
143+
- `edit:` - File editing tools
144+
- `web-fetch:` / `web-search:` - Web access tools
145+
- `bash:` - Shell command tools
146+
147+
### Compiling Workflows
148+
149+
```bash
150+
# Compile all workflows
151+
gh aw compile
152+
153+
# Compile a specific workflow
154+
gh aw compile <workflow-name>
155+
156+
# Compile with strict security checks
157+
gh aw compile --strict
158+
```
159+
160+
### Debugging Agentic Workflows
161+
162+
```bash
163+
# View status of all agentic workflows
164+
gh aw status
165+
166+
# Download and analyze logs from previous runs
167+
gh aw logs <workflow-name> --json
168+
169+
# Audit a specific run for issues
170+
gh aw audit <run-id> --json
171+
```
172+
173+
### Common Issues
174+
175+
1. **Missing Tool Calls**: Check `missing_tools` in audit output
176+
2. **Safe Output Failures**: Review `safe_outputs.jsonl` artifact
177+
3. **Permission Issues**: Verify `permissions:` block in frontmatter
178+
4. **Network Blocked**: Check `network:` configuration for allowed domains
179+
180+
## Understanding the AWF Firewall
181+
182+
### What is AWF?
183+
184+
AWF (Agent Workflow Firewall) is a tool that provides L7 (HTTP/HTTPS) egress control for GitHub Copilot CLI and other agents. It restricts network access to a whitelist of approved domains using Squid proxy and Docker containers.
185+
186+
### Architecture Overview
187+
188+
```
189+
┌─────────────────────────────────────────┐
190+
│ Host (GitHub Actions Runner / Local) │
191+
│ │
192+
│ ┌────────────────────────────────────┐ │
193+
│ │ Firewall CLI (awf) │ │
194+
│ │ - Parse arguments │ │
195+
│ │ - Generate Squid config │ │
196+
│ │ - Start Docker Compose │ │
197+
│ └────────────────────────────────────┘ │
198+
│ │ │
199+
│ ▼ │
200+
│ ┌──────────────────────────────────┐ │
201+
│ │ Docker Compose │ │
202+
│ │ ┌────────────────────────────┐ │ │
203+
│ │ │ Squid Proxy Container │ │ │
204+
│ │ │ - Domain ACL filtering │ │ │
205+
│ │ │ - HTTP/HTTPS proxy │ │ │
206+
│ │ └────────────────────────────┘ │ │
207+
│ │ ▲ │ │
208+
│ │ ┌────────┼───────────────────┐ │ │
209+
│ │ │ Agent Container │ │ │
210+
│ │ │ - Full filesystem access │ │ │
211+
│ │ │ - iptables redirect │ │ │
212+
│ │ │ - All traffic → Squid │ │ │
213+
│ │ └────────────────────────────┘ │ │
214+
│ └──────────────────────────────────┘ │
215+
└─────────────────────────────────────────┘
216+
```
217+
218+
### Key Containers
219+
220+
- **`awf-squid`** - Squid proxy container (IP: 172.30.0.10)
221+
- Filters HTTP/HTTPS traffic based on domain allowlist
222+
- Logs all traffic decisions
223+
224+
- **`awf-agent`** - Agent execution container (IP: 172.30.0.20)
225+
- Runs the actual command/agent
226+
- Has iptables rules to redirect traffic to Squid
227+
- Full filesystem access via `/host` mount
228+
229+
### Traffic Flow
230+
231+
1. Command runs in agent container
232+
2. All HTTP/HTTPS traffic → iptables DNAT → Squid proxy
233+
3. Squid checks domain against allowlist
234+
4. Allowed → forward to destination
235+
5. Blocked → return 403 Forbidden
236+
237+
### Squid Log Analysis
238+
239+
```bash
240+
# View Squid access log (shows traffic decisions)
241+
docker exec awf-squid cat /var/log/squid/access.log
242+
243+
# Find blocked domains
244+
docker exec awf-squid grep "TCP_DENIED" /var/log/squid/access.log | awk '{print $3}' | sort -u
245+
246+
# Count blocked by domain
247+
docker exec awf-squid grep "TCP_DENIED" /var/log/squid/access.log | awk '{print $3}' | sort | uniq -c | sort -rn
248+
249+
# Real-time blocked traffic
250+
docker exec awf-squid tail -f /var/log/squid/access.log | grep --line-buffered TCP_DENIED
251+
```
252+
253+
### Squid Decision Codes
254+
255+
- `TCP_TUNNEL:HIER_DIRECT` = **ALLOWED** (HTTPS)
256+
- `TCP_MISS:HIER_DIRECT` = **ALLOWED** (HTTP)
257+
- `TCP_DENIED:HIER_NONE` = **BLOCKED**
258+
259+
### Running Commands Through Firewall
260+
261+
```bash
262+
# Basic usage
263+
sudo awf --allow-domains github.com 'curl https://api.github.com'
264+
265+
# With debug logging
266+
sudo awf --allow-domains github.com --log-level debug 'your-command'
267+
268+
# Keep containers for inspection
269+
sudo awf --allow-domains github.com --keep-containers 'your-command'
270+
```
271+
272+
### Preserved Logs Locations
273+
274+
**With `--keep-containers`:**
275+
- Squid: `/tmp/awf-<timestamp>/squid-logs/access.log`
276+
- Agent: `/tmp/awf-<timestamp>/agent-logs/`
277+
278+
**Normal execution (after cleanup):**
279+
- Squid: `/tmp/squid-logs-<timestamp>/access.log`
280+
- Agent: `/tmp/awf-agent-logs-<timestamp>/`
281+
282+
```bash
283+
# Find preserved logs
284+
ls -ldt /tmp/awf-* /tmp/squid-logs-* 2>/dev/null | head -5
285+
286+
# View preserved Squid logs
287+
sudo cat $(ls -t /tmp/squid-logs-*/access.log 2>/dev/null | head -1)
288+
```
289+
290+
## Debugging Workflow Failures
291+
292+
### Step-by-Step Process
293+
294+
1. **Identify the failing workflow run**
295+
```bash
296+
gh run list --status failure --limit 5
297+
```
298+
299+
2. **Get run details**
300+
```bash
301+
gh run view <run-id> --verbose
302+
```
303+
304+
3. **Download logs**
305+
```bash
306+
gh run download <run-id> --dir ./logs
307+
# Or use the script:
308+
npx tsx .github/skills/debugging-workflows/download-workflow-logs.ts --run-id <run-id>
309+
```
310+
311+
4. **Analyze the failure**
312+
- Check job logs for error messages
313+
- Look for timeout issues
314+
- Check for permission errors
315+
- Review network-related errors
316+
317+
5. **For agentic workflows, audit the run**
318+
```bash
319+
gh aw audit <run-id> --json
320+
```
321+
322+
6. **If firewall-related, check Squid logs**
323+
```bash
324+
# If containers are still running
325+
docker exec awf-squid cat /var/log/squid/access.log
326+
327+
# Or check preserved logs
328+
sudo cat /tmp/squid-logs-*/access.log
329+
```
330+
331+
### Common Failure Patterns
332+
333+
#### Permission Denied
334+
```
335+
Error: Resource not accessible by integration
336+
```
337+
**Fix:** Check `permissions:` in workflow frontmatter
338+
339+
#### Domain Blocked
340+
```
341+
curl: (56) Recv failure: Connection reset by peer
342+
```
343+
**Fix:** Add domain to `--allow-domains` or `network:` configuration
344+
345+
#### Timeout
346+
```
347+
Error: The operation was canceled.
348+
```
349+
**Fix:** Increase `timeout-minutes` in workflow configuration
350+
351+
#### Missing Tool
352+
```
353+
Tool 'xyz' not found
354+
```
355+
**Fix:** Add tool to `tools:` configuration in workflow frontmatter
356+
357+
## Related Documentation
358+
359+
- [Architecture](../../../docs/architecture.md) - System architecture details
360+
- [Troubleshooting](../../../docs/troubleshooting.md) - Common issues and solutions
361+
- [GitHub Actions Integration](../../../docs/github_actions.md) - CI/CD setup
362+
- [Logging Documentation](../../../LOGGING.md) - Comprehensive logging guide
363+
- [Debug Firewall Skill](../debug-firewall/SKILL.md) - Firewall-specific debugging

0 commit comments

Comments
 (0)