[Pelis Agent Factory Advisor] Agentic Workflow Maturity Analysis & Recommendations - 2026-03-08

[github-actions[bot]]

📊 Executive Summary

The gh-aw-firewall repository has reached Level 4 agentic maturity with 28 compiled workflow files spanning security, testing, CI automation, issue management, and documentation. However, several high-value patterns from Pelis Agent Factory are absent — most critically, there is no issue triage/labeling agent and no daily static analysis report (despite this being a security-focused tool). There are also signs of agent ecosystem stress: multiple workflows are currently failing with no meta-analytics layer to detect these trends proactively.

---

🎓 Patterns Learned from Pelis Agent Factory

Documentation Site Key Takeaways

From Peli's Agent Factory and the 19-part workflow series:

1. Meta-agents are high ROI – The Audit Workflows meta-agent created 93 discussions in gh-aw, the Metrics Collector created 41 daily reports, and the Workflow Health Manager created 40 issues (14 PRs merged). This repo has no meta-analytics layer despite running 15+ scheduled workflows daily/hourly.

2. Causal chains multiply impact – Issue Triage → Issue Monster → Copilot agent creates a self-reinforcing loop. The Testify Expert had a 100% causal chain merge rate. This repo has issue-monster but no triage agent to feed it properly.

3. Security workflows need specialization – The factory runs dedicated workflows for: secrets analysis, malicious code scan, static analysis (zizmor/poutine/actionlint reports), security compliance, and firewall validation. This repo has security-guard (PRs) and security-review (daily), but is missing malicious code scan and static analysis reporting.

4. Code simplification is continuous – Code Simplifier (83% merge rate) and Duplicate Code Detector (79% merge rate) prove daily cleanup has high value. No continuous simplification workflow exists here.

5. Skip-if-match prevents pile-up – Workflow throttling is critical at scale. Some existing workflows here already use this pattern well.

Agentics Repository Insights

The githubnext/agentics repository contains 44 reusable workflow templates. Notable templates not yet used here: link-checker.md, daily-malicious-code-scan.md, contribution-guidelines-checker.md, grumpy-reviewer.md, daily-test-improver.md, ci-coach.md, weekly-issue-summary.md.

---

📋 Current Agentic Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test-{8 languages}	Build/test PRs in 8 language environments	PR open/sync	✅ Well-designed multi-language coverage
ci-doctor	Investigates CI failures, opens issues	workflow_run (failure)	✅ Strong; monitors 27 workflows. ⚠️ Currently failing itself (#1164)
ci-cd-gaps-assessment	CI/CD gap analysis	Daily	✅ Useful assessment; could be enriched with trend data
cli-flag-consistency-checker	CLI flag vs docs sync	Weekly	✅ Domain-specific, relevant
dependency-security-monitor	CVE detection + safe dep update PRs	Daily	✅ Excellent; creates actionable issues
doc-maintainer	Docs sync with code changes	Daily	✅ Good; skip-if-match prevents pile-up
issue-duplication-detector	Detect duplicate issues	Issues opened	✅ Smart use of cache-memory
issue-monster	Dispatch issues to Copilot	Issues opened + hourly	✅ Core orchestrator
pelis-agent-factory-advisor	This workflow	Daily	✅ Meta-advisory
plan	/plan slash command	Slash command	✅ Good interactive pattern
secret-digger-{claude,codex,copilot}	Red team: escape container, steal secrets	Every hour	✅ Unique firewall-testing pattern; ⚠️ All 3 currently failing
security-guard	PR security review	PR open/sync	✅ High-value for security repo
security-review	Daily comprehensive security review	Daily	✅ Good; outputs to issues
smoke-{chroot,claude,codex,copilot}	End-to-end firewall smoke tests	PR + every 12h	✅ Critical validation; ⚠️ Multiple failures open
test-coverage-improver	Coverage gap PRs	Weekly	✅ Good; skip-if-match prevents PR spam
update-release-notes	Enhance release notes on publish	Release published	✅ Useful release automation

---

🚀 Actionable Recommendations

⚡ P0 — Implement Immediately

P0.1 — Issue Triage Agent

What: Auto-label new issues with appropriate categories and leave a welcome comment.

Why: This is "hello world" for agentic workflows and is completely missing from this repo. Issues accumulate without labels, making issue-monster less effective (it can't filter by type). With 1100+ issues, triage discipline matters for discoverability. The factory's triage workflow is used as the starter example in Claude Code GitHub Actions docs.

How: Add issue-triage-agent.md triggered on issues: [opened, reopened]. Use safe-outputs add-labels with allowed set: bug, enhancement, documentation, security, question, good-first-issue, firewall, containers, ci.

Effort: Low (< 1 hour)

---
on:
  issues:
    types: [opened, reopened]
  lockdown: false
permissions:
  issues: read
tools:
  github:
    toolsets: [issues, labels]
safe-outputs:
  add-labels:
    allowed: [bug, enhancement, documentation, security, question, good-first-issue, firewall, containers, ci]
  add-comment: {}
timeout-minutes: 5
---
# Issue Triage Agent
Label and briefly comment on newly opened issues in this firewall repository.

Add with: gh aw add-wizard githubnext/agentics/issue-triage

---

P0.2 — Static Analysis Report (Daily)

What: Daily agent that runs zizmor, poutine, and actionlint on the compiled workflow lock files, then publishes a discussion report.

Why: This repo already runs these tools during gh aw compile but produces no persistent daily report. The factory ran 57 analysis discussions and 12 Zizmor security reports. For a security firewall tool, daily static analysis reporting is table-stakes. Recent PR merges (iptables rules, Squid config changes) should trigger daily verification.

How: Add static-analysis-report.md triggered daily. Run agenticworkflows-compile with --zizmor --poutine --actionlint flags, analyze results, create a discussion summary.

Effort: Low (the compilation tooling already supports this)

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/static-analysis-report.md

---

🔥 P1 — Plan for Near-Term

P1.1 — Audit Workflows (Meta-Analytics)

What: A meta-agent that analyzes the health of all other agentic workflows — their run patterns, failure rates, token usage, output quality — and creates weekly summary discussions.

Why: With 28 workflows running on various schedules (several hourly), there's significant blind spots. Right now, multiple workflows are failing silently (secret-digger x3, CI doctor, smoke tests, build tests) with issues piling up but no dashboard. The factory's Audit Workflows agent created 93 discussions and drove 9 PRs through downstream agents. This would be especially valuable here to track the secret-digger red team effectiveness.

Effort: Medium

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/audit-workflows.md

---

P1.2 — Daily Malicious Code Scan

What: Agent that reviews recent commits (past 24-48 hours) for suspicious patterns: unexpected network calls, new data exfiltration paths, obfuscated code, unusual subprocess executions.

Why: This repo is a firewall — it's particularly important that no malicious code sneaks into the codebase that could bypass its own restrictions. This is different from the secret-diggers (which test if agents can escape the container) — this scans the source code itself for suspicious patterns. The factory runs this daily as part of its security layer.

Effort: Low-Medium

Add with: gh aw add-wizard githubnext/agentics/daily-malicious-code-scan

---

P1.3 — Breaking Change Checker

What: Watches PRs for changes that could break backward compatibility: CLI flag removals/renames, API contract changes, configuration format changes, container interface changes.

Why: AWF has external users who depend on stable CLI flags (--allow-domains, --dns-servers, --image-tag, etc.) and the Docker container interface. The update-release-notes workflow handles after release; this prevents breaking changes from being merged without explicit acknowledgment. Especially relevant given frequent PRs changing CLI behavior.

Effort: Low-Medium

Example trigger: pull_request: [opened, synchronize] with focus on src/cli.ts, src/types.ts, and containers/

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/breaking-change-checker.md

---

P1.4 — Container Base Image Freshness Advisor (Domain-Specific)

What: Weekly agent that checks if base Docker images used in this firewall (ubuntu/squid:latest, ubuntu:22.04, etc.) have security updates, newer versions, or known CVEs, and creates an issue with recommended updates.

Why: The firewall's security guarantees depend on secure container base images. The existing container-scan.yml (Trivy) runs on pushes but there's no proactive monitoring of base image freshness. A weekly agent that checks upstream image digests and known CVEs would close this gap. This is highly domain-specific and high-value for a security tool.

Effort: Medium (custom workflow, no exact template)

---

📅 P2 — Consider for Roadmap

P2.1 — Code Simplifier (Continuous)

What: Daily agent analyzing recent TypeScript commits for simplification opportunities (over-complex conditionals, redundant code, opportunities for standard library usage), creating PRs with improvements.

Why: The factory's Code Simplifier had 83% merge rate. This codebase has complex TypeScript (docker-manager.ts is 1500+ lines), and AI-assisted development tends to produce verbose code. Daily cleanup would improve maintainability over time.

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/code-simplifier.md

---

P2.2 — Contribution Guidelines Checker

What: PR-triggered agent that verifies: commit message format (Conventional Commits), PR title follows allowed scopes, description length, test coverage for changed files.

Why: The repo has strict conventions (Conventional Commits, only specific PR scopes allowed), enforced via pr-title.yml and commitlint.config.js. But PR body quality, test coverage requirements, and documentation updates are not checked. Adding an agentic checker that comments with specific guidance would reduce back-and-forth.

Add with: gh aw add-wizard githubnext/agentics/contribution-guidelines-checker

---

P2.3 — Documentation Link Checker

What: Weekly agent that crawls the docs-site (docs-site/src/content/docs/), checks for broken internal/external links, and creates a PR with fixes.

Why: The repo maintains an Astro Starlight documentation site with cross-references to GitHub PRs, issues, external docs, and internal pages. Links go stale as PRs merge and issues close. The factory uses a Link Checker from the agentics repo. This would be low-maintenance and catch documentation rot.

Effort: Low

Add with: gh aw add-wizard githubnext/agentics/link-checker

---

P2.4 — Grumpy PR Reviewer

What: General code quality PR reviewer that comments on potential bugs, code smells, missing error handling, and test gaps — complementing the existing security-guard (which focuses on security specifically).

Why: The security-guard workflow is excellent but narrowly scoped to security. A general-purpose code reviewer would catch: missing null checks, inconsistent error propagation, TypeScript anti-patterns, and missing integration test coverage for new features.

Effort: Low

Add with: gh aw add-wizard githubnext/agentics/grumpy-reviewer

---

P2.5 — Changeset Generator

What: Automates version bump decisions and changelog entries after PRs merge, proposing a PR with the appropriate semver bump and updated CHANGELOG based on Conventional Commit types.

Why: update-release-notes runs after a release is published. A Changeset workflow would proactively manage versioning, analyzing commits since the last release to propose the next version. The factory had 78% merge rate for this workflow. Removes human cognitive load from versioning decisions.

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/changeset.md

---

💡 P3 — Future Ideas

P3.1 — Weekly Issue Summary

What: Weekly digest discussion summarizing open issues by category, age, and priority — helping maintainers understand backlog health at a glance.

Add with: gh aw add-wizard githubnext/agentics/weekly-issue-summary

P3.2 — Portfolio Analyst (Cost Optimization)

What: Weekly analysis of workflow run costs, token usage patterns, and identifies over-expensive workflows for optimization.

Why: With hourly secret-diggers (3 engines × 24 runs/day = 72 runs/day), cost monitoring matters. The factory's Portfolio Analyst identified unnecessarily chatty LLM calls.

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/portfolio-analyst.md

P3.3 — Agentic Wiki Coder (Domain Knowledge Base)

What: Builds and maintains a wiki with explanations of security concepts, iptables rules, Squid configuration patterns, and domain whitelisting strategies used in this firewall.

Why: As the firewall grows more complex, having an agent-maintained knowledge base would help new contributors understand iptables/Squid concepts and firewall design decisions.

Add with: gh aw add-wizard githubnext/agentics/agentic-wiki-coder

P3.4 — Issue Arborist (Sub-Issue Linking)

What: Links related issues as parent/sub-issues to build dependency trees for related security improvements.

Add with: gh aw add-wizard https://github.com/github/gh-aw/blob/v0.45.5/.github/workflows/issue-arborist.md

---

📈 Maturity Assessment

Current Level: 4/5 — "Automated Factory"

The repository operates a comprehensive fleet of specialized agentic workflows covering security, testing, CI investigation, documentation, and issue management. Most workflows are properly compiled with security constraints, skip-if-match guards, and appropriate permissions scoping.

Target Level: 5/5 — "Self-Optimizing Factory"

A Level 5 repository has meta-agents monitoring the agent ecosystem itself, continuous improvement loops with measurable merge rates, and domain-specific workflows that no other repository would need.

Gap Analysis:

1. Visibility gap: No meta-analytics layer (Audit Workflows, Metrics Collector) to track agent ecosystem health → multiple failures accumulating silently
2. Triage gap: No issue labeling → issue-monster works on all issues without priority filtering
3. Security gap: No malicious code scan and no static analysis report workflow → weaker defense-in-depth for a security tool
4. Quality loop gap: No continuous code simplification → technical debt accumulates

---

🔄 Comparison with Pelis Agent Factory Best Practices

What This Repo Does Well ✅

• Multi-engine strategy: Running secret-diggers with Claude, Codex, and Copilot simultaneously is a sophisticated pattern not seen in most repos
• Domain-specific smoke tests: Running the actual firewall as a test (smoke-{claude,codex,copilot}) is exemplary domain-specific testing automation
• Security-first permissions: Workflows are properly scoped with minimal permissions
• Continuous dependency monitoring: The dependency-security-monitor creating both issues AND PRs for safe updates is excellent
• Shared imports: Using shared/mcp-pagination.md and shared/version-reporting.md for DRY workflow composition

What Could Improve ⬆️

• Meta-observability: No way to track which workflows are working vs failing in aggregate
• Issue lifecycle: Issues created by agents (security-review, ci-doctor) aren't being triaged/labeled, making them hard to prioritize
• Code quality loop: Unlike gh-aw (76 duplicate-detection PRs, 83% simplifier merge rate), there's no active code quality improvement cycle
• Domain expertise automation: Given the firewall domain, specialized automation for container security, ACL validation, and iptables rule analysis could be uniquely valuable and isn't found in generic workflow templates

---

📝 Notes for Future Runs

Stored in /tmp/gh-aw/cache-memory/patterns.md

Key changes to track:

• Whether issue triage (P0.1) gets implemented — enables better issue-monster effectiveness
• Whether secret-digger failures ([agentics] Secret Digger (Claude) failed #1168, [agentics] Secret Digger (Copilot) failed #1171, [agentics] Secret Digger (Codex) failed #1172) get resolved — currently all 3 engines failing
• CI Doctor failure ([agentics] CI Doctor failed #1164) — the investigator itself is broken, leaving CI failures uninvestigated
• Smoke test failures — multiple open, need systematic resolution

AI generated by Pelis Agent Factory Advisor

• expires on Mar 15, 2026, 3:27 AM UTC

[github-actions[bot]]

🔮 The ancient spirits stir... The smoke test agent has passed through and left its mark in the ether.

🔮 The oracle has spoken through Smoke Codex