chore: add grill-me-qa skill

Author: fugazi

.agents/skills/grill-me-qa/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Douglas Fugazi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

.agents/skills/grill-me-qa/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: grill-me-qa
+description: "A relentless interview to stress-test QA automation plans, test strategies, and framework designs before implementation. Use when the user wants to validate a test architecture, challenge a testing decision, prepare an AI-assisted testing rollout, or uses any 'grill' trigger phrases. Covers test strategy, framework selection, AI integration, CI/CD, maintainability, and quality engineering."
+---
+
+# Grill Me QA
+
+A relentless, systematic interview that stress-tests every aspect of a QA automation plan or test strategy until all decisions are resolved. Designed for QA Automation Engineers, SDETs, and QA Leads working with AI-assisted testing tools.
+
+> **Activation:** Triggered when the user wants to validate, challenge, or stress-test a testing plan, framework choice, test architecture, or AI-testing strategy. Also activated by explicit reference: "use the skill grill-me-qa".
+
+## When to Use This Skill
+
+- Before starting a new test automation project or framework
+- When adopting AI tools for test generation, healing, or analysis
+- Before presenting a test strategy to stakeholders
+- When refactoring or migrating an existing test suite
+- When evaluating tool/framework trade-offs (Playwright vs Selenium vs Cypress vs k6)
+- Before scaling test coverage or reorganizing a regression suite
+- When flaky tests are undermining trust in the suite
+
+## How Grilling Works
+
+### Core Protocol
+
+1. **One question at a time.** Never ask multiple questions simultaneously. Wait for the user's answer before proceeding.
+2. **Provide a recommended answer** for every question, based on industry best practices, ISTQB principles, and the specific context of the project.
+3. **Explore before asking.** If a question can be answered by examining the codebase, test suite, CI configuration, or existing documentation — explore it instead of asking the user.
+4. **Walk the decision tree systematically.** Follow the seven dimensions in order. Resolve dependencies between decisions one by one before moving to the next branch.
+5. **Challenge vague answers.** If the user's answer is imprecise, follow up to sharpen it. "It depends" is not an answer — force a concrete decision with context.
+6. **Track decisions.** Maintain a running log of resolved decisions, deferred items, and open questions throughout the session.
+
+### Session Flow
+
+```
+Phase 1: Context Gathering
+  → What are we grilling? (plan, strategy, framework choice, AI adoption)
+  → Explore codebase, test suite, CI config, existing docs
+
+Phase 2: Systematic Grilling (7 Dimensions)
+  → Walk through each dimension, one question at a time
+  → Provide recommended answer, wait for user's decision
+  → Resolve dependencies before moving forward
+
+Phase 3: Decision Record Generation
+  → Summarize all decisions into a Test Strategy Decision Record (TSDR)
+  → Offer Markdown and/or HTML output
+```
+
+## The Seven Dimensions
+
+Each dimension contains a decision tree with specific interrogation points. Read `references/qa-decision-tree.md` for the full question bank.
+
+### Dimension 1: Test Strategy & Coverage
+- What testing pyramid shape fits this project?
+- What is the risk-based coverage priority?
+- What is the boundary between automated and exploratory testing?
+- What non-functional testing is required (performance, security, accessibility)?
+
+### Dimension 2: Framework & Tooling
+- Why this framework over alternatives? (Playwright / Selenium / Cypress / k6 / other)
+- What language and why? (TypeScript / Java / Python / other)
+- What design pattern? (Page Object Model / Screenplay / Fluent API / hybrid)
+- What reporting and observability tools?
+
+### Dimension 3: Test Architecture
+- How are tests isolated from each other?
+- What is the test data strategy? (seeding, cleanup, factories, fixtures)
+- How are environments managed? (local, staging, CI)
+- What is the parallelization strategy?
+- What is the retry and flakiness policy?
+
+### Dimension 4: AI Integration
+- Where does AI enter the testing workflow? (generation, healing, analysis, visual, triage)
+- How are AI-generated tests validated for correctness?
+- What is the human-in-the-loop boundary?
+- How are hallucinations and non-deterministic outputs mitigated?
+- What is the cost/token budget for AI-assisted testing?
+- What AI tools are selected and why?
+
+### Dimension 5: CI/CD Pipeline
+- What are the test stages and quality gates?
+- What is the execution budget? (time limit per stage)
+- What is the test selection strategy? (impacted tests, smart selection, full suite)
+- Fail-fast or comprehensive-then-report?
+- How are flaky tests handled in CI? (quarantine, auto-retry, block merge)
+
+### Dimension 6: Quality Engineering (Non-Functional)
+- What accessibility standard is targeted? (WCAG 2.1 AA / Section 508 / other)
+- What performance testing is needed? (load, stress, soak, spike)
+- What visual regression approach? (pixel diff, DOM diff, AI-assisted)
+- What is the cross-browser and cross-device matrix?
+- What security testing is integrated? (SAST, DAST, dependency scanning)
+
+### Dimension 7: Maintainability & Sustainability
+- What is the dead test detection and removal strategy?
+- How is test code quality enforced? (linting, review standards, DRY thresholds)
+- What is the refactoring cadence?
+- How are tests documented? (naming, comments, test plan links)
+- What is the onboarding path for new team members?
+
+## Interrogation Guidelines for AI-Assisted Testing
+
+When the grilling touches AI integration (Dimension 4), apply additional scrutiny. AI in testing is high-risk if unvalidated. Read `references/ai-testing-interrogation.md` for the full question bank.
+
+Key areas:
+- **Validation:** How do you know the AI-generated test is actually testing the right thing?
+- **Determinism:** Can the test pass for the wrong reason?
+- **Maintenance:** What happens when the AI tool changes its output format?
+- **Ownership:** Who owns the test — the human or the AI tool?
+- **Audit trail:** Can you explain why a test exists and what it verifies?
+
+## Output: Test Strategy Decision Record (TSDR)
+
+At the end of the grilling session, generate a TSDR summarizing all decisions. Use the templates provided:
+
+- **Markdown:** `templates/grilling-summary.md` — for developers and version control
+- **HTML:** `templates/grilling-summary.html` — for stakeholders, presentations, and sharing
+
+The TSDR includes:
+- Project name, date, and participants
+- Per-dimension status (Resolved / Deferred / Open)
+- Decision log with: question, recommended answer, chosen answer, rationale
+- Outstanding items and follow-up actions
+
+## Stopping Criteria
+
+End the grilling session when:
+1. All seven dimensions have been covered (even if some are deferred)
+2. The user explicitly asks to stop
+3. The remaining open questions are blocked by external dependencies (not decisions)
+
+Never end mid-dimension unless the user requests it. Always summarize the current state before closing.
+
+## Tone
+
+Be relentless but constructive. The goal is not to overwhelm — it is to surface hidden assumptions and force clarity. Treat every decision as important, but prioritize questions that have the highest impact on the testing strategy.
+
+Do not let the user skip questions without acknowledging the risk. If a question is deferred, mark it explicitly and note the impact of the deferral.

.agents/skills/grill-me-qa/references/ai-testing-interrogation.md

@@ -0,0 +1,161 @@
+# AI Testing Interrogation — Expanded Question Bank
+
+When the grilling touches Dimension 4 (AI Integration), apply heightened scrutiny. AI in testing is powerful but introduces non-determinism, trust issues, and maintenance burdens that must be explicitly addressed.
+
+---
+
+## 4.1 Where does AI enter the testing workflow?
+
+### Interrogation points
+
+- **Test generation:** Is AI generating test code from specs, Gherkin, or natural language descriptions?
+- **Test healing:** Is AI auto-fixing broken selectors, assertions, or page structure changes?
+- **Visual regression:** Is AI being used for intelligent visual diffing (vs pixel comparison)?
+- **Triage & analysis:** Is AI analyzing test failures, grouping similar failures, or suggesting root causes?
+- **Test selection:** Is AI selecting which tests to run based on code changes?
+- **Test data generation:** Is AI generating test data, edge cases, or boundary values?
+
+### Recommended answer
+
+Adopt AI incrementally, lowest-risk first:
+
+1. **Triage** (lowest risk) — AI analyzes failures and groups them. Wrong output wastes time, doesn't cause false confidence.
+2. **Visual regression** — Mature technology. AI-assisted diffing reduces false positives vs pixel diff.
+3. **Test generation** (medium risk) — AI generates test drafts. Mandatory human review before merge.
+4. **Test healing** (higher risk) — Auto-fixing broken tests. Must be opt-in, logged, and reviewed.
+5. **Test selection** (highest risk) — AI decides what to test. Always maintain a fallback to full suite.
+
+Never deploy AI-based test selection without a safety net (random sampling of non-selected tests).
+
+---
+
+## 4.2 How are AI-generated tests validated for correctness?
+
+### Interrogation points
+
+- Who reviews AI-generated tests before they are merged?
+- What criteria define a "correct" AI-generated test?
+- How do you verify the test is testing the right thing — not just passing?
+- What is the mutation testing strategy? (Can the test catch intentional bugs?)
+- How do you handle tests that pass for the wrong reason (tautological tests)?
+
+### Recommended answer
+
+Every AI-generated test must pass a four-gate review:
+
+1. **Behavioral gate:** Does the test verify the intended behavior? (Reviewer reads the test and confirms it matches the spec.)
+2. **Mutation gate:** Introduce a deliberate bug in the code under test. Does the test fail? If not, the test is worthless.
+3. **Stability gate:** Run the test 10 times. If it flakes even once, it fails.
+4. **Readability gate:** Can a new team member understand what the test does and why it exists without asking the author?
+
+AI-generated tests that fail any gate are rejected, not patched. Re-prompt and regenerate.
+
+---
+
+## 4.3 What is the human-in-the-loop boundary?
+
+### Interrogation points
+
+- What decisions can the AI make autonomously?
+- What decisions require human approval?
+- Is there a "trust threshold" after which the AI gets more autonomy?
+- How is the audit trail maintained? (Who decided what, when?)
+- What happens when the AI and the human disagree?
+
+### Recommended answer
+
+Define clear autonomy tiers:
+
+| Tier | AI Action | Human Role |
+|------|-----------|------------|
+| 1 | Suggest | Human reviews and decides |
+| 2 | Draft | Human reviews, edits, approves |
+| 3 | Execute + log | Human reviews logs post-action, can revert |
+| 4 | Autonomous | No human review (reserved for lowest-risk, well-established patterns) |
+
+Start everything at Tier 1-2. A task can only advance to Tier 3 after 95%+ accuracy over 100+ executions with human verification. Tier 4 is reserved for deterministic, reversible actions (e.g., formatting test output).
+
+Maintain a complete audit log: every AI action records what was done, what prompt triggered it, what model was used, and what the outcome was.
+
+---
+
+## 4.4 How are hallucinations and non-deterministic outputs mitigated?
+
+### Interrogation points
+
+- What temperature/decoding settings are used for test generation?
+- How are AI outputs validated against the actual application behavior?
+- What happens when the AI generates code that doesn't compile or references non-existent elements?
+- How do you handle AI tests that pass today and fail tomorrow (semantic drift)?
+
+### Recommended answer
+
+- Use low-temperature settings (0.0-0.3) for test generation to reduce randomness.
+- Validate every AI-generated selector/assertion against the live application during review. Never trust AI descriptions of UI structure.
+- For semantic drift: run AI-generated tests in a "canary" mode for 7 days before promoting to the main suite. If the test flakes or fails during canary, it is not ready.
+- Maintain a feedback loop: track AI test quality metrics (rejection rate, post-merge flakiness, mutation score) and feed back into prompt engineering.
+
+---
+
+## 4.5 What is the cost/token budget for AI-assisted testing?
+
+### Interrogation points
+
+- What is the per-PR cost of AI test generation?
+- What is the monthly cost of AI triage/analysis?
+- How is cost tracked and attributed?
+- What happens when the budget is exceeded?
+- Are there fallback strategies when AI is unavailable (rate limits, outages)?
+
+### Recommended answer
+
+- Track AI cost per category: generation, triage, healing, selection.
+- Set a monthly budget with alerts at 50%, 75%, and 100%.
+- When budget is exceeded: generation stops (falls back to manual), triage continues (low cost), healing continues (high value).
+- Always maintain a non-AI fallback path for every workflow. If the AI provider is down, tests must still run, triage must still happen (manually), and the pipeline must not break.
+- Cost efficiency metric: cost per valid test generated. If AI generates 100 tests at $50, but only 20 pass all four gates, the real cost is $2.50 per valid test. Track this.
+
+---
+
+## 4.6 What AI tools are selected and why?
+
+### Interrogation points
+
+- What tools were evaluated? (GitHub Copilot, Cursor, Claude, ChatGPT, specialized tools)
+- What are the selection criteria? (accuracy, integration, cost, ecosystem, security)
+- Does the tool support the team's framework and language?
+- Is the tool's output deterministic enough for CI use?
+- What is the vendor lock-in risk?
+- How does the tool handle proprietary or sensitive code?
+
+### Recommended answer
+
+Document a tool comparison matrix with these criteria:
+
+1. **Accuracy** — Mutation score of generated tests (objective metric)
+2. **Integration** — Does it work with the existing framework/CI/IDE?
+3. **Cost** — Per-developer or per-request pricing
+4. **Security** — Does the tool train on your code? Is data sent to external servers?
+5. **Determinism** — Same input, same output? (Critical for CI reproducibility)
+6. **Portability** — Can you switch tools without rewriting all tests?
+
+Prefer tools that integrate with the existing IDE and CI pipeline over standalone tools. Avoid tools that require sending proprietary code to external servers without explicit security review. For test generation specifically, evaluate with a standardized benchmark (e.g., generate tests for 10 known features and score the output).
+
+---
+
+## 4.7 Ownership: Who owns the test — the human or the AI tool?
+
+### Interrogation points
+
+- If an AI-generated test fails in production, who is responsible?
+- Is there a code ownership header or annotation on AI-generated tests?
+- How are AI-generated tests maintained over time?
+- Can the same AI tool update the test later, or must a human do it?
+
+### Recommended answer
+
+- **Humans own all tests, regardless of how they were created.** AI is a tool, not an author.
+- Every AI-generated test must have a human owner (code ownership annotation or team assignment).
+- Tag AI-generated tests in the test metadata (e.g., `@ai-generated @reviewed-by:username`) for tracking quality over time.
+- When an AI-generated test breaks, the human owner is responsible for fixing it — not delegating back to the AI tool. If the human cannot understand the test well enough to fix it, the test should be deleted and rewritten.
+- Track AI vs human test quality metrics separately: flakiness rate, bug detection rate, maintenance cost. If AI tests are consistently lower quality, adjust the generation strategy.