Skip to content

Commit 8e90877

Browse files
committed
Add Plan Understanding Quiz pre-flight check for start-rlcr-loop
Add an advisory quiz that runs before the RLCR loop starts, verifying the user understands the technical implementation details of their plan. An opus-model agent analyzes the plan and generates 2 multiple-choice questions. If the user answers incorrectly, the system explains the plan and offers to stop or proceed. New flags: - --yolo: skip quiz and enable --claude-answer-codex (full automation) - --skip-quiz: skip quiz only without other behavioral changes gen-plan auto-start now passes --skip-quiz since the user already demonstrated understanding through the convergence discussion.
1 parent 55f1ab9 commit 8e90877

File tree

7 files changed

+183
-4
lines changed

7 files changed

+183
-4
lines changed

agents/plan-understanding-quiz.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
name: plan-understanding-quiz
3+
description: Analyzes a plan and generates multiple-choice technical comprehension questions to verify user understanding before RLCR loop. Use when validating user readiness for start-rlcr-loop command.
4+
model: opus
5+
tools: Read, Glob, Grep
6+
---
7+
8+
# Plan Understanding Quiz
9+
10+
You are a specialized agent that analyzes an implementation plan and generates targeted multiple-choice technical comprehension questions. Your goal is to test whether the user genuinely understands HOW the plan will be implemented, not just what the plan title says.
11+
12+
## Your Task
13+
14+
When invoked, you will be given the content of a plan file. You need to:
15+
16+
### Analyze the Plan
17+
18+
1. **Read the plan thoroughly** to understand:
19+
- What components, files, or systems are being modified
20+
- What technical approach or mechanism is being used
21+
- How different pieces of the implementation connect together
22+
- What existing patterns or systems the plan builds upon
23+
24+
2. **Explore the repository** to add context:
25+
- Check README.md, CLAUDE.md, or other documentation files
26+
- Look at the directory structure and key files referenced in the plan
27+
- Understand the existing architecture that the plan interacts with
28+
29+
### Generate Multiple-Choice Questions
30+
31+
Create exactly 2 multiple-choice questions that test the user's understanding of the plan's **technical implementation details**. Each question must have exactly 4 options (A through D), with exactly 1 correct answer.
32+
33+
- **QUESTION_1**: Should test whether the user knows what components/systems are being changed and how. Focus on the core technical mechanism or approach.
34+
- **QUESTION_2**: Should test whether the user understands how different parts of the implementation connect, what existing patterns are being followed, or what the key technical constraints are.
35+
36+
**Good question characteristics:**
37+
- Derived from the plan's specific content, not generic templates
38+
- Test understanding of HOW things will be done, not just WHAT the plan describes
39+
- Not too low-level (no exact line numbers, exact syntax, or trivial details)
40+
- A user who has carefully read and understood the plan should pick the correct answer
41+
- A user who just skimmed the title or blindly accepted a generated plan would likely pick wrong
42+
- Wrong options should be plausible (not obviously absurd) but clearly incorrect to someone who read the plan
43+
44+
**Example good questions:**
45+
- "How does this plan integrate the new validation step into the startup flow?" with options covering different integration approaches
46+
- "Which components need to change and why?" with options describing different component sets
47+
48+
**Example bad questions (avoid these):**
49+
- "What is the plan about?" (too vague, tests nothing)
50+
- "What are the risks?" (generic, not about implementation)
51+
- "On which line does function X start?" (too low-level)
52+
53+
### Generate Plan Summary
54+
55+
Write a 2-3 sentence summary explaining what the plan does and how, suitable for educating a user who showed gaps in understanding. Focus on the technical approach, not just the goal.
56+
57+
## Output Format
58+
59+
You MUST output in this exact format, with each field on its own line:
60+
61+
```
62+
QUESTION_1: <your first question>
63+
OPTION_1A: <option A text>
64+
OPTION_1B: <option B text>
65+
OPTION_1C: <option C text>
66+
OPTION_1D: <option D text>
67+
ANSWER_1: <A, B, C, or D>
68+
QUESTION_2: <your second question>
69+
OPTION_2A: <option A text>
70+
OPTION_2B: <option B text>
71+
OPTION_2C: <option C text>
72+
OPTION_2D: <option D text>
73+
ANSWER_2: <A, B, C, or D>
74+
PLAN_SUMMARY: <2-3 sentence technical summary>
75+
```
76+
77+
## Important Notes
78+
79+
- Always output all 13 fields - never skip any
80+
- ANSWER must be exactly one letter: A, B, C, or D
81+
- Randomize the position of the correct answer (do not always put it in A or D)
82+
- The plan may be written in any language - generate questions and options in the same language as the plan
83+
- Focus on substance over format
84+
- If the plan is very short or lacks technical detail, derive questions from whatever implementation hints are available
85+
- Questions should feel like a friendly knowledge check, not an adversarial interrogation
86+
87+
## Example Output
88+
89+
```
90+
QUESTION_1: How does this plan integrate the new validation step into the existing build pipeline?
91+
OPTION_1A: By replacing the existing lint step with a combined lint-and-validate step
92+
OPTION_1B: By adding a new PostToolUse hook that runs between the lint step and the compilation step
93+
OPTION_1C: By modifying the compilation step to include inline validation checks
94+
OPTION_1D: By creating a standalone pre-build script that runs before any other steps
95+
ANSWER_1: B
96+
QUESTION_2: Why does the plan require changes to both the CLI parser and the state file, rather than just the CLI?
97+
OPTION_2A: The state file stores the original CLI arguments for audit logging purposes
98+
OPTION_2B: The CLI parser is deprecated and the state file is the new configuration mechanism
99+
OPTION_2C: The CLI parser adds the flag, the state file persists it across loop iterations, and the stop hook reads it at exit time
100+
OPTION_2D: Both files share a common schema and must always be updated together
101+
ANSWER_2: C
102+
PLAN_SUMMARY: This plan adds a build output validation step by hooking into the PostToolUse lifecycle event. It modifies the hook configuration to insert a format checker between linting and compilation, and updates the state file schema to track validation results across RLCR rounds.
103+
```

commands/gen-plan.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -590,13 +590,15 @@ If all of the following are true:
590590
Then start work immediately by running:
591591
592592
```bash
593-
/humanize:start-rlcr-loop <output-plan-path>
593+
/humanize:start-rlcr-loop --skip-quiz <output-plan-path>
594594
```
595595
596+
The `--skip-quiz` flag is passed because the user has already demonstrated understanding of the plan through the gen-plan convergence discussion.
597+
596598
If the command invocation is not available in this context, fall back to the setup script:
597599
598600
```bash
599-
"${CLAUDE_PLUGIN_ROOT}/scripts/setup-rlcr-loop.sh" --plan-file <output-plan-path>
601+
"${CLAUDE_PLUGIN_ROOT}/scripts/setup-rlcr-loop.sh" --skip-quiz --plan-file <output-plan-path>
600602
```
601603
602604
If the auto-start attempt fails, report the failure reason and provide the exact manual command for the user to run:

commands/start-rlcr-loop.md

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
---
22
description: "Start iterative loop with Codex review"
3-
argument-hint: "[path/to/plan.md | --plan-file path/to/plan.md] [--max N] [--codex-model MODEL:EFFORT] [--codex-timeout SECONDS] [--track-plan-file] [--push-every-round] [--base-branch BRANCH] [--full-review-round N] [--skip-impl] [--claude-answer-codex] [--agent-teams]"
3+
argument-hint: "[path/to/plan.md | --plan-file path/to/plan.md] [--max N] [--codex-model MODEL:EFFORT] [--codex-timeout SECONDS] [--track-plan-file] [--push-every-round] [--base-branch BRANCH] [--full-review-round N] [--skip-impl] [--claude-answer-codex] [--agent-teams] [--yolo] [--skip-quiz]"
44
allowed-tools:
55
- "Bash(${CLAUDE_PLUGIN_ROOT}/scripts/setup-rlcr-loop.sh:*)"
66
- "Read"
77
- "Task"
8+
- "AskUserQuestion"
89
hide-from-slash-command-tool: "true"
910
---
1011

@@ -57,9 +58,58 @@ If any condition fails, skip the pre-check and let the setup script handle path
5758

5859
---
5960

61+
## Plan Understanding Quiz
62+
63+
Before running the setup script, verify the user genuinely understands what the plan will do. This is an advisory check -- it never blocks the loop, but catches "wishful thinking" users who blindly accepted a generated plan without reading it.
64+
65+
**Skip this entire quiz if** any of these conditions are true:
66+
- `$ARGUMENTS` contains `--skip-impl` (no plan to quiz about)
67+
- `$ARGUMENTS` contains `--yolo` (user explicitly opted out of all pre-flight checks)
68+
- `$ARGUMENTS` contains `--skip-quiz` (user explicitly opted out of the quiz)
69+
- `$ARGUMENTS` contains `-h` or `--help` (just showing help)
70+
- No plan content is available (the compliance pre-check was skipped because no plan file path could be determined)
71+
72+
### Run the quiz agent
73+
74+
1. Reuse the plan content that was already read during the compliance pre-check above (do not re-read the file).
75+
76+
2. Use the Task tool to invoke the `humanize:plan-understanding-quiz` agent (opus model):
77+
```
78+
Task tool parameters:
79+
- model: "opus"
80+
- prompt: Include the plan file content and ask the agent to:
81+
1. Explore the repository structure for context
82+
2. Analyze the plan's technical implementation details
83+
3. Generate 2 multiple-choice questions (4 options each) and a plan summary
84+
4. Return in the structured format: QUESTION_1, OPTION_1A-D, ANSWER_1, QUESTION_2, OPTION_2A-D, ANSWER_2, PLAN_SUMMARY
85+
```
86+
87+
3. **Parse the result**: Extract all 13 fields from the agent output (QUESTION_1, OPTION_1A through OPTION_1D, ANSWER_1, QUESTION_2, OPTION_2A through OPTION_2D, ANSWER_2, PLAN_SUMMARY). If the output is malformed (any field missing or ANSWER not A/B/C/D), warn: "Plan understanding quiz unavailable, continuing without it." and proceed to the Setup section below.
88+
89+
### Ask questions and evaluate
90+
91+
4. Use AskUserQuestion to present QUESTION_1 as a multiple-choice question with the 4 options (OPTION_1A through OPTION_1D). Compare the user's choice against ANSWER_1:
92+
- If the user selected the correct answer, mark QUESTION_1 as **PASS**
93+
- Otherwise, mark as **WRONG**
94+
95+
5. Use AskUserQuestion to present QUESTION_2 as a multiple-choice question with the 4 options (OPTION_2A through OPTION_2D). Compare the user's choice against ANSWER_2 using the same criteria.
96+
97+
### Decide whether to proceed
98+
99+
6. **If both questions PASS**: Briefly acknowledge ("Your understanding of the plan looks solid. Proceeding with setup.") and continue to the Setup section below.
100+
101+
7. **If one or both questions are WRONG**: Show the PLAN_SUMMARY to the user to help them understand what the plan does and the correct answers to the questions they missed. Then use AskUserQuestion with the question: "Would you like to proceed with the RLCR loop anyway, or stop and review the plan more carefully first?" with these choices:
102+
- "Proceed with RLCR loop"
103+
- "Stop and review the plan first"
104+
105+
- If the user chooses **"Proceed with RLCR loop"**: Continue to the Setup section below.
106+
- If the user chooses **"Stop and review the plan first"**: Report "Stopping. Please review the plan file and re-run start-rlcr-loop when ready." and **stop the command**.
107+
108+
---
109+
60110
## Setup
61111

62-
If the pre-check passed (or was skipped), execute the setup script to initialize the loop:
112+
If the pre-check passed (or was skipped), and the quiz passed (or was skipped or user chose to proceed), execute the setup script to initialize the loop:
63113

64114
```bash
65115
"${CLAUDE_PLUGIN_ROOT}/scripts/setup-rlcr-loop.sh" $ARGUMENTS

docs/usage.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,9 @@ OPTIONS:
5050
--agent-teams Enable Claude Code Agent Teams mode for parallel development.
5151
Requires CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 environment variable.
5252
Claude acts as team leader, splitting tasks among team members.
53+
--yolo Skip Plan Understanding Quiz and let Claude answer Codex Open
54+
Questions directly. Alias for --skip-quiz --claude-answer-codex.
55+
--skip-quiz Skip the Plan Understanding Quiz only (without other changes).
5356
-h, --help Show help message
5457
```
5558

scripts/setup-rlcr-loop.sh

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,13 @@ OPTIONS:
8989
--agent-teams Enable Claude Code Agent Teams mode for parallel development.
9090
Requires CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 environment variable.
9191
Claude acts as team leader, splitting tasks among team members.
92+
--yolo Skip Plan Understanding Quiz and let Claude answer Codex Open
93+
Questions directly. Convenience alias for --skip-quiz
94+
--claude-answer-codex. Use when you trust the plan and want
95+
maximum automation.
96+
--skip-quiz Skip the Plan Understanding Quiz only (without other behavioral
97+
changes). The quiz is an advisory pre-flight check that verifies
98+
you understand the plan before committing to an RLCR loop.
9299
--allow-empty-bitlesson-none
93100
Allow BitLesson delta with action:none even with no new entries (default)
94101
--require-bitlesson-entry-for-none
@@ -120,6 +127,8 @@ EXAMPLES:
120127
/humanize:start-rlcr-loop docs/impl.md --max 20
121128
/humanize:start-rlcr-loop plan.md --codex-model ${DEFAULT_CODEX_MODEL}:${DEFAULT_CODEX_EFFORT}
122129
/humanize:start-rlcr-loop plan.md --codex-timeout 7200 # 2 hour timeout
130+
/humanize:start-rlcr-loop plan.md --yolo # skip quiz, full automation
131+
/humanize:start-rlcr-loop plan.md --skip-quiz # skip quiz only
123132
124133
STOPPING:
125134
- /humanize:cancel-rlcr-loop Cancel the active loop
@@ -235,6 +244,14 @@ while [[ $# -gt 0 ]]; do
235244
AGENT_TEAMS="true"
236245
shift
237246
;;
247+
--yolo)
248+
ASK_CODEX_QUESTION="false"
249+
shift
250+
;;
251+
--skip-quiz)
252+
# No-op in setup script; quiz logic lives in command markdown
253+
shift
254+
;;
238255
--allow-empty-bitlesson-none)
239256
BITLESSON_ALLOW_EMPTY_NONE="true"
240257
shift

skills/humanize-rlcr/SKILL.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,8 @@ Pass these through `setup-rlcr-loop.sh`:
107107
| `--push-every-round` | Require push each round | false |
108108
| `--claude-answer-codex` | Let Claude answer open questions directly | false |
109109
| `--agent-teams` | Enable agent teams mode | false |
110+
| `--yolo` | Skip quiz and enable --claude-answer-codex | false |
111+
| `--skip-quiz` | Skip Plan Understanding Quiz (implicit in skill mode) | false |
110112

111113
Review phase `codex review` runs with `gpt-5.4:high`.
112114

skills/humanize/SKILL.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,8 @@ Transforms a rough draft document into a structured implementation plan with:
9696
- `--push-every-round` - Require git push after each round
9797
- `--claude-answer-codex` - Let Claude answer Codex Open Questions directly (default is AskUserQuestion)
9898
- `--agent-teams` - Enable Agent Teams mode
99+
- `--yolo` - Skip Plan Understanding Quiz and enable --claude-answer-codex
100+
- `--skip-quiz` - Skip the Plan Understanding Quiz only
99101

100102
### Cancel RLCR Loop
101103

0 commit comments

Comments
 (0)