-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Description
This issue is actually an observation, and the community can confirm the point I'll make.
Validation-gates.md states the following:
3. Iterative Fix Process
When tests fail:
Analyze the failure carefully
Identify the root cause
Implement a fix
Re-run tests to verify the fix
Continue iterating until all tests pass
Document any non-obvious fixes
I see in your generate-prp.md that you are asking to pass context to the subagent, but I've seen it where this may not happen. If this validation-gates agent lacks context from Claude.md, such as PLANNING and TASK, then during the iteration phase, it will completely fail while attempting to pass the test. I've seen this repeatedly, where the LLM resorts to any code to pass the test, in clear violation of your rules and architecture.
Am I the only one who perceives this?
Also, apply a bit more 'engineering' to the whole concept. Kudos for the effort, but there are CLEAR gaps in your reasoning.
The whole INITIAL.md is very confusing since most of the features will come up after the product is in use.
I think it should be called "FEATURE_REQUIREMENTS.md" (or something similar that indicates it is an 'ever-changing' document) and populated with the conclusions of a conversation where the developer explains the feature's intent.
Then, using the information already provided in CLAUDE, PLANNING, etc., the LLM should formulate 10 clarifying questions and then update them with those answers.
All that conversation should then be extracted into the "FEATURE_REQUIREMENTS.md" file, and you can then start the cycle. You should also be able to specify commands to create folders according to feature requirements, allowing you to isolate these tasks and retain them for long-term memory.
Regarding validation gates, I think you should keep it as a 'revisionist', but the output should be very cohesive in terms of where to fix the code, rather than trying to fix it.
Then you could have a 'test subagent' that knows everything about the architecture, styling, patterns, etc., and let it fix it. But, it already sounds like something the main conversation should be responsible for, not the subagent.
Last, but not least, PRP sounds cool and all, but it is totally made up. It is nothing but a well-structured and rich PRD, which is the correct term in software development. It will help you with people adopting this repo.
Again, kudos and keep improving it. It is a good first step.