Magi PRD Generation Flow

[jhorst11]

What does this PR do?

This implements the first workflow for magi, our SDD process within Magen.

It implements the design here (some modifications to the design doc in this PR)

Refer to documentation for how this is structured.

Review should primarily focus on the graph, state and input/output schemas.

There is a lot of duplication with our main magen workflow which much will disappear as we work on abstracting out the common bits (actively being worked on)

What issues does this PR fix or reference?

W-20034988

[khawkins]

No action for this PR, but we should sync up on specifics for abstracting the "working directory" functionality in the extraction. We'll probably need some tweaking on one end or the other (or both).

[jhorst11]

@khawkins this one is ready for another look!

[khawkins]

I know this is intentionally a bit hand wavey here, but I'm specifically curious about how userUtterance relates to userInput. Are we envisioning that, as part of extraction, we would keep the user utterance wholesale, as part of the extraction?

[jhorst11]

agreed the naming is complicated - userUtterance is meant to capture the original user input so we never lose track of their original intent. userInput is the god-object that comes in for every orchestrator call

[khawkins]

This is a nit of consideration, but you may want to either validate that the LLM didn't create its own incremented numeric ID as part of creating the featureId, or explicitly tell the LLM not to do that.

[jhorst11]

moved validation (including enforcing non numerical starts) to the schema, good suggestion.

[jhorst11]

I guess the downfall of that approach however is if these methods were to used outside of this direct use case and the other consumers did not do upstream validation - we should probably put some stronger stakes in the ground from a code guidelines perspective of where validation SHOULD occur in magen in general

[khawkins]

This doesn't have to be now, but generally speaking when data manipulation logic gets sufficiently tangled inline with the prompt template it's populating, I prefer to pull out the data manipulation logic into its own method, and encapsulate that work away from the prompt template.

Honestly at some point we probably need to move to a strict prompt template model, where the prompt template text is wholly static, and we replace inline keys with passed-in string values.

[jhorst11]

do you think I'm better of giving the LLM instructions on formatting?

i.e.

{gapsJson}

### Format for Reference

When reviewing the gaps above, format each gap as follows:

**Example Gap Format:**
\`\`\`
1. **Gap Title** (severity level)
- ID: gap-id
- Category: Category Name
- Description: Detailed description of what is missing
- Impact: What happens if this gap is not addressed
- Suggested Requirements: Requirement Title 1, Requirement Title 2, ...
\`\`\`

**Example:**
\`\`\`
1. **Missing Authentication Flow** (high severity)
- ID: GAP-001
- Category: Security
- Description: The requirements do not specify how users authenticate to the app
- Impact: Users cannot securely access the application, leading to security vulnerabilities
- Suggested Requirements: User Login Requirement, Biometric Authentication Support
\`\`\`

[khawkins]

Nit: for readability in development, we probably need to establish a max column width for our string lines. I think my thumb in the air has traditionally been to line break around 90 chars, but we could just formally enforce it by configuring max-len in our ESLint rules.

[jhorst11]

agreed - I suggest we do that as a fast follow in case it touches a bunch across the project and creates a lot of noise

[khawkins]

Just an observation: this prompt is pretty heavy on instructions / responsibilities. Have you had any problems with deviations from the script?

[khawkins]

There's also a lot of use of "CRITICAL, IMPORTANT, MANDATORY, REMEMBER". This is often a sign that a set of tasks needs to be decomposed into smaller sets of responsibility.

[jhorst11]

yeah this is one of the more complicated nodes - so far it's behaved well for me but I think it's one to put on the watch list as we start testing. The complexity comes because there's a binary decision to be made: "do you want to review requirements or finalize with pending changes?" - technically we can split that decision out into a branching node but I think it would negatively impact user experience where a simple yes/no would kick of a chain of tool calls - but we'll see maybe that will be necessary... I always waffle between how small of tasks each node should do at the expense of user experience