Skip to content

feat: bring product discovery into Ideation — an idea-to-decision flow that feeds Inception (2.2.11)#526

Open
akash-doshi wants to merge 5 commits into
awslabs:v2from
akash-doshi:pr1-discovery-scope
Open

feat: bring product discovery into Ideation — an idea-to-decision flow that feeds Inception (2.2.11)#526
akash-doshi wants to merge 5 commits into
awslabs:v2from
akash-doshi:pr1-discovery-scope

Conversation

@akash-doshi

@akash-doshi akash-doshi commented Jul 6, 2026

Copy link
Copy Markdown

Implements the first slice of RFC #517 (Bring product discovery into the Ideation phase).

Summary

Ideation today starts every initiative from a blank questionnaire, and nothing between the first answer and the first build checks whether the direction is right. This PR brings product discovery into the Ideation phase as an opt-in flow. It extends the existing intent-capture stage in place so every scope works materials-first, and adds four new ideation stages under a new discovery scope that explore and validate an idea before any commitment to build: evidence how things work today, frame what the future could look like as genuinely different options, test the riskiest assumptions first, and end in an explicit decision with a plain meaning — commit (build it), pivot (keep what was learned, reframe), or park (stop for now, keep everything, reopen any time).

What changes, mechanically

  1. intent-capture (1.1), extended additively for every scope that runs it: one intake question (asks, materials, mandates, decide-by dates) asked through the stage's own questions file; answers found in materials carry their source and stay unconfirmed until the person validates them at the gate; the intent statement gains three conditional sections; the stage produces two new records (source-inventory, open-questions-record). The clarifying questions themselves are unchanged; a stock run gains the intake question, the two additive records, and a confirm-unconfirmed-answers duty at the gate (the CHANGELOG states this plainly).
  2. Four new stages, discovery scope only: discovery-current-state (1.8, ends in a playback the person confirms), discovery-future-state (1.9, at least three structurally different framings, the person chooses), discovery-experimentation (1.10, riskiest assumptions first, append-only evidence), discovery-decision (1.11, the decision pack and the decision).
  3. On commit the person chooses where the build happens, and the validated vision is wired into delivery rather than left to conductor initiative. The decision stage appends a structured handoff contract to the pack (settled entries as pre-answers, exclusions as boundaries, open items with recommendations, promoted starting points, and a map from delivery's first questions to where the record answers them) and folds a "What Discovery Validated" section back into the intent statement. requirements-analysis and user-stories declare decision-pack as an optional consume (required: false), and refined-mockups declares design-language-record and evidence-record the same way — when the artifacts exist, the run-stage directive hands delivery their paths and upstream-coverage verifies they were referenced; when they do not (every stock run), the resolver drops the entries silently. Continue here relays the engine's own verbs (scope-change, set-status, state skip with a reason naming the decision pack) so the same workflow and record proceed into Inception — pinned by a deterministic test (t214) in both directions. Hand off completes the workflow and the pack travels to the receiving team. Both paths verified live.
  4. One scope file (discovery, zero keywords, explicit selection only), nine framework-default templates (the first tier-2 templates to ship; intent-statement deliberately has none), a shared test-primitives knowledge file, a new user-guide chapter (Running a Discovery Initiative), glossary entries, and the full consistency surface (SKILL tables, stage graph 32 → 36, docs, tests). Version 2.2.11.

Why this shape

One intent stage, not two — the same human moment should not have two implementations, and the shared entry is what makes native continuation into delivery possible. Materials-first because nobody should re-type what their documents already say; every borrowed answer carries its source and is confirmed by a person before it becomes fact. Gates where errors are cheapest. Three structurally different framings minimum, because variations of one idea are a rubber stamp. Riskiest-first testing, because the assumption most likely to kill the initiative should be the first money spent. The pack travels before the meeting. Opt-in with zero keywords, so nobody's routing changes by accident. The six questionnaire stages are untouched — replacement is a later, evidence-gated proposal (graduation criteria in the RFC), not a silent switch.

Non-breaking evidence for the four stock scopes

  • Every stock scope's compiled plan gains exactly four SKIP rows, with every other row byte-identical — nothing removed, nothing reordered (see the mr9-parity fixture diffs, the strongest single artifact in this PR). Stock walk fixtures are byte-identical.
  • The engine's only hard artifact guard passes when any one declared produce exists, and sensors are advisory by contract, so no new hard-failure path exists.
  • The clarifying questions, their artifacts' shapes, and the gate are unchanged. A stock run gains one intake question plus two additive records.
  • t213 pins the contract with exact pins: intent-capture keeps exactly its four stock scopes plus discovery, produces as an exact ordered list, the four new stages declare scopes: [discovery] only. t214 pins the continuation verb sequence against a completed-discovery fixture (including a contrast test that recompose refuses the same state) and the optional-consume differential (pack present: the requirements-analysis directive carries its path; absent: no trace).

Pairing with PR 2

This PR and the skills PR that follows are a pair: this one ships the flow's structure, the next ships the craft the agents load to execute each move deeply (framing generation from structurally different starting points, design-language capture, the five test moves as per-move playbooks, narrative generation). The structure runs on its own and is provably non-breaking, but the method's full depth arrives with the pair, so we ask that both be reviewed together and merged together — packaged however this repo prefers: one PR with two clean commits, two PRs with the second marked stacked, or a staging branch on your side.

Change inventory and review guide

316 files; 187 are generated dist/ copies (verify with bun scripts/package.ts --check, never by hand). The branch is five commits — the feature, then four review-response fix rounds — so the fixes from the first review are individually diffable. The authored surface, in review order:

Read closely Why
core/aidlc-common/stages/ideation/intent-capture.md The one edit to a stage every scope runs. Diff it line by line: the four clarifying questions and gate text unchanged, every addition additive, every ask through the questions file
tests/integration/t213-discovery-scope.test.ts, t214-discovery-continuation.test.ts + tests/fixtures/mr9-parity/* diffs The safety case
The four discovery-*.md stages Frontmatter contracts; every named engine verb is real (jump, park, scope-change, set-status, state skip)
core/scopes/aidlc-discovery.md + nine core/tools/data/templates/ Template H2 sets are the sensor contract; intent-statement deliberately has no template
core/knowledge/aidlc-shared/discovery-test-primitives.md The five test moves and their honesty caps
docs/guide/18-discovery-walkthrough.md The user-truth check — what a first-time reader is promised

Mechanical (verify only): the pinned stage numbers 1.8–1.11 in the four committed stage-graph.json files (per the pin-and-harvest practice in docs/reference/15-stage-definition.md), one-line scope tags on the three initialization stages, statusline/state-template/SKILL tables/welcome banner, the version trio (t68 guards it), comment-only count corrections in two sensor sources and aidlc-graph.ts. Docs: 29 files, mostly count/table consistency plus the new chapter and six glossary entries. Tests: 20 count-pin moves plus regenerated fixtures.

Notes for maintainers

  • We support RFC [RFC]: v2 - Author stage number/name in core/, no hand-editing per-harness stage-graph.json #478 (authoring stage numbers in core) — it would remove this PR's pinned-row edits.
  • Found while building: the 2.2.0 CHANGELOG bullet saying a maintainer compile drops composed scopes appears stale — mergeComposedScopes preserves them (aidlc-graph.ts:1255-1277, wired at :1505-1521).
  • The nine templates ride here because the sensor contract binds them to the stages; say the word and they split into their own PR.
  • The RFC's portfolio view is deliberately absent here and severable.
  • Small upstream gaps we noticed and did not touch (happy to file issues): the Phase Progress statusline can display stale after a completed stage; sensor outcomes are silent on the terminal (events land in the audit shard, findings under .aidlc-sensors/); park promises /aidlc --resume, which works but is missing from the help text; the state-file legend renders [x]-beside-SKIP confusingly after a scope change. One pre-existing doc error (02-your-first-workflow.md named a nonexistent artifact and five questions where the stage asks four) is fixed here since we touched the page anyway.

Verification

bun scripts/package.ts --check green on all four harnesses. Deterministic suite: 240 files, 3,838 assertions, one known environmental failure (t92 — reproduced identically at the untouched v2.2.10 base in this environment). The three sdk-live tests that pin scope counts (t189, t192, t193) are updated for the tenth scope; they skip in this environment (no live credentials), so that fix is pin-correct by arithmetic rather than run-proven here — stated so this verification claim carries its environment scope. Coverage registry regenerated and fresh. Live smoke: discovery birth routes to intent-capture, the template-to-sensor round trip fires and fails correctly, a full run reaches the decision, the commit continuation proceeds into Inception in the same workflow, and the intent-statement write-back's sensor behavior was proven with a live hook run.

@apackeer apackeer left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes

The hypothesis-testing loop this PR builds is real, and the non-breaking case for the stock scopes genuinely holds (details at the bottom). The main ask is a design one: the flow's central promise - that discovery's validated vision sets up delivery - is only half-wired. After that come three test failures and a rebase collision, all mechanical.

1. The vision does not mechanically reach Inception (the main ask)

The tenet this PR implements: Ideation's output is a validated vision, which sets up Inception (the implementation plan), which sets up Construction (code). The discovery flow builds that vision honestly: future-state + assumptions-record (1.9) feed test-plans + evidence-record (1.10), riskiest-first and append-only, and the decision pack (1.11) is the vision artifact, with a machine-readable handoff contract on commit.

But on the continue-here path, no Inception stage consumes any of it. Grepping every inception and construction stage plus all agents: zero references to decision-pack, evidence-record, assumptions-record, or future-state. Inception's entry stage (requirements-analysis) declares only intent-statement and scope-document, both optional - and the intent statement is written at 1.1, before discovery ran; none of the four discovery stages writes back to it. Whether delivery ever reads the pack is left to the conductor LLM's initiative, nudged only by a skip-reason string. The upstream-coverage sensor cannot catch this because it only verifies declared consumes. The hand-off path is fine (the receiving run ingests the pack as supplied material at its own materials-first 1.1); it is the flagship continue-here path that leaks.

The fix is small - see the inline comment on discovery-decision.md Step 8 for two concrete options. Related: stage-protocol-recovery.md:60 loads resume context for ideation 1.1-1.7 only, so a run resumed inside or after discovery never reloads the pack either.

2. Blockers (mechanical)

  • Three deterministic test failures introduced by this PR. t189-compose-dispatch.sdk, t192-compose-front-journey.sdk, and t193-compose-report-journey.sdk pin the stock scope count at 9 (t189:103,108; t192:89; t193:68,96,100; plus t192:73 STOCK_SCOPES lacking discovery). All three fail with expected 9, received 10 at the PR head and pass at the merge base (A/B verified). The two e2e compose-front twins were updated for 9 -> 10; these three integration tests were missed. The PR body's "one known environmental failure (t92)" only holds in an environment where the sdk-live tests skip.
  • Version 2.2.2 collides. v2 already merged a different 2.2.2 and is currently at 2.2.10, so the PR needs a rebase and re-bump (GitHub already reports it conflicting on CHANGELOG.md, README.md, the five aidlc-version.ts copies, and tests/.coverage-registry.json). See the inline comment on aidlc-version.ts for a trap: the literal version string is baked into six shipped prose sites that t68 does not guard.
  • Test number t199 is taken. v2 already has tests/unit/t199-learnings-memory-path.test.ts; this PR adds tests/integration/t199-discovery-scope.test.ts. run-tests.sh --filter matches by filename regex, so t199 becomes ambiguous. Please renumber to a free slot on rebase.

3. Majors (inline comments carry the detail)

  • stage-protocol.md Part 4 now contradicts itself: the enterprise/feature template says /32 while its own example says /36.
  • The commit-continuation verb sequence has zero test coverage and depends on the absence of a status guard a sibling verb already has.
  • discovery-decision.md instructs direct state-tool calls that the orchestrator SKILL.md forbids; no other stage body does this.
  • PR body accuracy: the parity fixtures actually show four added SKIP rows per stock scope (zero removals), not "exactly one removed SKIP row" - the real property is stronger than the claim, but the sentence is wrong. And "with nothing supplied, the clarifying questions run exactly as before" oversells: a stock run always gains one intake question, two always-produced records, and a new confirm-unconfirmed-answers duty at the gate (the CHANGELOG states this correctly).

4. Minors

  • discovery-experimentation.md pre-decides the protocol's artifact re-use question (inline).
  • discovery-current-state.md says "four parts"; the template has five H2s (inline).
  • Two intent-capture ambiguities an executing agent can trip on (inline).
  • Stale refs the sweep missed: docs/guide/glossary.md:51 ("Ideation 7"), docs/guide/workshop-mode.md:59 ("1.1-1.7"), docs/guide/14-artifacts-reference.md:148 (heading says 1.1-1.7 above a table containing 1.8-1.11), core/aidlc-common/protocols/stage-protocol-recovery.md:60 (ships in dists).
  • t199's produces pin uses toContain rather than an exact match (inline).

What holds up well

  • All 9 stock parity fixtures change by exactly four added SKIP rows with every other row byte-identical; walk fixtures untouched; no existing stage number shifts.
  • The artifact guard passes on any one declared produce, so the added produces cannot create a hard-failure path; sensors stay advisory.
  • The sensor/graph tool diffs are comment-only as claimed (verified byte-level); validateScope("discovery") returns clean with zero advisories.
  • Every engine verb the stages name exists with the exact spelling and flags; keywords: [] makes the scope genuinely unreachable by inference.
  • The ~18 count-pin test edits strengthen rather than weaken (t30 gains an exact discovery EXECUTE pin; t66's enterprise check became strictly stronger); the coverage ratchet only went up.
  • bun scripts/package.ts --check is green on all four harnesses at the PR head, and settings.json changes announcement text only.

Verification

Full deterministic suite (smoke+unit+integration, 229 files) run at the PR head: 6 red. t121/t145/t165 pass green-alone (parallelism flakes); t189/t192/t193 stay red alone and pass at the untouched merge base, so those three are introduced here. Packaging parity green on all four harnesses.

4. Hand control back to the forwarding loop (`next`). Delivery begins at the
stage the engine named, with the intent statement, stakeholder map, and
the whole discovery record already in place — the delivery path's entry
stages consume the intent statement this workflow produced at 1.1.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the seam where the flow's central promise leaks. On the continue-here path the vision travels only as prose: no Inception stage consumes decision-pack, evidence-record, assumptions-record, or future-state (requirements-analysis declares only intent-statement and scope-document, both optional), and the intent statement being relied on here was written at 1.1, before any discovery ran - none of the four discovery stages writes back to it. So if the evidence narrowed or reframed the idea, delivery's entry stage is wired to read the day-one intent, and reaching the decision pack depends on the conductor's initiative. upstream-coverage cannot catch it because it only verifies declared consumes.

Two small fixes; either works, both is better:

  • On commit, fold the chosen framing and recorded exclusions back into the intent statement before the scope-change, so the one artifact that does flow is fresh.
  • Declare decision-pack as an optional consume on requirements-analysis (and ideally user-stories). Optional consumes are first-class since 2.2.9, and upstream-coverage would then verify Inception actually referenced the vision.

The hand-off path does not have this problem: the receiving run ingests the pack as supplied material at its own materials-first 1.1.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both options implemented, plus the same wiring one stage further where the leak repeated for visual content: optional decision-pack consumes on requirements-analysis and user-stories, the "What Discovery Validated" write-back before the scope-change, design-language-record and evidence-record consumes on refined-mockups, and the recovery protocol now spans 1.1-1.11. t214 pins the pack-present/pack-absent differential.

**For enterprise and feature scopes** (all 32 stages active):
**For enterprise and feature scopes** (the graph holds 36 stages; enterprise and feature execute all 32 delivery-path stages — every stage except the 4 discovery-only ideation stages):
```
Progress: [N]/32 overall | [phase-N]/[phase-total] [Phase] stages complete. Next: [Next Stage Name]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This template still says /32 while the enterprise example at line 210 below was updated to "13/36 overall", and the all-other-scopes template uses /36. The base was consistent (/32 everywhere); this PR introduces the split, and a conductor following the prose verbatim will render one or the other nondeterministically. Either this line should be /36 (matching the CHANGELOG's own claim that denominators move to N/36), or the example needs to change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: the template line reads /36, and the numerator rule is now disambiguated (plan SKIP rows count in neither the numerator nor the total, runtime [S] skips count in both). The old wording left it computable two ways.

Comment thread core/tools/aidlc-version.ts Outdated
// Bumped in the same commit that adds the matching ## [N.N.N] heading
// to CHANGELOG.md. Pinned by tests/unit/t68-version-changelog-sync.test.ts.
export const AIDLC_VERSION = "2.2.1";
export const AIDLC_VERSION = "2.2.2";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.2.2 is already taken: v2 merged a different ## [2.2.2] and is now at 2.2.10, so this needs a rebase and re-bump (t68 will fail on the duplicate CHANGELOG heading otherwise).

One trap to catch during the re-bump: the literal string "as of 2.2.2" is baked into shipped prose that t68 does not guard - aidlc-sensor-required-sections.ts:47 and :163, aidlc-sensor.ts:404, aidlc-graph.ts:271, stage-protocol.md:745, and templates/.gitkeep:10, plus their dist mirrors. Recommend versionless wording ("the framework ships nine discovery defaults") so future re-bumps cannot strand them.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto 242953e, now 2.2.11. The six literal-version prose sites are versionless now, so future re-bumps cannot strand them.

- **Commit** — append the handoff contract as a machine-readable section of the decision pack: every answered question with status, confidence, and source; every supported assumption as a requirement with its evidence attached; every refuted assumption as a recorded exclusion with the evidence as the reason; every still-unknown verdict exported as a low-confidence assumption to re-raise; every test artifact marked promote registered as a starting point. Then ask one follow-up structured question (same §3 machinery): **where does the build happen?**
- **Continue here** — the build happens in this workspace. The same workflow and the same record continue into delivery: proceed to Step 6 and the exit gate, then follow Step 8.
- **Hand off** — the build happens somewhere else (another repository, another team, a workspace that holds the code). Proceed to Step 6 and the exit gate; approval completes this workflow, and the decision pack's handoff contract is what the receiving team's workflow consumes.
- **Pivot** — everything learned is kept, and the initiative returns to the framing. Do NOT proceed to the exit gate and never mark this stage completed: after the §13 learnings ritual, relay the engine's backward jump — `bun {{HARNESS_DIR}}/tools/aidlc-jump.ts execute --target discovery-future-state --direction backward`. The jump re-opens discovery-future-state and the stages after it so their artifacts can be revised; nothing is deleted, and the evidence record is append-only — it is never rewound.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The orchestrator SKILL.md says never call the state tools directly (the engine's report dispatches them), but this stage instructs exactly that, here and in Step 8 - and no other stage body invokes aidlc-jump.ts or aidlc-state.ts directly. Mechanically the verbs exist and audit correctly (I verified each), so this is a contract inconsistency rather than a dead end, but a conductor obeying SKILL.md literally may refuse the move. One side needs an explicit carve-out.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carve-out stated on both sides: the SKILL rule (all four harness copies) names the stage-body relay exception, and the stage names the rule it excepts. Auditing that sentence surfaced a false claim of ours: set-status emits no audit event (the other four verbs do). Both surfaces now state the distinction.

1. Ask which delivery scope fits the committed build (a structured question:
`feature` is the default, `mvp` and `enterprise` are the usual
alternatives, or a composed scope via `/aidlc compose`), then relay
`bun {{HARNESS_DIR}}/tools/aidlc-utility.ts scope-change --scope <chosen>`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sequence has no test coverage, deterministic or live, and it works only because scope-change and next happen to lack the Completed-status guard that recompose already has (aidlc-utility.ts:3060 refuses non-Running state with a rationale that reads on exactly this situation). If that guard is ever extended to scope-change, this documented path breaks silently. Since this is the scope's advertised graduation path into delivery, one test pinning the sequence would make it load-bearing rather than accidental.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t214 now drives the sequence against a completed-discovery fixture, asserts each half of the no-guard behavior separately, and includes a contrast test that recompose refuses the same state. Extending the guard to scope-change now fails a test instead of silently breaking Step 8.

On re-entry (a new session, a resumed workflow), this stage re-runs from its
top per the standard stage ritual — but substep 1 reads what earlier cycles
already recorded, so work continues from the record rather than starting over.
At the artifact re-use question, the answer for `test-plans.md` and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pre-decides the person's answer to the protocol's artifact re-use question - stage-protocol defines it as a 3-option structured question, and Redo overwrites files. An executing agent will either skip the question (protocol violation) or ask it and risk a Redo that rewinds the append-only evidence record. Suggest rewording as "recommend Modify, and warn that Redo would rewind the append-only evidence record", or adding an explicit protocol carve-out for this stage.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded as suggested: recommend Modify, and warn that Redo rewinds the append-only evidence record.


### Step 4: Capture the Design Language Record

Create or extend `<record>/ideation/discovery-current-state/design-language-record.md` with four parts:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Four parts", but the shipped design-language-record template carries five H2s (## Gaps is the fifth), and the required-sections sensor checks output against the template's full H2 set. An agent following this prose alone writes four sections and collects an advisory SENSOR_FAILED. Say five, or name Gaps here.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: the prose lists five parts with Gaps as its own heading, matching the template the sensor enforces.

selected items through follow-up questions written into the same questions
file, each with a blank [Answer]: tag, before ending any turn that waits on
the person. Record declines as declines; never nag. When E is the answer,
skip ahead — the questionnaire path is unchanged.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Skip ahead" reads like "jump to Step 5", but Step 4 still requires creating source-inventory.md recording "None". An agent taking the skip literally never writes the file - no hard failure (the artifact guard passes on any one declared produce), but the "an empty inventory is itself a fact the record keeps" intent silently fails. Worth a clause: "skip ahead after recording the empty inventory in Step 4".

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now reads: skip ahead after recording the empty inventory in Step 4.

Step 3, verbatim. Several asks can feed one initiative; an ask that turns
out to contain two problems is split into two rows and noted.

When either exists, add:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"When either exists" - the intended antecedent (a mandate or a decide-by date) is 70+ lines up in the intake options; the nearest conditional is "When the person brought asks" a few lines above, so a conductor can read "either" as "asks or materials" and emit empty Already Decided / Decide-By sections. Naming the two things explicitly removes the misread.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now names the referents: a mandate or a decide-by date.

expect(node?.scopes).toEqual(["enterprise", "feature", "mvp", "poc", "discovery"]);
// The two produces the extension added — every scope that runs the stage
// gets them; downstream discovery stages consume them by these names.
expect(node?.produces).toContain("source-inventory");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toContain lets a sixth, unintended produce pass silently; an exact toEqual on the ordered array would pin the append-only contract fully. The rest of this file is genuinely falsifiable - this is the one soft spot. (Separately: the t199 number is already taken on v2 by tests/unit/t199-learnings-memory-path.test.ts, and --filter matches by filename regex, so please renumber on rebase.)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renumbered to t213 (t203, the first choice, was also taken upstream in the interim). The produces pin is an exact ordered toEqual, so a sixth produce cannot pass silently.

…ery scope — ideation as an idea-to-decision flow (2.2.11)
…ner cells, plain-language gaps, prose pins (2.2.11)
…late heading names, sensor-form provenance, doc mirrors, and the design-language wiring into refined-mockups (2.2.11)
…ive-part record honest, who-runs-the-tests, hand-off transport truth, appetite elicitation, glossary gaps (2.2.11)
@akash-doshi akash-doshi force-pushed the pr1-discovery-scope branch from 94a105a to 79aa58c Compare July 8, 2026 23:34
@akash-doshi akash-doshi changed the title feat: bring product discovery into Ideation — extend intent-capture materials-first, add the opt-in discovery scope (2.2.2) feat: bring product discovery into Ideation — an idea-to-decision flow that feeds Inception (2.2.11) Jul 8, 2026
@akash-doshi

akash-doshi commented Jul 8, 2026

Copy link
Copy Markdown
Author

Cheers Arden, all items from the review are addressed on the updated branch (rebased onto v2.2.10, version 2.2.11). What changed and why, in the review's order:

1. Wiring the vision into Inception (the design ask)

The gap was real and ran wider than the continue-here path: no Inception stage declared any discovery output as an input, so validated content reached delivery only through conductor initiative. Changes:

  • requirements-analysis and user-stories declare decision-pack as an optional consume (required: false). When the pack exists, the run-stage directive carries its path and upstream-coverage verifies the output referenced it. When it does not, the resolver drops the entry silently, so stock scopes are unaffected — parity fixtures unchanged.
  • At commit, the decision stage folds a "What Discovery Validated" section into intent-statement.md (chosen framing, recorded exclusions, provenance in file-name form). Rationale: that file is the input requirements-analysis already reads, so after a commit it must reflect the validated direction, not the pre-discovery intent.
  • The same leak existed for visual content one stage further. refined-mockups consumed nothing discovery produces, and its documented fallback designs from requirements and stories, which carry no visual content — so a validated design language and people-tested mock-up variants died at the phase boundary. It now declares design-language-record and evidence-record as optional consumes, with prose to design within the recorded language and treat promote-marked variants as starting points.
  • The handoff contract is restructured to instruct its consumer rather than describe content: a consumption note (settled entries are pre-answers to confirm, never re-elicit; open items are questions to raise), a uniform entry shape (value, status, confidence, confirming gate, source), exclusions with evidence, open items with recommendations, promoted starting points, and a closing map from delivery's first questions to where the record answers them.
  • stage-protocol-recovery.md now spans ideation 1.1–1.11 and names the pack as load-bearing resume context, closing the resume half of the same gap.
  • t214 pins the consume differential in both directions: pack present, the requirements-analysis directive carries its path; pack absent, no trace of it in the directive.

2. Blockers

  • t189/t192/t193: scope-count pins updated (9→10 for stock counts, 10→11 in the composed-on-top journeys, discovery added to STOCK_SCOPES). These tests skip without live SDK credentials, which is why local runs reported green here — the fix is arithmetic-correct rather than run-proven in this environment, and the PR body's verification statement now states its environment scope explicitly.
  • Version: rebased onto 242953e, re-bumped to 2.2.11. The six shipped prose sites carrying a literal version string are now versionless ("the framework ships nine defaults"), so future re-bumps cannot strand them — the collision was routine, the baked-in strings were the recurring cost.
  • Test number: renamed to t213. t203, the first rename, was also taken upstream in the interim.

3. Majors

  • The /32 vs /36 template split was fixed in the branch's second commit, which predates the review. The numerator rule is now also disambiguated (plan SKIP rows count in neither the numerator nor the total; runtime [S] skips count in both) — the old wording let a careful reader compute contradictory numbers from the protocol and the guide examples.
  • The continuation sequence is a pinned behavioral contract, not an accident of a missing guard: t214 drives scope-change → next → set-status → skip → next against a completed-discovery fixture, asserts each half of the no-guard behavior separately, and includes a contrast test proving recompose refuses the identical state. If a Completed-status guard is ever extended to scope-change, the test fails and Step 8 must be redesigned with it.
  • The verb-relay carve-out is stated on both sides: the orchestrator SKILL rule (all four harness copies) names the stage-body relay exception, and the stage names the rule it excepts. Auditing that sentence surfaced a false claim of ours: set-status emits no audit event (jump, park, scope-change, and skip each do). Both surfaces now state the distinction precisely.
  • PR body accuracy: the fixture sentence now states the actual diff (each stock scope's plan gains exactly four SKIP rows, nothing removed), and "exactly as before" is replaced with the CHANGELOG's wording — one intake question, two additive records, a confirm-unconfirmed duty at the gate.

4. Minors

All applied: the artifact re-use guidance recommends Modify and warns that Redo would rewind the append-only evidence record (a stage file cannot pre-decide a protocol question), the two intent-capture ambiguities are resolved by naming their referents (record the empty inventory in Step 4 before skipping ahead; a mandate or a decide-by date), workshop-mode reads 1.1–1.11, and the t213 produces pin is an exact ordered list so a sixth produce cannot pass silently.

Corrections and disclosure

  • Two details in the review, for the record: required: false consumes predate 2.2.9 (2.2.9 added optional_produces on the produce side), and two of the listed stale refs (glossary phase count, artifacts-reference heading) were already fixed at the local head the stale PR could not show. Neither changes an action taken.
  • One previously undisclosed change, now disclosed in the CHANGELOG: the welcome banner's scope table had two pre-existing cells disagreeing with the compiled grid (infra INCEPTION reads PD, RA; security-patch RE reads RE, RA). Both corrected.

Other Things

  • Right now, we have two flows in the AI-DLC Ideation stage: 1/ the default workflow and 2/ this discovery phase that embeds stronger design thinking patterns. Am aware this is introducing more stuff in already a step-heavy workflow, and would like to first validate the capability of this workflow before making this new discovery phase the default ideation path.

@apackeer

Copy link
Copy Markdown

@raj-jain-aws @SiddhJog @spraja08 @leandrodamascena Can you guys take a look at this? I think directionally (and mechanically) it fits in with the model of Ideation produces an idea, vision, Inception produces a design or plan, Construction produces code etc.

@leandrodamascena leandrodamascena marked this pull request as ready for review July 11, 2026 14:06
@leandrodamascena

Copy link
Copy Markdown
Contributor

I ran a max-effort pass over this — nine angles, each finding reproduced or refuted against the PR tree, focused on what the four fix commits might have introduced. Short version: the code is healthy, all of Arden's findings landed, and what's left is purely rebase.

Arden's findings — all fixed, I checked each:

  • Vision reaches Inception: discovery-decision Step 8 now writes the ## What Discovery Validated section back into intent-statement.md, and requirements-analysis/user-stories declare decision-pack as an optional consume. I traced the chain and it holds — the record references the write-back emits match the slugs the upstream-coverage sensor greps for.
  • The three test failures (t189/t192/t193): now pin 10 and include discovery in STOCK_SCOPES. Strengthened, not loosened.
  • /32 vs /36: no /32 remnant, and the progress arithmetic (a finished feature run reads 32/36) is right.
  • t199 collision: renamed to t213/t214.
  • The stale refs (glossary, workshop-mode, artifacts-reference, stage-protocol-recovery): swept. Recovery now loads discovery context 1.8–1.11.

New bugs: I could not find one. I ran the sensor greps against the templates, traced the commit/pivot/park continuation, diffed the discovery nodes against a core node, and checked the empty-keywords path (discovery is genuinely unreachable by inference). The stock scopes are byte-safe — the only fixture change is the four added SKIP rows, everything else identical.

What actually blocks the merge — both rebase, neither is your code:

  1. Version 2.2.11 collides (t68 will fail). v2 has moved to 2.3.0 and already carries a ## [2.2.11] - 2026-07-08 heading (the feat: preview stage and gate counts at scope confirmation (2.2.12) #508/fix: backfill a skipped gate rejection at approve time (2.2.11) #510 backstop, a different feature). This PR adds a second ## [2.2.11]. t68 asserts headings are unique, so two of them fail deterministically. This is the conflict-trap AGENTS.md documents — rebase and re-bump to the next free 2.3.x, and resolve the duplicate CHANGELOG heading.

  2. bun scripts/package.ts --check fails after the rebase, but not because of this PR. v2 itself is currently broken: the feat: AIDLC plugin mechanism (2.3.0) #475 squash-merge dropped the produces_kinds parse call in aidlc-lib.ts and its OPTIONAL_FIELDS entry (aidlc-stage-schema.ts:152), while four construction stages still carry produces_kinds frontmatter — so --check fails on clean v2 today. This PR's head passes its own --check (it predates feat: prune per-unit design artifacts by unit kind (2.2.18) #511, no produces_kinds anywhere), but once you rebase onto v2 the dist regen inherits the broken tree. I'm filing this separately since it blocks every PR, not just yours — please don't treat it as a feat: bring product discovery into Ideation — an idea-to-decision flow that feeds Inception (2.2.11) #526 defect.

Non-blocking:

  • The PR body still says "exactly one removed SKIP row" (fixtures show four added, zero removed) and "clarifying questions run exactly as before" (a stock run gains an intake question, two records, and a gate duty). The CHANGELOG states it correctly — just fix the body.
  • The version string is baked into a few shipped prose sites t68 does not guard; watch those on the re-bump.

Net: the discovery flow itself is in good shape. Rebase onto v2, re-bump off 2.2.11, and the --check will pass once the v2 produces_kinds regression is fixed (separate issue). I would approve once the rebase is done.

@RachnaC1234

RachnaC1234 commented Jul 15, 2026

Copy link
Copy Markdown

I had opened same request for AIDLC 1.0 that was rejected earlier. I am curious to learn the drivers of the new phase in AIDLC 2.0. What are the key goals and what is driving including it as that determines the correctness and scope of the solution. My solution I proposed for AIDLC has landed eventually here as open source solution https://github.com/aws-samples/sample-ai-plc
Few learnings I had from running this as pre-req to AIDLC for workshops -
1/ Your personas are not technical usually for discovery phase. So simplicity of the design is the key to success, something that is understandable and not overwhelming.
2/ What works for one customer, does not work for other, can non technical personas extend what your provide or customize with Natural language ?
3/ Output creation from the phase should be standardized, though the output can come from any implementations of Ideation phase done using various tools. ( Including Amazon Quick )
4/ Ideation brings together a new methodology and personas in the mix, could be more than PMs .. think of strategists, CxO. This need to be accounted in the methodology and collab phase and how we the flow will work with the users.
5/ Idea needs to be proven with prototypes, prototypes should be done simply, with least friction from technology so ideas can be validated before inception happens and user stories are written.
6/ while it may first feel like that greenfield needs ideation, fact is most of the enterprises are not starting from scratch, they have existing products that they are extending, so the soln has to accomodate brownfield to be useful for enterprises.

@akash-doshi

Copy link
Copy Markdown
Author

Hi @RachnaC1234 thanks for your comment, please take some time to review this PR as keen to see your feedback on this approach - as it addresses most of the points you've described. You make great points around greenfield/brownfield and rapid prototyping to validate a decision, but also how stakeholders may differ in technical capability. It opens up a question where the start point and end point of AI-DLC workflows should be.

While this PR lands the technical parts components of introducing a product discovery scope within ideation, a few areas need to be evolved:

  • More elaborated playbooks for design system harvesting and prototyping
  • If we recognise that teams will have their own workflows outside of AI-DLC, more relevant to their stakeholders (eg: Product Ideation with Amazon Quick) - a handover contract between other frameworks to AI-DLC is required. So far, we've based this design off the work of @clairmit https://github.com/subcontext/agentic-product-discovery. We can extend this contract beyond just discovery --> inception and ensure external tools can leverage this same contract.

To help steer in the right direction, let's frame these feedback and changes into RFCs to get feedback on the mental model and bring your expertise into the AI-DLC Ideation Stage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants