Skip to content

[v0.2] BGSTMReporter should transmit bgstm:requirement annotations and BGSTM should resolve external→UUID for case-result linking #318

Description

@bg-playground

Goal

Close two related v0.1 gaps that the BGSTM external-results smoke workflow (PR #314) surfaced:

  1. The TypeScript reporter (bgstm-playwright-frameworks v0.1) never reads bgstm:requirement test annotations, despite the step-1 prompt asking for it. Annotations are dropped on the floor.
  2. Even if the reporter did transmit them, the merged BGSTM CaseResultCreate schema accepts requirement_ids: list[UUID] only. There is no path for the reporter to link a case result to a requirement by external ID (the natural shape from a bgstm:requirement annotation, e.g. REQ-CRM-HOMEPAGE).

Together these gaps mean the documented requirement-link flow doesn't actually work end-to-end on main today. The smoke workflow's "passed case has requirement links to REQ-CRM-HOMEPAGE" assertion was relaxed in PR #314 as a result. This issue tracks closing both halves.

Why this is v0.2, not v0.1

The v0.1 contract was deliberately frozen (#291 epic) to ship the basic write path first. Annotation-driven linking is a feature widening; better as v0.2.

Discovered during

Scope

Frameworks side (bg-playground/bgstm-playwright-frameworks)

  • BGSTMReporter.onTestEnd reads test.annotations for entries of type === 'bgstm:requirement' and collects each description as a string requirement reference.
  • Add a new optional field on CaseResultCreate payload: requirement_external_ids: string[].
  • Document the annotation contract in the reporter README: how to author test.info().annotations.push({ type: 'bgstm:requirement', description: 'REQ-CRM-HOMEPAGE' }).
  • Unit tests cover: zero, one, multiple annotations; mixed annotation types (only bgstm:requirement is captured); deduplication.

BGSTM backend

  • CaseResultCreate schema gains requirement_external_ids: list[str] = [] alongside the existing requirement_ids: list[UUID].
  • On case-result create, BGSTM resolves each external id via the existing requirements table (likely Requirement.external_id or equivalent — verify the column name on main).
  • Resolution behavior:
    • Hit → link via case_result_requirements join row.
    • Miss → record under unresolved_requirement_ids in the audit-log details (the field is already wired for UUIDs; add a sibling unresolved_requirement_external_ids for strings).
    • Do not auto-create requirement rows from external ids (that'd be auto-magic with bad failure modes).
  • Spec doc docs/specs/external_results_v1.md updated to document the new field, the resolution semantics, and the "unknown external id is non-fatal" rule.

Smoke workflow

  • Re-tighten the assertion in scripts/smoke/assert.py so the passed case is verified to link to a REQ-CRM-HOMEPAGE-equivalent requirement (real or seeded).
  • Add a seed step in the bootstrap script to ensure the REQ-CRM-HOMEPAGE requirement exists before the smoke run, OR rely on a DB-level seed in docker-compose.test.yml. Pick whichever yields the simpler diff.
  • Bump the frameworks pin to whatever SHA includes the annotation-reading reporter.

Out of scope

  • Auto-creating requirement rows from external ids.
  • Cascading delete/edit semantics across requirements ↔ case-result links.
  • Bidirectional requirement search filtered by case-result outcome (separate analytics issue if we want it).
  • Changes to artifact handling, session lifecycle, or audit-log schema.

Acceptance criteria

  • Reporter reads bgstm:requirement annotations and includes them in the case-result payload.
  • BGSTM accepts requirement_external_ids: list[str] on CaseResultCreate.
  • BGSTM resolves known external ids to UUID links; logs unknowns to audit-log details.
  • Spec doc updated for the new field and resolution rules.
  • Smoke workflow's "passed case has requirement link" assertion is restored, not relaxed.
  • Frameworks-side unit tests cover annotation parsing edge cases.
  • BGSTM-side integration test covers known/unknown external id resolution paths.

References

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions