[Spec] External Results API v1 — Pydantic schemas, spec doc, tests (BGSTM#299)

[Copilot]

Drafts the v1 contract for the External Results API — the ingestion surface external test runners (Playwright first) use to report execution into BGSTM. Spec-only: no router, no DB migration, no auth implementation, no main.py edits.

Schemas (backend/app/schemas/external_results.py)

Pydantic v2 types-only module covering the full lifecycle:

• Enums: RunStatus · CaseOutcome · ArtifactKind
• Session: SessionCreate / SessionResponse / SessionFinish
• Case result: CaseResultCreate / CaseResultResponse / CaseResultUpdate
• Artifact: ArtifactCreate / ArtifactResponse
• Error envelope: ErrorResponse

Two validators enforced at schema level:

# SessionFinish: only terminal statuses accepted
SessionFinish(status=RunStatus.started)  # → ValueError

# CaseResultCreate: at least one identity field required
CaseResultCreate(session_id=..., title=..., outcome=..., duration_ms=50)  # → ValueError
# ok: external_id only, test_case_id only, or both

CaseResultResponse uses composition (not inheritance from CaseResultCreate) to avoid exposing request validators in response models.

Spec (docs/specs/external_results_v1.md)

Eight sections: overview + versioning model, auth (bearer token shape, scopes, error codes — forward-refs #296), all 7 endpoints with request/response JSON examples and error tables, idempotency rules (dedup by external_id, SHA-256 for artifacts), session + outcome transition tables, unified error code registry, audit-log action taxonomy (forward-ref #297), and reference implementation pointer (bgstm-playwright-frameworks#3, #295).

Nav + README

• mkdocs.yml: added Specs → External Results v1 nav entry
• README.md: added "📐 API Contracts & Specs" table near "Related projects"

Tests (backend/tests/test_external_results_schemas.py)

27 tests — enum value coverage, both validators (happy + failure paths), duration_ms bounds, and full model_dump_json → model_validate_json round-trips for every response model. No new dev dependencies.

Original prompt

Goal

Draft the v1 contract for the External Results API — the surface that external test runners (Playwright first; Pytest/Cypress later) use to report execution into BGSTM. Spec only — no router wiring, no business logic, no DB migration.

This implements BGSTM#299 and unblocks parallel work on:

• BGSTM#296 — runner-token auth
• BGSTM#300 — router + session endpoints
• BGSTM#303 — case results
• BGSTM#298 — artifact upload
• BGSTM#297 — audit-log integration
• BGSTM#295 — TypeScript reference reporter
• bgstm-playwright-frameworks#3

Parent epic: BGSTM#291.

Repository conventions to match

• Backend is FastAPI + Pydantic v2 + SQLAlchemy (async). Existing routers live in backend/app/api/. Existing schemas live in backend/app/schemas/. Inspect a couple of these to match style:
  • backend/app/api/auth.py
  • backend/app/api/test_cases.py
  • backend/app/api/traceability.py
  • Any file under backend/app/schemas/ for Pydantic conventions
• Docs use mkdocs (verify by checking mkdocs.yml at repo root or in docs/). Match existing doc page style and frontmatter.

Tasks

1. backend/app/schemas/external_results.py — Pydantic models

Create the module with types only — no FastAPI imports, no DB. The models must be importable and pass pytest / mypy / ruff per the repo's existing config.

Required models (use Pydantic v2 syntax — model_config, Field(...), etc., matching the repo's existing style):

• RunStatus — Enum: started, passed, failed, skipped, aborted
• CaseOutcome — Enum: passed, failed, skipped, flaky
• ArtifactKind — Enum: screenshot, trace, video, log, other
• SessionCreate — payload for POST /external-results/session
  • Fields: runner: str, project_id: UUID, git_sha: str | None, git_branch: str | None, ci_url: HttpUrl | None, metadata: dict[str, Any]
  • Add a model_config with a JSON Schema example
• SessionResponse — id, status, started_at, finished_at, plus echoed metadata fields
• SessionFinish — status: RunStatus (only passed/failed/aborted allowed; enforce with a validator), summary: dict[str, Any]
• CaseResultCreate:
  • session_id: UUID
  • test_case_id: UUID | None
  • external_id: str | None
  • title: str
  • outcome: CaseOutcome
  • duration_ms: int = Field(ge=0)
  • error_message: str | None
  • requirement_ids: list[UUID]
  • Validator: at least one of test_case_id or external_id must be set, else raise ValueError
• CaseResultResponse — extends CaseResultCreate with id, created_at, auto_registered: bool
• CaseResultUpdate — partial: outcome, duration_ms, error_message all optional
• ArtifactCreate — case_result_id: UUID, kind: ArtifactKind, filename: str, content_type: str, size_bytes: int = Field(ge=0)
• ArtifactResponse — adds id, url: HttpUrl, created_at
• ErrorResponse — code: str, message: str, details: dict[str, Any] | None

Add a top-of-file docstring linking to the spec doc and to BGSTM#291.

2. docs/specs/external_results_v1.md — Canonical spec

Create the canonical human-readable spec. Use whichever heading style and frontmatter the existing docs use. Required sections:

a. Overview

• Purpose, audience (test framework authors), versioning model (/api/v1/..., no breaking changes within v1)
• Reference implementation pointer (bgstm-playwright-frameworks#3)

b. Authentication

• Bearer token shape: Authorization: Bearer bgstm_runner_<opaque>
• Forward-reference BGSTM#296 for issuance/revocation flow
• Required scopes per endpoint
• Note: tokens are scoped, revocable, attributable; never log them

c. Endpoints

For each endpoint, document: method, path, auth scope, request body, response body, all error codes (400/401/403/404/409/413/415/422/500), idempotency behavior, and a JSON example of request + success response.

Endpoints:

1. POST /api/v1/external-results/session (scope external_results:write)
2. PATCH /api/v1/external-results/session/{session_id}
3. `GET /api/v1/external-results/session/{sessi...

This pull request was created from Copilot chat.