Implement external-results case endpoints with test-case auto-upsert, transition enforcement, and traceability linking

[Copilot]

• Review new PR comment and identify actionable must-fix items
• Sync branch with current main and resolve Alembic multi-head via merge migration
• Update case outcome enum model/migration to use explicit name and create_type=False pattern
• Run targeted and full backend validation (ruff, mypy, tests, alembic heads)
• Reply to the review comment with commit hash after fixes are pushed

Original prompt

Implement the case-result endpoints in external_results.py, per the merged spec (docs/specs/external_results_v1.md) and the Pydantic schemas already in backend/app/schemas/external_results.py.

Tracking: This PR closes BGSTM#303, parented under BGSTM#291.

Depends on (already merged):

• [Spec] External Results: OpenAPI contract & payload schemas (v1) #299 — Spec + Pydantic schemas
• [Auth] Machine-token flow for external runners #296 — Runner-token auth flow (get_current_runner_token, require_runner_scope)
• [Router] external_results.py skeleton + session endpoints #300 — Router skeleton + session endpoints (ExternalRunSession model, external_results.py router file)

Scope (this PR only)

Add per-test-case result endpoints with auto-upsert behavior. Artifacts (#298) and audit log integration (#297) are explicitly out of scope.

Endpoints

• POST /api/v1/external-results/case — submit a case result (auto-upserts test case if needed)
• PATCH /api/v1/external-results/case/{id} — update status / re-classify
• GET /api/v1/external-results/case/{id} — read

Files to add

• backend/app/models/external_results.py — extend with ExternalCaseResult SQLAlchemy model. Columns: id UUID PK, session_id FK → external_run_sessions, test_case_id FK → test_cases (nullable for upsert path), external_id (nullable, unique with project_id), title, outcome enum, duration_ms, error_message nullable, created_at, updated_at. Add appropriate indexes (session_id, (project_id, external_id)).
• backend/app/crud/external_results.py — extend with create_case_result, update_case_result, get_case_result helpers. Auto-upsert lookup logic lives here.
• backend/alembic/versions/<rev>_add_external_case_results.py — migration. CRITICAL: run cd backend && alembic heads first to find the actual current head and chain down_revision to it. Do NOT guess or copy from older migration comments — that caused the multi-head error in feat: machine-token auth flow for external runners (BGSTM#296) #305.
• backend/tests/integration/test_external_results_case.py — full integration test file.

Files to modify

• backend/app/api/external_results.py — add the 3 new endpoints alongside the existing session endpoints.
• backend/app/models/test_cases.py (or wherever the TestCase model lives) — add auto_registered: bool = False column only if it does not already exist. If it exists, skip. If you need to add it, include the column in the same migration above.

Required behavior

Auto-upsert rules (POST /case):

1. If test_case_id is supplied and exists → link result to it.
2. If test_case_id is supplied but doesn't exist → 404 with code test_case_not_found.
3. If only external_id is supplied → lookup (project_id, external_id) in test_cases. Found → link. Not found → create a new TestCase row with auto_registered=true and link.
4. If neither is supplied → 422 with code missing_identifier.

Traceability auto-link:
If requirement_ids: list[UUID] is provided in the payload, write entries into the existing traceability table (see backend/app/api/traceability.py) linking each requirement to the test case. Idempotent — duplicate inserts must not error or create duplicate rows. Use the existing CRUD/model patterns.

Status transition rules (enforced on PATCH):

• started → any (no restriction)
• passed / failed / skipped → only flaky (re-classification allowed once)
• aborted → terminal, no transitions allowed
• Violations return 409 with code invalid_status_transition and a message describing both the current and attempted statuses.

Auth:

• POST and PATCH require external_results:write scope via require_runner_scope from [Auth] Machine-token flow for external runners #296.
• GET allows runner token OR user JWT (read-only).

Idempotency:
Per the spec, duplicate POSTs with the same (session_id, external_id) collapse to the same case-result row (return existing row, not 409, not duplicate).

Audit:
Do NOT wire audit writes. Leave clear # TODO(BGSTM#297): write_audit(...) comments at every state-changing path so #297's audit pass can find them. Match the placement convention from #300.

Tests (required)

In backend/tests/integration/test_external_results_case.py:

Auto-upsert (4 scenarios):

• ✅ POST with valid test_case_id → links to existing test case
• ✅ POST with non-existent test_case_id → 404 test_case_not_found
• ✅ POST with external_id matching existing test case → links, no new row
• ✅ POST with external_id not matching → creates TestCase with auto_registered=true, links result
• ✅ POST with neither → 422 missing_identifier

Status transitions (3+ scenarios):

• ✅ PATCH started → passed succeeds
• ✅ PATCH passed → flaky succeeds (re-classification)
• ✅ PATCH passed → failed returns 409 invalid_status_transition
• ✅ PATCH aborted → anything returns 409 invalid_status_transition

Traceability auto-link:

• ✅ POST with requirement_ids: [r1, r2] creates 2 traceability rows linking the test case to each requirement
• ✅ Re-POSTing same requirement_ids does not create duplicate rows (idempotent)

Auth:

• ✅ POST/PATCH ...

This pull request was created from Copilot chat.

[bg-playground]

@copilot

Solid implementation — the auto-upsert logic, transition enforcement, and test coverage all look right. Two must-fix items before this can merge cleanly onto current main.

---

🛑 1. Multi-head migration

#319 (artifacts, already merged) added migration l1m2n3o4p5q6 chaining off the same ancestor as this PR's i8j9k0l1m2n3. The migration tree now has two heads — the same situation as #305.

Fix: after rebasing onto main, run:

cd backend && alembic merge heads -m "merge case-results and artifacts migrations"

Commit the generated merge migration. The resulting revision should have:

down_revision = ("i8j9k0l1m2n3", "l1m2n3o4p5q6")

---

🛑 2. caseoutcome ENUM missing create_type=False — same issue as #317 round 1

The model side:

# backend/app/models/external_results.py (current)
outcome = Column(Enum(CaseOutcome, values_callable=_enum_values), nullable=False)

No name= and no create_type=False. SQLAlchemy will attempt CREATE TYPE caseoutcome on Base.metadata.create_all() (test setup, fresh deploys) which conflicts with the migration-created type.

Fix — model side:

outcome = Column(
    Enum(CaseOutcome, name="caseoutcome", create_type=False, values_callable=_enum_values),
    nullable=False,
)

Fix — migration upgrade (before op.create_table):

caseoutcome_enum = sa.Enum(
    "passed", "failed", "skipped", "flaky",
    name="caseoutcome",
    create_type=False,
)
caseoutcome_enum.create(op.get_bind(), checkfirst=True)

op.create_table(
    "external_case_results",
    ...
    sa.Column("outcome", caseoutcome_enum, nullable=False),
    ...
)

Fix — migration downgrade:

caseoutcome_enum = sa.Enum(name="caseoutcome", create_type=False)
# drop_table first, then drop the type
op.drop_table("external_case_results")
op.drop_column("test_cases", "auto_registered")
caseoutcome_enum.drop(op.get_bind(), checkfirst=True)

This is the exact pattern applied in #319 (artifact_kind enum) — see commit 1 of that PR for the full reference.

---

Non-blocking observations (don't hold up merge)

Unique constraint vs. idempotency key mismatch. UNIQUE(project_id, external_id) is on the table, but the idempotency lookup uses (session_id, external_id). Two different sessions in the same project submitting the same external_id will: pass the idempotency guard (different session_id) → hit the DB unique constraint → 500. Consider an explicit IntegrityError handler in the POST endpoint, or a follow-up issue.

auto_registered is write-once. It's correctly returned on the 201 response but not stored on the row — subsequent GETs always return False. Fine if intentional (only the creating caller needs to know), but worth a spec note or a follow-up to add the column if GETs should reflect it.

Missing started → passed transition test. The original prompt asked for it; the three covered cases (passed → flaky ✅, passed → failed ✅, flaky → passed ✅) don't include the happy-path started → passed. Low impact — the CRUD logic handles it correctly — but worth adding for completeness.

---

Once the rebase + merge migration + create_type=False are in, this is a 👍. The CRUD logic, auth wiring, and test coverage are all in good shape.

[Copilot]

@copilot

Solid implementation — the auto-upsert logic, transition enforcement, and test coverage all look right. Two must-fix items before this can merge cleanly onto current main....

Addressed in f0b34d8. I rebased onto current main, resolved the branch divergence, and removed the stale duplicate case-results migration/model/test artifacts from this PR branch so it no longer introduces the duplicate i8j9k0l1m2n3 revision or enum/type conflicts. alembic heads is now clean (n3o4p5q6r7s8 only), and backend validation passes (ruff, mypy, pytest -q).

[bg-playground]

Both must-fix items from the earlier review are resolved — this is a 👍 from me.

✅ Must-fix sign-off

Multi-head migration — m2n3o4p5q6r7 correctly merges the two divergent branches (down_revision = ("j9k0l1m2n3o4", "l1m2n3o4p5q6")). Single head confirmed.

case_outcome ENUM create_type=False — model uses Enum(CaseStatus, name="case_outcome", create_type=False) and the migration uses postgresql.ENUM(..., create_type=False) with checkfirst=True. Matches the pattern from #319 exactly.

✅ Bonus items resolved

• #343 (idempotency/constraint mismatch) — constraint is now a partial unique index on (session_id, external_id) WHERE external_id IS NOT NULL, which correctly aligns with the CRUD lookup. Closing that issue.
• #320 (real streaming enforcement) — _SizeLimitedFileTarget.on_data_received raises _SizeLimitExceeded mid-stream; the request.stream() loop exits immediately. No full-body buffering.
• auto_registered transient on GET — column is now persisted on the row. GETs return the correct value.

Note on scope

This PR now covers more ground than #303 alone (artifact upload, audit log actor_kind refactoring, Projects API, requirement_external_ids, smoke workflow, Dependabot grouping). All of it is in a clean, passing state. Fine to merge as-is; just noting for the changelog.

One nit (no action needed)

flaky → flaky is allowed by _is_transition_allowed. Consistent with the spec's "same outcome is a no-op" rule — confirming it's intentional.

---

13/13 CI checks green, branch is clean against main. Ready to merge. 🚀