Description
Propose and document a Playwright-compatible BGSTM reporter API contract that enables Playwright-based automation frameworks (and any external runner) to report test execution and traceability results directly to BGSTM.
This is the foundation for a planned, separate repository of domain-specific Playwright framework packs (CRM, Accounting, Financial, etc.) that will integrate natively with BGSTM — closing the loop between test execution and test management. The contract should specify endpoints, payload schemas, authentication, and a versioning strategy so any framework (Node, Python, Java) can integrate cleanly.
Key requirements:
- Report run start/finish, test case results, optional step-level details, artifacts (screenshots, logs, videos), and traceability links.
- Allow optional auto-registration of test cases not yet present in BGSTM.
- Align to BGSTM's data model (requirement → test case → execution → evidence) and methodology phases.
- Provide an example payload set and a basic reference reporter implementation.
- Document authentication (API tokens) and surface the contract in BGSTM docs.
Relevant code references
The new router would mount alongside the existing routers in main.py:
Existing patterns the new endpoints should follow:
Suggested new file:
backend/app/api/external_results.py — new router (/external-results/session, /external-results/case, /external-results/artifact).
Use Case
Bridges the gap between test execution frameworks (especially the planned Playwright-based, domain-focused automation packs) and test management/reporting in BGSTM. It solves disconnected pipelines, enables seamless traceability from requirement → case → execution evidence, and lets any external framework add value to a BGSTM installation. Strategic value: easier integration with other platforms, lower long-term maintenance, increased BGSTM adoption, and a clean execution-reporting hand-off into NAT once hosted execution lands.
Proposed Solution
Add a new REST router (backend/app/api/external_results.py) and register it in backend/app/main.py alongside the existing routers. Endpoints (initial draft):
• POST /api/v1/external-results/session — start/finish a test run
• POST /api/v1/external-results/case — submit a case result (and optionally upsert)
• PATCH /api/v1/external-results/case/{id} — update status or attach steps
• POST /api/v1/external-results/artifact — upload screenshot/video/log
Each payload carries a BGSTM requirement_id and/or test_case_id so results land on the correct trace link (see traceability.py). Auth via bearer token (extend auth.py with a runner-token flow). Document the contract in docs/ (mkdocs) and ship a minimal TypeScript reference reporter and a Python equivalent. Keep it generic enough to work for Pytest/Cypress later. FastAPI's auto-generated OpenAPI schema becomes the formal spec.
Alternatives Considered
- (Rejected) Ad-hoc CSV or JUnit XML import: Not scalable, brittle, loses context/artifacts, not real-time.
- (Rejected) Direct DB / ORM access from external runners: Tightly couples to BGSTM internal schema and version.
- (Considered) OpenAPI/GraphQL schema first: More flexible long-term, but a small documented REST contract is a faster MVP and can be formalized into OpenAPI from FastAPI's auto-generated schema.
Related Phase(s)
Description
Propose and document a Playwright-compatible BGSTM reporter API contract that enables Playwright-based automation frameworks (and any external runner) to report test execution and traceability results directly to BGSTM.
This is the foundation for a planned, separate repository of domain-specific Playwright framework packs (CRM, Accounting, Financial, etc.) that will integrate natively with BGSTM — closing the loop between test execution and test management. The contract should specify endpoints, payload schemas, authentication, and a versioning strategy so any framework (Node, Python, Java) can integrate cleanly.
Key requirements:
Relevant code references
The new router would mount alongside the existing routers in
main.py:backend/app/main.py(router registration)Existing patterns the new endpoints should follow:
backend/app/api/test_cases.py— test case CRUD; the new contract should reference/upsert these resources.backend/app/api/traceability.py— trace links between requirements and test cases; execution results should be attachable to these links.backend/app/api/links.py— relationship modeling pattern.backend/app/api/audit_log.py— every external result write should produce an audit entry.backend/app/api/auth.py— auth pattern; extend with a token-based "machine user" flow for runners.Suggested new file:
backend/app/api/external_results.py— new router (/external-results/session,/external-results/case,/external-results/artifact).Use Case
Bridges the gap between test execution frameworks (especially the planned Playwright-based, domain-focused automation packs) and test management/reporting in BGSTM. It solves disconnected pipelines, enables seamless traceability from requirement → case → execution evidence, and lets any external framework add value to a BGSTM installation. Strategic value: easier integration with other platforms, lower long-term maintenance, increased BGSTM adoption, and a clean execution-reporting hand-off into NAT once hosted execution lands.
Proposed Solution
Add a new REST router (
backend/app/api/external_results.py) and register it inbackend/app/main.pyalongside the existing routers. Endpoints (initial draft):•
POST /api/v1/external-results/session— start/finish a test run•
POST /api/v1/external-results/case— submit a case result (and optionally upsert)•
PATCH /api/v1/external-results/case/{id}— update status or attach steps•
POST /api/v1/external-results/artifact— upload screenshot/video/logEach payload carries a BGSTM
requirement_idand/ortest_case_idso results land on the correct trace link (seetraceability.py). Auth via bearer token (extendauth.pywith a runner-token flow). Document the contract indocs/(mkdocs) and ship a minimal TypeScript reference reporter and a Python equivalent. Keep it generic enough to work for Pytest/Cypress later. FastAPI's auto-generated OpenAPI schema becomes the formal spec.Alternatives Considered
Related Phase(s)