Skip to content

[CRCR] Failed-test drill-down from RFC-0054 is absent end-to-end (failed_tests_json is never populated) #8545

Description

@atalman

What the RFC specifies

RFC-0054 — Page 2: Per-Backend Dashboard (full doc):

Failure drill-down: click a red cell to see failed test details (parsed from failed_tests_json)

The RFC's Unresolved Question #1 was closed by choosing the storage shape for this feature:

Failed test detail storage Resolved: Embedded as failed_tests_json String column in both DynamoDB and ClickHouse.

Current behaviour

The storage decision landed; the feature never did. It is missing at every stage of the pipeline, not just the UI:

  1. No way for a downstream repo to report it. .github/actions/cross-repo-ci-relay-callback/action.yml has nine inputs — status, conclusion, delivery-id, event-type, test-results, callback-url, artifact-url, max-time, job-name. test-results carries only the counts (passed/failed/skipped). There is no input for per-test failure detail.
  2. Not extracted. torchci/lib/crcr/crcrUtils.tsextractDynamoRecord() never sets failed_tests_json on the record, so the DynamoDB item never carries it and the ClickHouse column stays at its DEFAULT ''.
  3. Queried but always empty. torchci/clickhouse_queries/crcr_hud_results/query.sql:15 selects failed_tests_json — reading a column that is always the empty string.
  4. Never consumed. Neither CrcrJobRow (torchci/pages/crcr/[org]/[repo].tsx) nor CrcrPrResult (torchci/components/crcr/CrcrPrSection.tsx) declares the field, and the Page 2 query (crcr_backend_dashboard/query.sql) doesn't select it at all.

What does work is the aggregate count — the per-repo dashboard tooltip renders Tests: {passed_tests}/{total_tests} passed from the test-results input. So backends can report how many tests failed, but not which.

Why it matters

The per-backend dashboard is the self-service CI health page L2 repos are given in exchange for onboarding, and drilling into a failure is the main thing a backend maintainer wants from it. Today a red cell is terminal — the only next step is to follow the external workflow-run link into the downstream repo's own CI.

Suggested scope

  1. Add a callback-action input for failed-test detail (name/classname/message/stacktrace), consistent with the existing test-results JSON-string convention.
  2. Extract and persist it in extractDynamoRecord(), applying the caps from [CRCR] Three of four RFC-0054 payload caps are unimplemented #8548 (they were specified for exactly this field and are a prerequisite, not a follow-up).
  3. Add failed_tests_json to crcr_backend_dashboard/query.sql and to CrcrJobRow.
  4. Render the drill-down on click, per the Page 2 spec.

Steps 1-2 and 3-4 are separable if it's easier to land the ingest path first.

References

  • RFC-0054 — Page 2 spec · Unresolved Question #1 · full doc
  • clickhouse_db_schema/default.crcr_workflow_job/schema.sqlfailed_tests_json String DEFAULT '' COMMENT 'JSON array of failed/errored test details'

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions