Skip to content

Commit 4fe41b2

Browse files
authored
Render combined workflow execution traces
1 parent 458b11c commit 4fe41b2

6 files changed

Lines changed: 991 additions & 0 deletions

File tree

.github/scripts/tracing/YA_TRACE_TO_OTLP.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -751,3 +751,51 @@ bash .github/scripts/tracing/render_ya_trace_bundle.sh \
751751
--component cloud/blockstore \
752752
--operation tests
753753
```
754+
755+
## Workflow-level merging
756+
757+
Ya conversion produces one trace bundle per build/test invocation.
758+
[`workflow_trace_report.py`](workflow_trace_report.py) can later fetch the
759+
completed GitHub run's jobs and raw ya OTLP bundles. It creates workflow queue,
760+
job queue, job, and step spans, rewrites imported ya span IDs into the workflow
761+
trace ID, and attaches each imported root to the smallest containing/overlapping
762+
job. This step consumes OTLP; it does not reparse ya trace files.
763+
764+
For the first attempt, the workflow root retains GitHub's creation-to-update
765+
interval and its queue span. For reruns, `created_at` still refers to the
766+
original attempt and is ignored. The root instead begins at the earliest usable
767+
current `run_started_at` or job timestamp, then an imported-span timestamp; if
768+
none is usable, it deterministically falls back to `updated_at`.
769+
770+
GitHub `success` conclusions map to OTLP `OK`. Failure-like conclusions,
771+
including `stale`, map to `ERROR`; neutral and skipped conclusions remain
772+
`UNSET`.
773+
774+
## Code map
775+
776+
| Component | Responsibility |
777+
| --- | --- |
778+
| [`ya_trace_report.py`](ya_trace_report.py) | CLI, discovery, resource metadata, file list, bundle orchestration |
779+
| [`yatrace/projection.py`](yatrace/projection.py) | Root span, logical/evlog projection order, longest-test ranking |
780+
| [`projector.py`](projector.py) | Bound trace/resource/scope/parent context and stable span creation |
781+
| [`yatrace/trace_collection.py`](yatrace/trace_collection.py) | Safe discovery and enrichment lookup API |
782+
| [`yatrace/trace_loader.py`](yatrace/trace_loader.py) | JSONL parsing, merging, chunk assignment, attempt normalization |
783+
| [`yatrace/trace_model.py`](yatrace/trace_model.py) | `SuiteTrace`, `Chunk`, `TestAttempt`, `TestEvent`, durations, and logs |
784+
| [`yatrace/trace_spans.py`](yatrace/trace_spans.py) | Suite, chunk, ordered test timing, result markers, and inferred test stages |
785+
| [`yatrace/evlog_loader.py`](yatrace/evlog_loader.py) | Event-log parsing, details, failures, and statistics selection |
786+
| [`yatrace/node.py`](yatrace/node.py) | One normalized event-log `YaNode` and classification |
787+
| [`yatrace/evlog_data.py`](yatrace/evlog_data.py) | Immutable event-log aggregate |
788+
| [`yatrace/evlog.py`](yatrace/evlog.py) | Ya phases and event-log projection orchestration |
789+
| [`yatrace/test_operations.py`](yatrace/test_operations.py) | Direct worker/chunk matching and test operation spans |
790+
| [`yatrace/build_operations.py`](yatrace/build_operations.py) | Build envelope, direct selection, nodes, and commands |
791+
| [`yatrace/worker_spans.py`](yatrace/worker_spans.py) | Worker, phase, build-node, and command projection |
792+
| [`yatrace/statistics.py`](yatrace/statistics.py) | Cache, graph, execution, and aggregate attributes |
793+
| [`yatrace/critical_path.py`](yatrace/critical_path.py) | Ya critical-path parsing and build/test matching |
794+
| [`otlp`](otlp) | Official proto-backed types plus IDs, attributes, `Ns`, `Interval`, and `Trace` |
795+
| [`trace_io.py`](trace_io.py) | Validated, batched OTLP JSONL read/write |
796+
| [`trace_report.py`](trace_report.py) | Static HTML model/rendering and bundle output |
797+
| [`render_ya_trace_bundle.sh`](render_ya_trace_bundle.sh) | Shared nonblocking CI wrapper |
798+
| [`pack_ya_trace_inputs.sh`](pack_ya_trace_inputs.sh) | Literal-path raw-input tar archive |
799+
| [`workflow_trace.py`](workflow_trace.py) | Workflow, queue, job, step, and imported-span projection |
800+
| [`workflow_trace_report.py`](workflow_trace_report.py) | GitHub/S3 input loading and combined workflow bundle CLI |
801+
| [`render-workflow-trace.yaml`](../../workflows/render-workflow-trace.yaml) | Trusted post-workflow rendering and upload |

0 commit comments

Comments
 (0)