[CI] Parity: fix upstream job links - #3536
Open
ethanwee1 wants to merge 13 commits into
Open
Conversation
The "Job ID" links in the parity summary were built by pairing every shard
dir's "_<job_id>" with a single "_wf_run_id" file. But the default,
distributed and inductor configs resolve to DIFFERENT upstream runs (e.g.
mi350 default+inductor come from a trunk push while distributed comes from
periodic / a trunk fallback), and they all unzip into the same rocm_xml
folder. "_wf_run_id" is overwritten per config (last write wins), so shards
from the other runs got a URL like runs/<trunk_run>/job/<distributed_job_id>
- a run the job never belonged to, which 404s ("busted" in the summary).
Concrete case: run 30444136839 (mi350) linked
runs/30416143612/job/90430654935, but job 90430654935 (mi350 distributed
shard 1) actually lives in run 30404499951.
download_testlogs now records a job_id -> run_id map in "_wf_run_ids.json"
next to the shard dirs (accumulated across configs), and
summarize_xml_testreports.py looks each shard's job id up in that map,
falling back to the legacy single "_wf_run_id" only when absent. Job URLs
are now built with the run that actually produced each shard.
Read CUDA and ROCm shard totals from workflow YAML at the tested commit, self-heal renamed job prefixes, and follow reusable-workflow artifacts to their hosting run.
6 tasks
Keep topology discovery aligned with the scheduled rocm-preview lane and remove the retired nightly key.
Keep job-link generation aligned with the scheduled rocm-preview lane and remove the retired nightly key.
4 tasks
Prevent same-SHA jobs from another ROCm workflow from self-healing to the wrong prefix by traversing nested reusable runs and filtering candidates by architecture.
Use the live rocm-preview mi350 topology and remove the retired nightly CLI/config key on the current develop history.
ethanwee1
changed the base branch from
develop
to
ethanwee/parity-preview-replacement
August 11, 2026 15:22
Match the unique rocm-preview lane without pinning its OS, Python, or runner suffix so routine upstream renames do not disable auto-parity.
Resolve the stacked Preview regex change while preserving canonical per-shard job URLs. # Conflicts: # .automation_scripts/pytorch-unit-test-scripts/parity_job_config.json
ethanwee1
changed the base branch from
ethanwee/parity-preview-replacement
to
main
August 11, 2026 19:33
ethanwee1
requested review from
jataylo,
jeffdaily,
jithunnair-amd and
pruthvistony
as code owners
August 11, 2026 19:33
ethanwee1
changed the base branch from
main
to
ethanwee/parity-preview-replacement
August 11, 2026 19:36
ethanwee1
changed the base branch from
ethanwee/parity-preview-replacement
to
develop
August 11, 2026 19:37
Use the restored eight-shard default matrix so parity downloads every live MI300 shard.
Use the restored eight-shard default matrix so parity downloads every live MI300 shard.
Resolve the Preview matcher in favor of the resilient workflow token and refresh fallback shard counts to current upstream topology.
ethanwee1
changed the base branch from
develop
to
ethanwee/parity-discover-jobs-develop
August 25, 2026 19:44
jithunnair-amd
pushed a commit
that referenced
this pull request
Aug 25, 2026
## Summary - replace the retired `nightly` parity key with `preview` - match the live daily `rocm-preview` `linux-noble-rocm-preview-py3.12-mi350` jobs - use the current 8/3/2 default/distributed/inductor fallback matrix This clean replacement is based on current `develop`; it supersedes #3406, whose pre-reset history cannot merge without a force-push or unrelated-history merge. ## Test plan - [x] Parse parity config JSON - [x] Compile `download_testlogs` - [x] Parse `parity.yml` as YAML - [x] Verify all 13 test jobs against scheduled [pytorch/pytorch run 31344407133](https://github.com/pytorch/pytorch/actions/runs/31344407133) - [x] Run end-to-end Preview parity dispatch ## Dependencies Land this before the Preview auto-trigger update in #3555 and before retargeting the operational parity PRs. Made with [Cursor](https://cursor.com) ## Alignment landing order This is the root dependency. After it lands, retarget the stacked PRs to `develop` in this order: 1. #3535 dynamic workflow/prefix/shard discovery 2. #3536 canonical per-shard job URLs 3. #3523 consolidated classifier rules (independent base; may land earlier) 4. #3559 variant-run filtering 5. #3558 graceful missing-config handling 6. #3557 partial-report preservation 7. #3560 authoritative flaky attribution 8. #3556 latest-SHA resolution 9. #3555 scheduled multi-architecture automation last Validation: full Preview [run 31506711303](https://github.com/ethanwee1/pytorch/actions/runs/31506711303) and combined Preview+MI350 [run 31507307756](https://github.com/ethanwee1/pytorch/actions/runs/31507307756) both succeeded on matching SHA `cf18dda7`. ## August 25 refresh - Merged current ROCm `develop` and reran JSON, Python, and workflow YAML validation. - The previously documented Preview and combined Preview+MI350 runs satisfy the end-to-end checklist item.
ethanwee1
changed the base branch from
ethanwee/parity-discover-jobs-develop
to
develop
August 26, 2026 16:10
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
Made with Cursor
Preview topology update
This branch now uses
previewonly and builds canonical per-shard links for the scheduledrocm-previewtopology. #3554 owns the canonical Preview config; duplicate Preview/config hunks are now inherited from #3535. Land after #3535.Alignment landing order
Depends on #3554 and lands after #3535. It is the canonical job-link layer consumed by subsequent reports.
August 25 refresh
Stacked on the refreshed [CI] Parity: discover job topology from workflow YAML #3535 topology branch.
The PR diff is now limited to per-job run-ID recording and summary-link generation.
Revalidated downloader and summarizer compilation plus parity config parsing.
Parity stack merge order
All open parity PRs are targeted to
develop. Merge in this order:#3259 is a separate follow-up: reconstruct FrameworkWeb ingestion after the operational stack lands.