Skip to content

[autorevert] Fix bug on job retrival for autorevert lambda - #6979

Merged
izaitsevfb merged 6 commits into
mainfrom
jeanschmidt/autorevert_lambda_ergonomics
Aug 7, 2025
Merged

[autorevert] Fix bug on job retrival for autorevert lambda#6979
izaitsevfb merged 6 commits into
mainfrom
jeanschmidt/autorevert_lambda_ergonomics

Conversation

@jeanschmidt

@jeanschmidt jeanschmidt commented Aug 6, 2025

Copy link
Copy Markdown
Contributor

After fixing the bug, the new statistics are:

==================================================
SUMMARY STATISTICS
==================================================
Workflow(s): Lint, trunk, pull, inductor, linux-binary-manywheel
Timeframe: 4320 hours
Commits checked: 33519
Auto revert patterns detected: 1345
Actual reverts inside auto revert patterns detected (precision): 219 (16.3%)
Total revert commits in period: 585

Revert categories:
  nosignal: 202 (34.5%)
  ghfirst: 156 (26.7%)
  uncategorized: 104 (17.8%)
  ignoredsignal: 68 (11.6%)
  weird: 45 (7.7%)
  landrace: 10 (1.7%)

Total reverts excluding ghfirst: 429
Reverts (excluding ghfirst) that dont match any auto revert pattern detected (recall): 250 (58.3%)
Per workflow precision:
  Lint: 45 reverts out of 75 patterns (60.0%) [excluding ghfirst: 41 (54.7%)]
  trunk: 30 reverts out of 136 patterns (22.1%) [excluding ghfirst: 28 (20.6%)]
  pull: 104 reverts out of 859 patterns (12.1%) [excluding ghfirst: 92 (10.7%)]
  inductor: 39 reverts out of 269 patterns (14.5%) [excluding ghfirst: 38 (14.1%)]
  linux-binary-manywheel: 1 reverts out of 6 patterns (16.7%) [excluding ghfirst: 0 (0.0%)]

The main bug is that when checking for commits before/after with the same job, it actually concatenated all commits+jobs before and after, instead of only returning the next one.

I added also the ergonomics for lambda invocation

@pytorch-bot pytorch-bot Bot added the ci-no-td label Aug 6, 2025
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 6, 2025
@vercel

vercel Bot commented Aug 6, 2025

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Updated (UTC)
torchci ⬜️ Ignored (Inspect) Visit Preview Aug 7, 2025 9:53pm

@zxiiro
zxiiro requested a review from Copilot August 7, 2025 15:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a critical bug in the autorevert lambda's job retrieval logic and improves its ergonomics. The main bug was that when checking for commits with the same job before/after, the system concatenated all commits+jobs instead of returning only the next one. This resulted in inaccurate pattern detection.

  • Fixed job retrieval logic in _find_last_commit_with_job method
  • Added support for ignoring common error types (GHA errors, timeouts, sccache errors)
  • Enhanced lambda invocation ergonomics with better command-line options

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
autorevert.py Added ignore_common_errors parameter and improved output formatting
autorevert_checker.py Fixed job retrieval bug and added classification rule filtering
main.py Added lambda ergonomics and new command-line options
Makefile Updated run command to use new parameters

Comment thread aws/lambda/pytorch-auto-revert/pytorch_auto_revert/autorevert_checker.py Outdated
Comment thread aws/lambda/pytorch-auto-revert/pytorch_auto_revert/autorevert_checker.py Outdated
Comment thread aws/lambda/pytorch-auto-revert/pytorch_auto_revert/testers/autorevert.py Outdated

@zxiiro zxiiro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should resolve the typos but otherwise looks good to me!

@izaitsevfb
izaitsevfb merged commit 4eb9d0c into main Aug 7, 2025
5 checks passed
@izaitsevfb
izaitsevfb deleted the jeanschmidt/autorevert_lambda_ergonomics branch August 7, 2025 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-no-td CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants