[TD] Script to determine which reverts are caused by bad TD

[clee2000]

Honestly a pretty messy script

Determines a commit was caused by bad TD, basically:

1. find revert
2. find originally merged commit
3. find last commit on the PR before the merge
4. check merged commit for failures
5. check if failure was excluded by TD on the last commit on the PR

Obviously not perfect, but spot checking it does seem to be ok

Counts are pretty volatile if granularity is too small (week granularity is very volatile), so I'm not sure if this can really be displayed. Maybe should make this periodic and still upload to clickhouse? idk

End of the output looks like this but theres a lot of extra output before it to help debug

CAUSED BY BAD TD: 27 / 184 = 14.67%
Unable to check (lack run id) on PR: 1 / 184 = 0.54%
Total caused by bad TD: 27 / 184 = 14.67%
Month 674: 13 bad TD / 77 total = 16.88%
Month 675: 14 bad TD / 107 total = 13.08%

Also make clickhouse.py client able to be used in thread pool executor (http client didn't like having 1 http client for multiple threads i think)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Updated (UTC)
torchci	⬜️ Ignored (Inspect)	Visit Preview	Aug 5, 2025 10:36pm

[github-advanced-security]

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[Copilot]

Pull Request Overview

This PR adds a script to analyze reverts and determine which were caused by bad TD (Test Dependency) exclusions, along with making the ClickHouse client thread-safe.

Key changes:

• Adds a comprehensive script to analyze Git commit history and correlate reverts with TD exclusions
• Removes the @lru_cache decorator from get_clickhouse_client() to enable thread-safe usage
• Implements parallel processing using ThreadPoolExecutor for performance optimization

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
tools/torchci/td/get_reverts_caused_by_td.py	New script that analyzes Git history, queries ClickHouse for job failures, and determines if reverts were caused by bad TD exclusions
tools/torchci/clickhouse.py	Removes @lru_cache decorator from get_clickhouse_client() to make it thread-safe for concurrent usage

[Copilot]

This line uses timestamp_of_merge but the loop is iterating over commits_reverted, not caused_by_bad_td like the previous loop. This will incorrectly group all reverted commits by their merge timestamp instead of revert timestamp for the month calculation.

Suggested change

	month = commit.timestamp_of_merge // (30 * 24 * 60 * 60)
	month = commit.timestamp_of_revert // (30 * 24 * 60 * 60)

[Copilot]

[nitpick] Explicit deletion of futures is unnecessary as it will be garbage collected when it goes out of scope. This adds clutter without benefit.

Suggested change

del futures

[Copilot]

If alt_last_pr_sha[0] is empty string (when no matching commit is found), setting commit.last_pr_sha to an empty string could cause issues in subsequent queries. Consider checking if alt_last_pr_sha[0] is not empty before assignment.

Suggested change

	if commit.last_pr_sha is None:
	if commit.last_pr_sha is None and alt_last_pr_sha[0]:

[Copilot]

[nitpick] This commented-out line should either be removed if it's not needed or properly documented with a comment explaining why it's disabled.

Suggested change

	x.last_pr_sha,
	x.last_pr_sha,
	# Disabled: revert_commit_sha_prev is not always available or needed for this analysis

[ZainRizvi]

Code is kinda hard to read, but the methodology makes sense. This is fine for a tool you run by hand, but if you plan to integrate it into automation I'd ask that you add some sort of tests to it so that folks can tell if a change regresses behavior.