Skip to content

fix: three false positives found by re-triaging warnings and notes - #50

Merged
HimanshuJanbandhu merged 1 commit into
mainfrom
fix/triage-tg007-tg002-tg001
Aug 19, 2026
Merged

fix: three false positives found by re-triaging warnings and notes#50
HimanshuJanbandhu merged 1 commit into
mainfrom
fix/triage-tg007-tg002-tg001

Conversation

@HimanshuJanbandhu

Copy link
Copy Markdown
Contributor

Read all 283 findings across the seven repos — individually, and at every severity, not just errors. The errors had already been re-read three times across this work. TG007 had never been looked at since it shipped, because it never produces an error. It was the worst rule in the set.

TG007 was 6/6 false, and every finding was the rule reporting its own advice

correct += (predicted == labels).sum().item()

That is verbatim what TG007's hint tells you to write: "Do the reduction on the device and sync once — (preds == targets).sum().item()." All six findings were this shape, in a validation loop nested inside a training loop.

The rule already had a batch-loop exemption — but it matched iterable names (loader, dataloader, batches), so it missed dev_iter and valloader.

Lengthening that list would have patched the instance. Instead the rule now requires evidence of per-element iteration: a loop over range(...) indexes elements, a loop over anything else yields batches whatever it happens to be called. That is also the shape the rule's own docstring uses to explain itself.

TG002 reported a .backward() nine lines above the call

fgsm_tutorial.py's test() was reported as "runs a forward pass with autograd enabled but never calls .backward()". It calls it on line 292 — an adversarial attack iterates test_loader and backwards through it deliberately, to get gradients with respect to the input.

The carve-out at fault exists for functions that both train and validate, where a backward elsewhere shouldn't excuse the validation loop. It just has to check the backward isn't in this loop.

TG001's bare-name return rule was too blunt

From #43: a returned bare name is not evidence of a deferred backward. Correct for return losses, where the container itself is handed back. Wrong here:

logps = torch.cat(all_logps, dim=0)
return logps, entropies, aux_loss      # not the container -- a reduction of it

Returning that hands the caller a graph the GRPO objective backwards. A returned bare name now counts unless it is itself a holder — which can't be decided until every holder has been seen, so it resolves at leave_Module. Six findings across trl and torchtune.

Result

errors warnings notes
before 21 55 207
after 13 49 207

14 removed, zero new findings.

What came out clean

  • TG004 is accurate. A sample of the 207 notes were all genuine DataLoader calls missing num_workers or pin_memory. The problem was only volume and actionability, which making it a note under RFC 0003 already solved. No rule bug.
  • TG003 ×3 are true, and already fixed upstream in pytorch/examples#1424.
  • The 8 torchtune running_loss += current_loss warnings are true, at the right severity since the severity split.

Left open, recorded in the TODO

  • TG013 (10) is the one rule not carried to a verdict — mixed shapes, needs individual reads.
  • TG008 (31) deserves a decision rather than a fix: factually right, but mostly synthetic-data examples where seeding changes nothing observable. Same profile that moved TG004 to note, so it should face the same test — is this code defective, or merely untuned?
  • Two false-positive causes remain, both surfacing in TG002: non-model callables read as forward passes (tokenizer(...), feature_extractor(...)), and deliberate gradient use for attribution.

Testing

443 tests, 3 new — one per fix, each reduced from the file that exposed it.

Read all 283 findings across the seven repos rather than clustering
them, and at every severity. The errors had been re-read three times
across this work; TG007 had never been looked at since it shipped,
because it never produces one. It was the worst rule in the set.

TG007 was 6/6 false, and each finding was the rule reporting its own
advice. Every one was `correct += (predicted == labels).sum().item()` in
a validation loop nested in a training loop -- verbatim what the hint
tells you to write. The batch-loop exemption matched iterable *names*
(`loader`, `dataloader`, `batches`) so it missed `dev_iter` and
`valloader`. Lengthening the list would have patched the instance; the
rule now requires evidence of per-element iteration instead, since a
loop over `range(...)` indexes elements and a loop over anything else
yields batches whatever it is called.

TG002 reported that `fgsm_tutorial.py`'s `test()` "never calls
`.backward()`" while the call sat nine lines below. An adversarial
attack iterates `test_loader` and backwards through it deliberately, to
get gradients with respect to the input. The carve-out being corrected
exists for functions that both train and validate, where a backward
elsewhere should not excuse the validation loop -- it just has to check
the backward is not in *this* loop.

TG001's bare-name return rule was too blunt. `return logps` where
`logps = torch.cat(all_logps, dim=0)` is not the container being handed
back, it is a reduction of it, and returning it hands the caller a graph
the GRPO objective backwards. A returned bare name now counts unless it
is itself a holder, which cannot be decided until every holder has been
seen. Six findings across trl and torchtune.

14 removed, no new findings. Errors 21 -> 13, warnings 55 -> 49.

TG004 came out accurate: a sample of the 207 notes were all genuine
DataLoader calls missing num_workers or pin_memory. The problem was
volume, which making it a note already solved.
@HimanshuJanbandhu
HimanshuJanbandhu merged commit a7dc6c3 into main Aug 19, 2026
8 checks passed
@HimanshuJanbandhu
HimanshuJanbandhu deleted the fix/triage-tg007-tg002-tg001 branch August 19, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant