GitHub buildbots & Windows Defender updates - possible means to spread malware?

[julianrendell]

Latest release of Ruff + latest Windows defender -> malware detected and downloaded file immediately quarantined -> hash failure.

User reported error, but buildbot auto-closed it: #7287

It looks like there is a difference in (common) end user MS Defender updates/configuration and the buildbots.

Assuming end-user machines have newer defender & malware definition versions than the buildbots, should build-bot be automatically closing hash-errors triggered by Windows Defender?

e.g. a user's machine with latest defender and definitions correctly blocks an infected package and reports the issue. Buildbot closes the issue, as it has an older version of defender/definitions. If the user incorrectly trusts buildbot and overrides local quarantine, wouldn't they now be infected?

[Lutra-Fs]

Our CI is intentionally deterministic, hash-based, and AV-agnostic. GitHub’s Windows runners don’t give us a reliable, up-to-date, or reproducible Microsoft Defender signal, and Defender’s heuristic/PUA detections are non-deterministic by design. So we verify integrity (hashes) and basic installability; we do not try to “judge safety” in CI.

Details

• Runners don’t provide a stable Defender signal. Our actions run on windows-latest GitHub-hosted runners and the runner images are rebuilt on a weekly cadence. That means AV engines/definitions are not “live” the way an end-user box is, and they change across weeks—bad for reproducibility.

• Defender is set to Passive mode on the runner images. The runner-images team explicitly documents Microsoft Defender for Endpoint (MDE) running in passive mode on GitHub runners, precisely to reduce interference with builds. In passive mode, you won’t get the same quarantine/real-time actions that users see locally. Relying on it in CI would be misleading.

• Our CI’s scope is integrity & mechanics, not malware adjudication. The Scoop CI (“GithubActions” repo) verifies JSON, required fields, hashes of freshly downloaded files, checkver, and autoupdate. There’s no Defender step by design. When the Issues action can reproduce a good download & matching hash, it replies with “hashes are right” and closes as “no problem” with possible causes—because, from the bucket’s perspective, there isn’t one.

• What reporters hit is often an endpoint AV quarantine, not a bad hash. Examples:
  – ruff was quarantined by Defender on a user machine, which then made the hash step blow up because the file no longer existed. CI re-downloaded and hashed fine.
  – ngrok and ninja have had periodic Defender hits; again, that’s a Defender classification event, not a manifest integrity failure.

• We already warn users that AV can disrupt installs. Scoop’s own guidance and prior issues call out Windows Defender as a source of slowdowns/quarantines and suggest exclusions as a user-side mitigation. That’s another reason the bot treats these as environment issues rather than bucket defects.

So why not “enable Defender in CI” anyway?

1. Non-determinism & flakiness. Heuristic detections/PUA policy vary by definition version, cloud reputation, org policy, and even region—antithetical to deterministic CI. Weekly image rebuilds and passive mode make it worse.

2. Wrong layer of responsibility. Scoop points to upstream vendor artifacts and pins hashes. We don’t (and can’t) make a global “this binary is safe” assertion in CI; that’s an endpoint security policy decision.

3. Poor signal-to-noise. Many legitimate developer tools trip heuristics occasionally; baking that into PR gates would create churn without improving bucket quality (integrity remains the same). Cases above illustrate this.

Recommended path for reporters

• When Defender blocks/quarantines an installer, this is not a bucket bug. The fix is to submit the file/hash to Microsoft Security Intelligence as a false positive (or to the upstream vendor, who often does this). Microsoft provides a public submission portal for this exact workflow. Or if you have no trust for the software, you can uninstall the software immediately or install an earlier version using scoop install app@version. Or, after we introducing install history versions using git history feature, we might be able to introduce an option that the version installed on the user's machine must be published after a fixed period. However, this option needs to be discussed.

[julianrendell]

Thanks @Lutra-Fs for the clear explanation!

I didn't know about the MS false detection submission form (https://www.microsoft.com/en-us/wdsi/filesubmission).