[8.0.1xx] Binaries detected

[MichaelSimons]

The following binaries are being detected in recent builds (example):

src/aspnetcore/src/Servers/IIS/AspNetCoreModuleV2/AspNetCore/HtmlResponses.rc
src/fsharp/vsintegration/src/FSharp.LanguageService/VSPackage.resx
src/installer/src/finalizer/native.rc
src/sdk/src/GenAPI/genapi.slnf
src/source-build-externals/src/application-insights/BASE/.PreReleaseVersion

[NikolaMilosavljevic]

Hmm, this is odd - first file in the list was added 3 years ago, so it isn't new. Baseline file was last updated 2 years ago.

Investigating...

[NikolaMilosavljevic]

Investigation details.

None of the files are new - present in VMR for more than 2 years. This is likely a result of updates in the tooling that runs the binaries check - darc vmr scan-binary-files.

My plan is to update the list of allowed binaries. @MichaelSimons any concerns with this approach?

[MichaelSimons]

Something seems fishy here. There are numerous other files of these same types/extensions. I feel like we need a more definite RCA to ensure this doesn't regress again and to ensure the binary scanning is even working correctly. Looking at the diff from when this first occurred versus the previous success, the os version upgrade seems suspicious.

[NikolaMilosavljevic]

All of these flagged files are Unicode text files that have a Unicode BOM. This seems to be triggering the scanning tool and marking them as binaries. It is possible that the updated OS image is contributing.

None of the other .rc, .slnf, .resx files are Unicode. Some of them have UTF-8 BOM, which isn't flagged by the scanner.

Few options to fix:

1. add these files to baseline
2. change these files to UTF-8
3. investigate OS/scanner environment and fix the bug or update the OS

Option 3 seems like the right one, but possibly as a separate lower-pri issue. I think option 1 is the right immediate fix.

[MichaelSimons]

Thanks for analysis. I agree option 1 is the right immediate fix. Regarding a longer term fix, I would verify the behavior in main, perhaps this is resolved there. If this is not an issue in main I don't think any follow-up is necessary. If it is an issue in main, please log a new issue to track a long term solution.

[NikolaMilosavljevic]

Not an issue in main or in release/9.0.1xx - those branches do not use darc vmr scan-binary-files, they use BinaryToolKit which is not flagging these particular files as binaries.