Weak Hashing findings vanished from 1.1.11 ruleset?

[davewichers]

I maintain the OWASP Benchmark project:

• https://owasp.org/www-project-benchmark/
• https://github.com/OWASP-Benchmark/BenchmarkJava

Which is a test suite for testing the effectiveness of software security analysis tools. I've had codeQL scripts for scanning OWASP Benchmark for a while.

They can be found in the scripts folder here: https://github.com/OWASP-Benchmark/BenchmarkJava/tree/master/scripts

• translateCodeQL.sh
• runCodeQL.sh

You have to follow the codeQL install instructions listed in the translateCodeQL.sh script, then run translateCodeQL.sh and then runCodeQL.sh.

After completing, it puts the codeQL SARIF results file into the results/ folder and you can then score the tool against Benchmark by running createScoreards.sh.

I noticed that when using the latest version of CodeQL 2.20.1, with the 1.1.9 ruleset, it properly detected 69% of the Weak Hashing test cases in Benchmark (and had zero false positives). But when I upgraded to ruleset 1.1.11, it now detects none of them.

Is this on purpose? Or was a bug introduced. or mistake made, to cause those rules to go away in 1.1.11?

[redsun82]

Hey @davewichers, thanks a lot for reaching out with this! I'll circle this back to our internal team responsible for Java analysis 👍

[redsun82]

Hey @davewichers, as reported in this change note, we recently removed reporting of MD5 and SHA1 hashing from the java/weak-cryptographic-algorithm to the less precise java/potentially-weak-cryptographic-algorithm, as the former was alerting on too many cases of legitimate non cryptographic usages of those hashes. Maybe you can switch to using that query instead in your benchmarking?

[davewichers]

@redsun82 - the codeQL documentation related to finding out which rulepacks exist and how to use them is super confusing. Where is the list of ALL the published codeql rulepacks? I only found: codeql\java-queries and codeql\java-all, but when I try to use java-all, it says 'this is a library and does not contain queries to run'. I want to use ALL the codeQL java rules in my run. How do I do that?