Release v1.10.0

1.10.0 - 2023-02-08

Added

• Experimental support for XML (gh-5939)
• Rust: Beta support for Rust. (gh-6545)
• Rule syntax: Metavariable bindings bound within metavariable-pattern now persist to outside of the metavariable-pattern (pa-2490)
• Updated all lockfile parsers (except Cargo.lock) to produce better error messages, at the cost of a couple seconds of lowdown on large (>10k lines) lockfiles (sc-better-parsers)

Changed

• Pro: --pro will now enable all Pro features, including Apex, inter-procedural taint
  analysis, and also inter-file analysis for supported languages. For Apex support only
  (and more languages in the future) now use --pro-languages. For intra-file analysis
  only now use --pro-intrafile. Flags --interproc and --interfile are now
  deprecated. (pa-2488)
• The output formatting of semgrep ci is getting revamped in the coming weeks.
  This release includes the first couple changes to the output. (sc-590)
• Packages from the maven ecosystem are now parsed to include their org slug. This means a log4j rule must now use org.apache.logging.log4j:log4j-core instead of just log4j-core. This change is backwards incompatible, in that any Java Supply Chain rules not taking into account will stop producing any findings, since the packages parsed from lockfiles will include the org, but the old rules will not. (sc-maven-org)

Fixed

• Rust: correctly parse the last expression in blocks (gh-7071)
• Dataflow traces: Findings now always display the separating line with --dataflow-traces in the CLI, to reduce
  confusion over where the findings fall between the dataflow traces. (pa-2471)
• CLI: Added install-semgrep-pro to the list of commands in the semgrep --help help text. (pa-2505)
• Fixed bug where gradle.lockfile files would fail to parse if they contained a trailing newline, and bug where an error on a trailing newline would cause our lockfile parse error pretty printing to fail (sc-trailing-newline)

Release v1.9.0

1.9.0 - 2023-02-01

Added

• Pro: If the "Semgrep Pro Engine" toggle is enabled in App, semgrep ci will add
  support for Apex in all scans (including diff scans). (pa-2462)

Fixed

• Fix incorrectly reformatted Bitbucket repository URLs (app-3279)

Release v1.8.0

1.8.0 - 2023-02-01

Added

• Experimental support for Clojure, Lisp, and Scheme. (gh-3328)
• Experimental support for Jsonnet (writing semgrep rules to find
  bugs/security-isses/... in jsonnet files). (pa-1849)

Fixed

• Regexps within string patterns such as "=~/hello/" now support the
  full PCRE syntax, including backreferences and other advanced features
  not supported by ocaml-re. (gh-6913)
• Removed new lockfile parsers for all lockfiles except requirements.txt and maven_dep_tree.txt, for performance reasons (sc-557)

Release v1.7.0

1.7.0 - 2023-02-01

Added

• Experimental support for Clojure, Lisp, and Scheme. (gh-3328)
• Experimental support for Jsonnet (writing semgrep rules to find
  bugs/security-isses/... in jsonnet files). (pa-1849)

Fixed

• Regexps within string patterns such as "=~/hello/" now support the
  full PCRE syntax, including backreferences and other advanced features
  not supported by ocaml-re. (gh-6913)
• Removed new lockfile parsers for all lockfiles except requirements.txt and maven_dep_tree.txt, for performance reasons (sc-557)

Release v1.6.0

1.6.0 - 2023-01-27

Added

• Added exposure property to sarif output for sca findings (sarif)

Changed

• Tests: Allow --test to process entire file trees rather than single files (gh-5487)
• metavariable-pattern: For performance reasons Generic mode ignores target files
  that look like machine generated. Unfortunately, this also prevented using the
  metavariable-pattern operator on text that may look like (or in fact be)
  machine generated, such as an RSA key contained in a legit file. Now, when the
  analysis is requested within a metavariable-pattern operator, Generic mode
  will always match any text even if it looks like machine generated. (pa-2386)
• Pro: Add experimental flags --pro and --interproc. Using --pro you can
  enable Apex support, and with --fast-deep you can enable intra-file inter-procedural
  taint analysis. Also, --deep has been renamed to --interfile. Note that to use
  any of the Pro features you must first run semgrep install-semgrep-pro while being
  logged in. (pa-2440)
• New lockfile parsers with better error messages for all supported ecosystems except Rust (sc-495)

Fixed

• Solidity: parse correctly 'unchecked' blocks (gh-6055)
• Terraform: Implicit coercions between strings, bools, ints, and floats are now allowed to match. (gh-6898)
• Python: Taint now propagates via the splat operators * and **, thus both
  sink(*tainted) and sink(**tainted) will result in findings. (gh-6920)
• Solidity: support ellipsis in contract inheritance
  Thanks to Raz0r for his contribution! (gh-6935)
• CLI: No longer fails when running tests against a config file with no rules in it (gh-6954)
• Fixed a crash that could occur when using the GitHub API to fetch the merge base rather than computing it locally. (merge-base)
• JS/TS: Object types now support metavariables properly (pa-2414)
• CLI: No longer reports the wrong command if you are using the semgrep-core-proprietary executable. (pa-2417)
• Pro: Findings that occur due to rules run with the Semgrep Pro Engine are now displayed separately in the CLI. (pa-2432)
• Fixed bug where Semgrep would generate reachable findings for transitive dependencies, even if a direct dependency at a safe version exists (sc-547)
• Fixed bug in poetry.lock parser where quoted square brackets could cause failures (sc-548)
• Fixed bug where Semgrep would fail to generate unreachable findings for a particular library if a reachable finding for that library had already been found in the same lockfile, even if there are multiple copies of this library at different vulnerable versions. (sc-549)
• Fixed bug where npm lockfiles with packages that listed multiples hashes would fail to parse (sc-550)

Release v1.5.1

1.5.1 - 2023-01-20

Added

• Python: Constant propagation will now recognize the idiom cond and X or Y,
  as well as True and X and False or X. So e.g. cond and "a" or "b" will
  be identified as a constant string. (gh-6079)
• Julia: Julia is now experimental (pa-2366)

Changed

• DeepSemgrep is now Semgrep PRO! To install the Semgrep PRO engine run:
  semgrep install-semgrep-pro. This engine is still invoked using the
  --deep flag, but please expect changes to the CLI in the near future.
  The new Semgrep PRO engine adds support for Apex! (pa-2389)

Fixed

• New 'transform:' field in extract mode rules, with 'concat_json_string_array'
  option useful to extract python code from jupyter notebooks.
  Thanks to Jose Selvi for his contribution! (gh-4477)
• Java: Fixed regression introduced in 0.123.0 that could cause a private class
  attribute to be incorrectly regarded as a constant. (gh-6793)
• Make $F(x) match eval(x). Previously, eval was special-cased and metavariable function call patterns would not match it. (gh-6877)
• DeepSemgrep: Enabled --dataflow-traces by default when --deep is specified (pa-2274)
• In rare situations, mainly in DeepSemgrep and related to naming bugs, the use of
  symbolic propagation could make Semgrep fall into an infinite loop during matching.
  This has been fixed by bounding the number of times that Semgrep can follow
  symbolically-propagated values. (pa-2324)
• CLI: Made an error message for when two autofix matches overlap have a more helpful message, as well as be displayed as a debug message. (pa-2393)
• CLI: Made the warning message when using Semgrep Pro more friendly when logged in (pa-2396)

Release v1.3.0

1.3.0 - 2023-01-04

Changed

• Removed the poor support for reading dependencies from pom.xml files. Instead semgrep will try to read dependencies from a maven_dep_tree.txt file,
  which can be generated using the command mvn dependency:tree -DoutputFile=maven_dep_tree.txt (sc-pom)

Fixed

• Use the GitHub REST API when possible to compute the merge base for semgrep ci, improving performance on shallow clones of large repositories. (gha-mergebase)

• YAML: Fixed a bug where metavariables matching YAML double-quoted strings would not capture the entire range of the string, and would
  not contain the double-quotes. Also added the ability to properly use patterns like "$FOO", which will unpack the contents of the matched string. (pa-2332)

• Fixed a race condition related to the parsing cache that could lead to internal errors (pa-2335)

• YAML: Fixed a bug where literal or folded blocks would not be parsed properly.

  So for instance, in:

  key: |
    string goes here
  

  A metavariable matching the contents of the string value might not be correct. (pa-2347)

• Julia: Greatly improved parsing support (pa-2362)

Release v1.2.1

1.2.1 - 2022-12-16

Fixed

• Go: fix NoTokenLocation for metavariables matching function type without
  an argument (e.g. func()) (gh-6715)
• typed-metavariables: handle ternary expressions so we can type correctly
  expressions like foo(cond ? new A() : this.a) (pa-2328)
• Reverted a change which caused findings with different sources (but the same sink) to be deduplicated. This would cause findings which
  looked identical in range and data, but had different taint traces. (pa-2336)

Release v1.2.0

1.2.0 - 2022-12-15

Fixed

• Fixed rare crash that could occur due to stale file caches when temp file names overlap (cache-invalidation)
• PHP: support metavariables in string (as in foo("$VAR")) (gh-6311)
• Java: support static block patterns (gh-6366)
• Rust: parse correctly scoped identifier in constructor (gh-6594)
• Java: support super(...) patterns (gh-6638)
• C#: bugfix on bad interaction between -fast and the internal code generated
  for LINQ queries (gh-6666)
• Java: support the Java 10 'var' keyword by not using 'var' as a valid type when
  using typed metavariables. (gh-6672)
• When matching module specifiers in imports, prevent metavariables from capturing quotes. (gh-6674)
• Swift: support complex expression in switch case (gh-6704)
• Constant propagation: Fixed a bug where constant propagation would only run within functions. Now,
  it runs on the top-level of the program as well. (pa-1656)
• DeepSemgrep: Fixed a bug where imports which reached type names (among other things) would not resolve properly (pa-2260)
• DeepSemgrep: Fixed a bug which caused Java functions with interfaces to wipe taint (instead of propagating taint by default) (pa-2265)
• Fix matching of qualified names. For example, new $X.Foo() will now match
  new a.b.Foo(). (pa-2296)
• DeepSemgrep: Fix regression in taint-mode introduced by Semgrep v1.1 that caused
  some findings to be missed. Also, DeepSemgrep will assume, for now, that a method
  call on a tainted object is always tainted. (pa-2304)
• Improved matching behavior on JS require calls (require-match)

Release v1.1.0

1.1.0 - 2022-12-05

Added

• JSON output: Added a max_memory_bytes field to the semgrep --time output which corresponds to the amount of memory allocated during the OCaml phase of Semgrep. This is useful for telemetry purposes. (pa-2075)

Changed

• Don't print out summary of blocking rules unless running with semgrep ci subcommand (gh-6651)

Fixed

• taint-mode: In 0.94.0 we made that when a pattern-source (or pattern-sanitizer)
  matched a variable exactly, this was understood as that variable being tainted
  (sanitized, resp.) by side-effect. For example, given tainted(x) we would taint x
  by side-effect, and subsequent occurrences of x were also considered tainted.
  This allowed to write rules like c.lang.security.use-after-free.use-after-free
  in a very succint way, and it also addressed some limitations of the workarounds that
  were being used to simulate this until then.

  This worked well initially, or so we thought, until in 0.113.0 we added
  field-sensitivity to taint-mode, and in subsequent versions we made sources and
  sanitizers apply by side-effect to more kinds of l-values than just simple variables.
  It was then that we started to see regressions that were fairly unintuitive for users.
  For example, if $_GET['foo'] was a taint source, this would make $_GET itself to
  be tainted by side-effect, and a subsequent expression like $_GET['bar'] was also
  considered tainted.

  We now correct the situation by adding the by-side-effect option to sources and
  sanitizers, and requiring this option to be explicitly enabled
  (that is, by-side-effect: true) in order to apply the source or the sanitizer by
  side-effect. Otherwise, the default is that sources and sanitizers matching l-values
  apply only to the precise occurrences that they match. (pa-1629)

• taint-mode: Fixed matching of pattern-sinks to be more precise, so that e.g.
  it will no longer report sink(ok1 if tainted else ok2) as a tainted sink, as
  the expression passed to the sink is actually not tainted. (pa-2142)

• CLI: Separated experimental rules from normal rules in semgrep --debug output. (pa-2159)

• Taint: Fixed an issue where findings with the same sink would be identified as the same, and cause
  only one of them to be reported, even if they had different sources. (pa-2208)

• DeepSemgrep: When the "DeepSemgrep" setting is enabled in Semgrep App, semgrep ci
  will try to run the analysis using the DeepSemgrep engine. But if this engine was
  not installed, semgrep ci failed. Now semgrep ci will automatically try to
  install DeepSemgrep if it is not already present. Note that, if DeepSemgrep is
  already installed, semgrep ci does not attempt to upgrade it to a newer version. (pa-2226)

• CLI: Made the number of jobs when using semgrep --deep default to 1. (pa-2231)

• Autofix: If multiple autofixes are targeting an overlapping range, then one of them is picked arbitrarily to occur, to prevent autofixes which may produce incorrect code. (pa-2276)

• DeepSemgrep: Time data now outputs properly when running semgrep --deep --time (pa-2280)

• DeepSemgrep: Added a message which suggests that users update their version of DeepSemgrep, if the DeepSemgrep binary crashes (pa-2283)

• Yarn 2 parse failure on versions like @storybook/react-docgen-typescript-plugin@canary. This is only present as some kind special version range specifier and never appears as a concrete version. It would only be used to check if the dependency was in the manifest file, so we just parse the version as "canary"
  Yarn 2 parse failure on versions like @types/ol-ext@npm:@siedlerchr/[email protected]
  Yarn 2 parse failure on versions like resolve@patch:resolve@^1.1.7#~builtin<compat/resolve>. These are now just ignored, as they appear to always come with a non-patch version as well. (sc-406)