commoncode-v32.5.1

This is a minor bugfix release

• Minor fix in pyproject.toml to release wheels at pypi properly.

commoncode-v32.5.0

This is a minor release of commoncode with added functionality
and migrated source repo.

• Merge commoncode back into scancode-toolkit
  #5116

• Add support to create codebase from multiple input paths by
  starting codebase walk from these inputs and then ignoring
  based on path patterns. Improves codebase and resource
  collection and creation performance for multi-path scan inputs
  #5055

v33.0.0rc1

This is a major release candidate with major changes in packaging,
with the licensedcode-data and licensedcode-index being published
in two seperate wheels. Also adds linux/macos ARM support in
release archives and pypi wheels.

• Remove the licensedcode data and built license indexes from the
  main scancode-toolkit built wheel, and release them as
  seperate wheels which scancode-toolkit depends on.
  This enables to release scancode licensedb data/index wheels
  seperately whenever necessary and also reduces the scancode
  wheel size greatly. For more details see:
  #3459

• Add macos ARM and linux arm support in release archives and
  pypi wheels through scancode-toolkit-mini which uses
  system provided libraries instead of bundled binaries.
  Also use updated non pure-python dependencies supporting
  linux/macos arm wheels. For more details see:
  #3205
  #3958

• Add new and updates licenses, and license rules, including
  support for the SPDX License List 3.28
  #4956

• Add support for the Python UV package manager. Two new package data
  handlers parse pyproject.toml files containing a [tool.uv] table
  and uv.lock lockfiles, including PEP 735 [dependency-groups],
  and the package assembly walks both files together so that the project
  metadata and the resolved transitive dependencies are reported as a
  single Python package.
  #4501

What's Changed

• Fix pypi python3.14 wheel release by @AyanSinhaMahapatra in #4686
• Rules for golang related false positives by @CsatariGergely in #4694
• Pynacl false positive upstream by @CsatariGergely in #4695
• Fix dead scancode documentation link by @ShraddhaSharma3 in #4692
• Add new licenses and update SPDX License List to v3.28.0 by @AyanSinhaMahapatra in #4625
• Update Pixar regression fixture by @kumarasantosh in #4878
• Remove duplicate '/project.clj' key in _MANIFEST_ENDS dict by @codewithfourtix in #4781
• Update licenses by @AyanSinhaMahapatra in #4956 thanks to @DennisClark @armijnhemel @pombredanne for reporting and adding these.
• Publish licensedcode data/index as seperate pypi wheels and improve arm installation support by @AyanSinhaMahapatra in #4847
• Make 'no such option' assertion tolerant of newer click error format by @GuillemSeCa in #5009
• Adding UV package manager support for Python projects by @GuillemSeCa in #4980
• Bump version and CHANGELOG to v33.0.0rc1 and add new licenses by @AyanSinhaMahapatra in #5041

New Contributors

• @CsatariGergely made their first contribution in #4694
• @ShraddhaSharma3 made their first contribution in #4692
• @kumarasantosh made their first contribution in #4878
• @codewithfourtix made their first contribution in #4781
• @GuillemSeCa made their first contribution in #5009

Full Changelog: v32.5.0...v33.0.0rc1

v32.5.0

This is a minor scancode release with:

• license, package detection performance improvements

• better copyright, license and package detection

• python3.14 support, and removed python3.9 wheels

• Improve package scan performance by:

  • Skipping binary package detection steps by default,
    and introducing a new CLI option --package-in-compiled
    to detect packages in compiled binaries like rust/go binaries
    Note: previously these were detected in --package CLI
    option directly but this is no longer the case, you've to
    use the new --package-in-compiled to detect packages
    from go/rust and other binaries

  • Creating cached regex patterns and multiregex pre-matchers,
    for a fast package path detection filtering step

  #4606

• Add gibberish detection to copyright scanning. This is done using a
  2-character Markov chain. A new CLI command,
  scancode-train-gibberish-model, has been added to regenerate the model
  used by the detector.
  #4610
  #2402

• Use a cython implementation of sequence matcher to improve
  license detection performance
  #4500

• Add python3.14 support in scancode
  #4595
  #4541

• Always print license references table in html output
  #4474

• Enable License References table for HTML Output without requiring
  --license-references by implementing a fallback license reference
  collection based on the behavior of v32.0.0.
  #4474
  #4101

• Replace unmaintained toml library with tomllib / tomli.
  #4532

• Pin fingerprints and normality to avoid pyicu
  #4493

• Fix click compatibility issues and failures
  #4572

• Remove deprecated ast module attributes
  #4539

• Fix cargo scanning failures
  #4581

• Reorg and improve docs user experience
  #4629

• Handle is_private strings in npm properly
  #4635

• Restructure README docs
  #4667

• Add new licenses, license rules and misc license detection
  improvements
  #4562
  #4674
  #4666
  #4622

What's Changed

• feat(OutputHTML): Always print license references table by @pepper-jk in #4474
• Cython seq by @JonoYang in #4500
• Update cyseq to v0.0.2 by @JonoYang in #4512
• Add new licenses to LicenseDB by @AyanSinhaMahapatra in #4517
• Pin fingerprints and normality by @AyanSinhaMahapatra in #4531
• Fix click compatibility issues by @AyanSinhaMahapatra in #4591
• packagedcode: replace unmaintained toml with tomllib/tomli by @AyanSinhaMahapatra in #4594
• packagedcode: replace unmaintained toml with tomllib/tomli by @gotmax23 in #4532
• packagedcode: don't use removed ast module attributes by @gotmax23 in #4539
• Add new licenses from dejacode by @AyanSinhaMahapatra in #4562
• Fixes #4581: Failure to scan cargo by @omsuneri in #4582
• Add tag to ruby license rule by @pombredanne in #4622
• docs: Reorg and improve user experience by @tsteenbe in #4629
• Handle string is private by @AyanSinhaMahapatra in #4635
• Restructure the README by @DennisClark in #4667
• Detect plantuml license by @uttam282005 in #4666
• Improve package scan performance by @AyanSinhaMahapatra in #4606
• Detect gibberish copyright #2402 by @JonoYang in #4610
• Make CC-BY-NC-3 rule more specific by @alexzurbonsen in #4674
• Add python3.14 to test support by @AyanSinhaMahapatra in #4595

New Contributors

• @pepper-jk made their first contribution in #4474
• @omsuneri made their first contribution in #4582
• @uttam282005 made their first contribution in #4666

Full Changelog: v32.4.1...v32.5.0

v32.4.1

This is a patch release with misc bugfixes and improvements:

• Fix broken scancode release archives and release scripts
  #4469

• Support licenses from SPDX License List 3.27
  #4468

• Add new licenses, license rules and license data updates
  #4478

• Use updated license-expression v30.4.4 with updated licenses
  https://github.com/aboutcode-org/license-expression/releases/tag/v30.4.4

• Use pygmars v1.0.0 with copyright detection performance
  improvements by removing unnecessary regex substitutions.
  #4063

• Fix misc scancode doumentation related issues
  #4457
  #4458
  #4462
  #4465
  #4470
  #4471
  #4476

What's Changed

• Fixed #3276 - Added notes on docker build command on windows by @chinyeungli in #4458
• Documentation update by @chinyeungli in #4457
• Update links reference under the "Source code and downloads" section by @chinyeungli in #4465
• Fix some minor typos in home.rst by @joshuagl in #4462
• Add support for SPDX License List 3.27 by @AyanSinhaMahapatra in #4468
• Update pygmars by @JonoYang in #4477
• Add new licenses, license rules and license updates by @AyanSinhaMahapatra in #4478
• Add new FAQ entry on --license-text by @pombredanne in #4476
• Improve wording for windows' installation #1881 by @chinyeungli in #4471
• Updated/Corrected KEY file's description #2506 by @chinyeungli in #4470
• Release prep v32.4.1 by @AyanSinhaMahapatra in #4482

New Contributors

• @joshuagl made their first contribution in #4462

Full Changelog: v32.4.0...v32.4.1

v32.4.0

Note: The scancode app archives were broken for this release, please use the next release: v32.4.1 or later where this issue is fixed. See #4469 for more details.

This is a feature release with:

• python 3.13 support
• support added for adding required phrases to rules automatically
• misc license and package detection improvements
• new and updated license detection rules and new licenses
• misc bugfixes, dependency and documentation updates

There are new data attributes, and we have a output format version bump
from 4.0.0 to 4.1.0. The changes in Output Data Structure are:

• A new resource level attribute sha1_git is added, which has
  the corresponding checksum value for files, and is empty for
  directories. This is returned optionally with the --info plugin.

• A new resource level attribute is_community is added, which is
  True from commonly used files used for community/project maintainence.
  This is returned optionally with the --classify plugin.

These are the details for the most important changes introduced::

• Add support for adding required phrases in rules automatically using
  some console scripts and CLI options using already marked required
  phrases for the same license-expression and license field attributes
  The new console scripts are:

  • add-required-phrases to add required phrases from other rules or
    license attributes
  • gen-new-required-phrases-rules to add required phrase rules for
    marked required phrase in rules
    This improves detection accuracy and reduces false positives.
    #3924
    #4237
    #4241

• Default value of processes used for scancode scans is changed from
  1 to N-1, where N is the number of CPU processes available in the
  system. #4104

• Also return sha1_git checksums for each files with --info plugin.
  #624

• Equivalent words like license and licence, as well as plurals are
  now treated as the same in license detection. With this,
  many redundant rules have been deprecated.
  #4215

• Support running scancode with python3.13
  Update and use latest native dependencies with py3.13 support,
  update and test py3.13 usage in CI and other scripts, and
  update other third-party dependencies, use latest skeleton
  #4430

• Misc license detection improvements, new licenses and license
  detection rules.
  #4261
  #4412
  #4405
  #4278
  #4093

• Fix an issues where pip install scancode-toolkit was failing
  because of a compatibility issue with Click
  #4427

Note: scancode-toolkit-mini could not be published because of pypi limits, tracked in #4452

What's Changed

• 4181 update about to use license expression by @chinyeungli in #4184
• Refine postgresql RULE by @alok1304 in #4111
• Update rules with required phrases automatically by @AyanSinhaMahapatra in #3924
• Add new rule for eupl license by @alok1304 in #4204
• Add required phrase markers to CC license rules by @dotarjun in #3644
• Fix minor message typo by @pombredanne in #4228
• Add tests for all PyPI METADATA versions (#4175) by @alok1304 in #4180
• Drop Ubuntu 20 by @pombredanne in #4240
• Fix false positive detection heuristics by @alexzurbonsen in #4009
• Add autosar license rules by @pombredanne in #4242
• Add new rule for mit by @alok1304 in #4121
• Update various license rules by @AyanSinhaMahapatra in #4093
• Add new license rules for new Elasticsearch notices by @NucleonGodX in #4041
• Refine required phrases with stopwords #4238 by @pombredanne in #4241
• Ensure opam tests are running by @pombredanne in #4271
• Support equivalent words in license detection #4190 by @pombredanne in #4215
• Improve maven license detection by @pombredanne in #4261
• Improve license required phrase generation by @pombredanne in #4237
• Fix and enhance support for different bazel metadata versions by @abraemer in #4194
• Add DUMB License (#4058) by @alok1304 in #4143
• Add test for false positive GPL3 license by @alok1304 in #4106
• Update test_detect.py by @pombredanne in #4311
• Update licenses and add rules by @AyanSinhaMahapatra in #4278
• Add test for equivalent word by @alok1304 in #4305
• Display extra-words in detection_log if present by @alok1304 in #4402
• fix: change version number in field "key" for code-credit-license-1.0-0 by @leoreinmann in #4416
• fix: change version number in field "name" for license "LiLiQ-R-1.1" by @leoreinmann in #4418
• Fix click compatibility issues with commoncode v32.3.0 by @AyanSinhaMahapatra in #4427
• Return sha1_git checksum for all files as info by @AyanSinhaMahapatra in #4425
• Improve license detection by @AyanSinhaMahapatra in #4412
• Improve npm license detection by @AyanSinhaMahapatra in #4405
• Documentation enhancement by @chinyeungli in #4448
• change default value of --process to (number of CPUs)-1 by @xsuchy in #4104
• Replace broken link for Sun Public License SPL 1.0 by @alok1304 in #4109
• Support python 3.13 with updated dependencies by @AyanSinhaMahapatra in #4430

New Contributors

• @alok1304 made their first contribution in #4111
• @dotarjun made their first contribution in #3644
• @NucleonGodX made their first contribution in #4041
• @abraemer made their first contribution in #4194
• @leoreinmann made their first contribution in #4416

Full Changelog: v32.3.3...v32.4.0

v32.3.3

This is a patch release with license and package detection improvements, bugfixes and with new and updated license detection rules and new licenses added.

• Add new and updated licenses and license rules #4165 #3819
• Bump commoncode to v32.2.1 and pin bs4 to fix copyright scan issues #4149 #4176
• Refactor and fix package assembly for pypi installed wheels and fix pypi manifest parsing #4171

What's Changed

• Refactor and fix package assembly for installed wheels by @AyanSinhaMahapatra in #4171
• Added support for "Caramel" license by @aayushkdev in #4159
• Bump commoncode to v32.2.0 and pin bs4 by @AyanSinhaMahapatra in #4149
• Sync licenses from dejacode license list by @AyanSinhaMahapatra in #4165
• Release prep v32.3.3 by @AyanSinhaMahapatra in #4176

Full Changelog: v32.3.2...v32.3.3

v32.3.2

This is a patch release with license and package detection
improvements, bugfixes and with new and updated license detection rules
and new licenses added.

Bugfixes:

• Fix package resource assignment for JAVA jars in scancode.io #3983
• Fix missing spdx license expression in license detections #4015
• Enforce --path as a required parameter for scancode-license-data
  console script. #4024
• Fix conda environment.yaml parsing errors. #4078
• Fix npm package parsing bug for packages with workspaces. aboutcode-org/scancode.io#1521

New features/licenses:

• Adds support for pnpm lock YAML v9 https://github.com/pnpm/spec/blob/master/lockfile/9.0.md
• Add licenses from SPDX License List 3.26 #4045
• Add assembly and identification of conda package files in
  root filesystem installations #4083

What's Changed

• Fix pnpm workspace parsing and udpate package detection by @AyanSinhaMahapatra in #4079
• Update licenses from SPDX License List 3.26 and others by @AyanSinhaMahapatra in #4081
• Fix missing spdx license expression in license detection by @alexzurbonsen in #4023
• Enforced --path as a required parameter for scancode-license-data module by @lyr-ast in #4029
• Update package assembly in conda installations by @AyanSinhaMahapatra in #4089
• Fix conda environment yaml parsing errors by @AyanSinhaMahapatra in #4078
• Fix package resource assign bug for jar manifests by @AyanSinhaMahapatra in #3983
• Release scancode-toolkit v32.3.2 by @AyanSinhaMahapatra in #4090

New Contributors

• @lyr-ast made their first contribution in #4029

Full Changelog: v32.3.1...v32.3.2

v32.3.1

This is a minor release with license and package detection
improvements, bugfixes and with new and updated license detection rules
and new licenses added.

• We can now collect packages from a Rust binary using rust-inspector
  for rust binaries built with cargo-auditable(Linux-only)
  Also adds a plugin for colelcting rust symbols with the option
  --rust-symbol. See the initial release for more info:
  https://github.com/aboutcode-org/rust-inspector/releases/tag/v0.1.0
  #4043

• Improves and adds bugfixes for package detection in conda and npm.
  #4073

• Updates go-inspector to v0.5.0 . GoReSym is now built from source and has
  been updated to v3.0.1. #3972

• Adds new and updated licenses, license detection rules.
  #3963

• Adds the latest license-expression with an updated licenseDB.
  #3960

What's Changed

• 3955 update dockerfile by @JonoYang in #3957
• Bump license-expression to v30.4.0 by @AyanSinhaMahapatra in #3960
• Update go inspector by @JonoYang in #3972
• Declare ngram variable in select_ngrams by @JonoYang in #3976
• Improve package datafile handlers by @pombredanne in #3873
• Fix failing --no-check-version cli option by @alexzurbonsen in #4003
• Add new licenses and license updates by @AyanSinhaMahapatra in #3963
• Add rust binary support by @AyanSinhaMahapatra in #4043
• Fix misc package scanning bugs by @AyanSinhaMahapatra in #4073
• Release prep v32.3.1 by @AyanSinhaMahapatra in #4074

New Contributors

• @alexzurbonsen made their first contribution in #4003

Full Changelog: v32.3.0...v32.3.1

v32.3.0

Major API/other changes:

• Output Format Version updated to 4.0.0 (major version bump)
• Dependency attribute rename: is_resolved renamed to is_pinned See #3888 for more details.
• License Match attribute rename: spdx_license_expression is renamed to license_expression_spdx.

Changes in Output Data Structure:

• The data structure of the JSON output has changed for: - dependencies at file level package_data, and at top-level. - license matches at file level or unique codebase level license detections Note that the change is a modification to the JSON output, so we have a major version bump 3.2.0 to 4.0.0:
• Dependency attribute is_resolved renamed to is_pinned
• LicenseMatch attribute spdx_license_expression renamed to license_expression_spdx

What's Changed

• Improve npm workspace processing by @AyanSinhaMahapatra in #3857
• Fix Package/PackageData creation bugs in purldb by @AyanSinhaMahapatra in #3710
• Fix python package detection issues #3859 by @AyanSinhaMahapatra in #3869
• typo in how_to_run_a_scan.rst by @spheex in #3886
• Fix typo in NuGet package manager reference in README.rst by @Crown0815 in #3880
• Synchronize Licenses by @AyanSinhaMahapatra in #3897
• Improve copyright detection by @pombredanne in #3910
• Improve copyrights detection more by @pombredanne in #3917
• Update license rules and license detections by @AyanSinhaMahapatra in #3905
• Update licenses by @AyanSinhaMahapatra in #3887
• Consolidate Swift package assembly under a single BaseSwiftDatafileHandler by @keshav-space in #3855
• Improve Copyright Detection by @AyanSinhaMahapatra in #3929
• Apply miscellaneous license detection updates by @pombredanne in #3936
• textcode markup: fix SyntaxWarning in regex by @gotmax23 in #3891
• Update yarn lock parser #3931 by @JonoYang in #3943
• Detect go binary packages by @pombredanne in #3894
• Rename dependency is_resolved to is_pinned by @AyanSinhaMahapatra in #3888
• Rename license match attribute for spdx expression by @AyanSinhaMahapatra in #3851
• Prepare release v32.3.0 by @AyanSinhaMahapatra in #3952

New Contributors

• @spheex made their first contribution in #3886
• @Crown0815 made their first contribution in #3880
• @gotmax23 made their first contribution in #3891

Full Changelog: v32.2.1...v32.3.0