ignored_dependency_scopes does not exclude npm devDependencies #2215

Description

The ignored_dependency_scopes setting in scancode-config.yml does not appear to exclude npm devDependencies during a ScanCode.io scan.

I encountered this while scanning ScanCode Workbench 4.0.2. I intentionally used the same configuration that is documented in the official ScanCode.io end-to-end tutorial:

product_name: scancode-workbench
product_version: '4.0.2'
ignored_patterns:
  - 'tests/*'
ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

The scancode-config.yml is located at the root of the project inside a single uploaded source archive.

The scan completes successfully, but dependencies with the scope devDependencies are still included instead of being ignored.

This seems to affect the ignored_dependency_scopes filtering itself rather than npm dependency detection, since ScanCode Toolkit correctly identifies these dependencies with the scope devDependencies.

How To Reproduce

Steps:

1. Use ScanCode Workbench 4.0.2 as the npm project.
2. Add the following scancode-config.yml to the project root:

product_name: scancode-workbench
product_version: '4.0.2'
ignored_patterns:
  - 'tests/*'
ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

3. Create a source archive containing the project and configuration.
4. Create a ScanCode.io project and upload the archive as a single input.
5. Run package inspection/scanning so that npm dependencies from package-lock.json are collected.
6. Inspect the resulting dependencies.

Dependencies with:

scope: devDependencies

are still present even though they should be ignored by the project configuration.

The provided package-lock.json uses lockfile version 3 and contains a large number of dependencies marked as development dependencies, so the behavior is easy to reproduce.

Expected behavior

npm dependencies with the scope devDependencies should be ignored when the following configuration is present:

ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

According to the ScanCode.io documentation, dependencies matching an ignored package type and scope should not be created or resolved.

This configuration is also the exact example currently used in the ScanCode.io tutorial with ScanCode Workbench 4.0.2.

Actual behavior

npm dependencies with scope: devDependencies are still created/included in the scan results.

Suspected root cause

I looked into the implementation and suspect that the issue is in ignore_dependency_scope() in:

scanpipe/pipes/__init__.py

The function currently retrieves the package type and scope from the dependency data:

dependency_package_type = dependency_data.get("package_type")
dependency_scope = dependency_data.get("scope")

and only evaluates the ignore rule when both values are present.

However, ScanCode Toolkit's DependentPackage / Dependency model does not appear to contain a package_type field.

The dependency model contains fields such as:

purl
extracted_requirement
scope
is_runtime
is_optional
is_pinned
is_direct
resolved_package
extra_data

For an npm dependency, the package type is already represented by the PURL, for example:

pkg:npm/...

while the dependency scope is correctly represented as:

devDependencies

Therefore, for a normal dependency generated by ScanCode Toolkit, this lookup appears to result in:

dependency_data.get("package_type")  # None

As a consequence, the following condition is not entered:

if dependency_package_type and dependency_scope:

and the configured ignore rule is never evaluated.

Relation to the original implementation

ignored_dependency_scopes was introduced as part of #1197 and implemented in #1235.

There is another interesting detail in the tests added with #1235: for the positive ignore_dependency_scope() test case, package_type appears to be manually added to the dependency dictionary before calling the function.

In other words, the test verifies the behavior with something similar to:

dependency_data["package_type"] = "pypi"
dependency_data["scope"] = "tests"

This may explain why the implementation passes its unit test even though a normal dependency produced by ScanCode Toolkit does not contain that package_type field.

A regression test using an actual ScanCode Toolkit Dependency / DependentPackage object may expose the problem.

Possible approach

One possible solution could be to derive the package type from the dependency PURL when package_type is not explicitly available.

For example, a dependency such as:

pkg:npm/example@1.0.0

already provides the required package type (npm).

Alternatively, the dependency data could be normalized so that the package type is populated before ignore_dependency_scope() is called.

The exact implementation is of course up to the maintainers.

Related issues / changes

• Enhance and create detailed documentation for scancode-config.yml #1197 – Enhance and create detailed documentation for scancode-config.yml
• Add support for ignored_dependency_scopes field for configuration #1235 – Add support for ignored_dependency_scopes field for configuration
• Support storing scancode-config.yml files in the scanned codebase #1236 – Support storing scancode-config.yml files in the scanned codebase

#1236 specifically documents that, for a single archive input, a scancode-config.yml located at the root should be used.

The official ScanCode.io end-to-end tutorial also currently uses ScanCode Workbench 4.0.2 together with:

ignored_dependency_scopes:
  - package_type: npm
    scope: devDependencies

which makes this project a convenient reproducer.

System configuration

• ScanCode.io: 37.2.0
• ScanCode Toolkit: 32.5.0
• Installation: Docker
• Package ecosystem: npm
• npm lockfile: package-lock.json, lockfile version 3
• Input: single source code archive containing scancode-config.yml

I also checked the current ScanCode.io main branch, and ignore_dependency_scope() still appears to use dependency_data.get("package_type") in the same way.