discussion: Verifying package names

[ianlewis]

For some scenarios it might be necessary to verify a language ecosystem's package name (or other metadata) which requires inspecting the contents of the package artifact(tarball) itself.

For example, npm package provenance references the artifact by package name. The subject of the in-toto attestation is a purl referencing the package name with a sha512 of the package tarball.

If you run npm install package.tgz it will install the package with the name in the package.json metadata located inside the tarball. This could open users up to attacks where users think they are downloading and verifying package A but are in reality installing (and potentially overwriting) package B.

What should a SLSA verifier do (if anything) in this case? If verification is checking the source code repo, is that good enough?

[joshuagl]

I'm not very familiar with npm, so I'm writing down my interpretation to check whether I'm understanding the issue correctly.

• it's possible to do an npm install of a package tarball downloaded through a separate step
• an npm package's provenance describes the package (subject) by purl and digest of the package tarball
• the install name (and location?) of the package is determined by the package.json within the package tarball

Therefore, it's possible to download a package foo.tgz with provenance that is verified as foo.tgz but which ends up being installed as bar (and potentially overwriting any existing bar)?

In this scenario, wouldn't the combination of trusted builder and canonical source repository be sufficient to detect a malicious package? i.e., provenance metadata signed by a trusted builder would not describe the expected source repository for producing the foo.tgz tarball?

[laurentsimon]

I'm not very familiar with npm, so I'm writing down my interpretation to check whether I'm understanding the issue correctly.

• it's possible to do an npm install of a package tarball downloaded through a separate step
• an npm package's provenance describes the package (subject) by purl and digest of the package tarball
• the install name (and location?) of the package is determined by the package.json within the package tarball

Therefore, it's possible to download a package foo.tgz with provenance that is verified as foo.tgz but which ends up being installed as bar (and potentially overwriting any existing bar)?

This is correct. The package name is contained in the provenance (subject) and in the publish attestation (subject and other metadata).
For provenance, the builder is sort of oblivious to the package name. It may take it from the package.json, but ultimately the registry makes a decision to publish it under a package name or not. In npm, iiuc, the package name in the provenance need not match the package name in the package.json - at least that validation is not done today, as far as we know.. but is planned for future improvements.

So during verification (in particular of the publish attestation), should the verifier extract the package.json from the tarball to verify it's consistent with the provenance / publish attestation? Not doing so mean we must rely on the registry to do that verification - which today does not happen afaik.

If someone downloads the tarball and install from tarball, the package name used during installation is effectively the one in the package.json - which may be different from the one in publish / slsa attestations. (Note: for installation by package-name, npm CLI does not make use of the package.json in the tarball - @ianlewis to keep me honest).

Regardless of whether the registry does that verification, independent verifiers may want to verify provenance / publish attestation on their own, so as to keep the registry honest and detect problems; and to improve trust in this supply-chain metadata.

Extracting the package.json means the tarball need sot be available, so a verification-as-a-service would not work well, for example.

Hope this provides some clarification

[laurentsimon]

Well, a timely post https://blog.vlt.sh/blog/the-massive-hole-in-the-npm-ecosystem illustrating exactly what @ianlewis described

[ianlewis]

(Note: for installation by package-name, npm CLI does not make use of the package.json in the tarball - @ianlewis to keep me honest).

I believe that is correct. It's a bit inconsistent between installing by package name where it can get the metadata from the registry at the same time as it downloads the tarball, and installing by tarball on the local machine where all it has is the package.json inside the tarball.

[joshuagl]

I think in this instance it does make sense to verify package names, but I do not think we need to make that a stronger recommendation in general.

In SLSA we trust platforms, verify artifacts and as part of the verification we evaluate provenance against the expectations for a package. As the npm ecosystem doesn't (yet?) have an architecture for forming expectations, the slsa-verifier must determine how best to form expectations based on its view of the npm ecosystem.

Given the issue raised in this thread, it seems prudent that slsa-verifier's expectations for npm packages would include that the package name in the attestation matches the package name in the package's package.json.

[arewm]

Based on my reading of the npm issue presented above, this arises when there is no package lock present. The inconsistency arises when packages are installed from local caches vs. from npm directly.

If a future SLSA track were to try to address the best-practice of pinning dependencies, would we be able to bypass this issue? There might be a problem in generating the lock file due to the above use cases, but once the lockfile is generated, then that can be depended on by the build system.

[laurentsimon]

Based on my reading of the npm issue presented above, this arises when there is no package lock present.

I don't think that's the problem. You may have a lock file but the name of the package from the API != name in package.json

[arewm]

What is the actual problem that we are trying to resolve with the mismatch of package names between the API and the package.json? The ability to create an accurate provenance?

I was thinking from the perspective of a specific build platform. If the build platform requires a lock file to be used, then that lock file can be reused for pulling dependencies (any name consistencies should already be "resolved" and represented in the lockfile itself). If a lockfile isn't used and the build environment is potentially "dirty" (i.e. there are local caches that are used resulting in the aforementioned behavior), then you have to be concerned about the mismatch between package names since the two methods of collecting names can result in different results.

By respecting the lockfile, we respect what the package manager itself attempts to do. In the blog post, they claim that there are likely many non-malicious discrepancies in the wild already.

I am not trying to indicate that a mismatch between the package.json and the API isn't an issue. Instead, if a build/build platform have the ability to completely define the dependencies (i.e. greater than SLSA v1.0 Build L3 ... maybe a future L4 around what was once hermetic builds) then the name mismatch issue present above should be mitigated.

I am not convinced that it should be a role of a slsa-verification mechanism to patch potential issues with various languages' package managers. Being aware of this issue and being able to react to it are important when you are including additional dependencies, but as long as you have complete and accurate provenance and SBOM data, verification can happen based on the packages and versions which are actually included in an artifact. If there are malicious side effects of this behavior, then they can be appropriately identified. Detecting the presence of these mismatches can be inputs into a classification of potentially malicious packages which further investigation can then be leveraged to appropriately classify.

[laurentsimon]

What is the actual problem that we are trying to resolve with the mismatch of package names between the API and the package.json? The ability to create an accurate provenance?

"accurate" verification of provenance. If the publish attestation claims it's package A, installing the package should not end up installing a package under name B.

I was thinking from the perspective of a specific build platform. If the build platform requires a lock file to be used, then that lock file can be reused for pulling dependencies (any name consistencies should already be "resolved" and represented in the lockfile itself). If a lockfile isn't used and the build environment is potentially "dirty" (i.e. there are local caches that are used resulting in the aforementioned behavior), then you have to be concerned about the mismatch between package names since the two methods of collecting names can result in different results.

By respecting the lockfile, we respect what the package manager itself attempts to do.

That's the discrepancy. What is the "package manager" (the CLI or the registry or both)? npm registry will install package P under the name A if the user types npm install P but under package B if user types npm download P && npm install P.tar.gz. In effect, the registry's attestation is inconsistent with the package manager's metadata and the provenance. The resolution happens either at build time (no lock file) or it's pre-resolved (there's a lock file) but the inconsistency remains.

I am not convinced that it should be a role of a slsa-verification mechanism to patch potential issues with various languages' package managers.

That was our initial position, but we were not satisfied with the guarantees during verification, hence this issue. Thanks for sharing your opinion

Being aware of this issue and being able to react to it are important when you are including additional dependencies, but as long as you have complete and accurate provenance and SBOM data,

In this scenario, which package name would be reported in the SBOM? The sha512 will uniquely identify it, but the name may be inconsistent. I suppose it should report the package name from the registry... but if SLSA provenance was used, it would report possibly another package name. Same for a lock file: lock file is one resolution by the package manager (taken from the API or the package.json depending on user's command)

[arewm]

Apologies for the detour. I was coming at this from a perspective of a build platform which is only consuming npm packages and not one that is producing the packages (and therefore whose provenance and verification would be different). I realize that this was an inaccurate interpretation of the discussion.

After re-reading the artifact verification, this mismatch seems like it would fall well in check expectations. Therefore, when verifying the provenance of an npm package, the expectation that the package name is consistent should be checked.

The rationale for this would be to enable proper attribution of sources and clarity of the provenance subject. This would resolve the line of questions above around which name should be included.

The fact that a package's behavior can change when installed from the registry or from a tarball, however, would seem to fall outside of provenance verification. As long as provenance can be appropriately associated, the behavior of a package (i.e. for determining potential maliciousness) is extra-verification and might be more relevant to the build process when consuming the published npm artifact.

I am not convinced that it should be a role of a slsa-verification mechanism to patch potential issues with various languages' package managers

When worded in terms of expectations being formed around the packages, I no longer think that this previous argument holds. We wouldn't be trying to patch issues with the package ecosystems in general. The precedence that this decision would set is that if there are multiple ways that a name can be defined in some package ecosystem then all names should be consistent in order to disambiguate provenance references and associations.

[kpk47]

I agree with @arewm's conclusion as to what the precent should be:

if there are multiple ways that a name can be defined in some package ecosystem then all names should be consistent in order to disambiguate provenance references and associations.

I'm still a little confused about what we intend to do for npm.

Do we need to involve the publish attestation in this disambiguation? Doing so treats the npm registry as a definitive mapping from package digest to package name, and I don't know the npm ecosystem well enough to know if that's appropriate. Do people distribute npm packages by tarball without uploading them to the registry? If so, then we need a SLSA verification path for them.

Separately, I don't like the idea of the SLSA verifier having to open the tarball and inspect its contents, which seems to be the proposal here (if I'm following the thread correctly). Doing so seems to be developing expectations on the fly to work around a deficiency in the package's provenance.

The verifier already trusts the build platform to record build metadata faithfully in the provenance, so I don't see why we can't trust the build platform to record the package name in the subject's name (or URI) field. You can detect any changes to package.json because they would change the package's digest. Then the verifier needs to set an expectation on the package name, which they can verify against the provenance. Would this convention work, or am I missing something?