Draft: providers: add BellSoft OSV provider#924
Conversation
|
I tried but unfortunately failed to test this locally with https://github.com/i-bs/grype/commits/main/: Grype outputs empty results. Anybody can help me out with this? Thanks! |
|
File to test with: syft.json.gz |
|
anybody to look at this please? |
|
please?? |
Data sourceOSV Ecosystem "BellSoft SA" providing data for these Linux distros:
Inspected imagesLinux distros images: The images correctly parsed by Syft which determines the distro and packages included.
Current status and issuesVunnel patched (this PR). Grype patched (commit) Patched Vunnel pulls provider data(provider=bellsoft). But Grype cannot find this data to link it to the found packages:
Please help with finding the problem and fix it. Thanks. |
|
Hi @i-bs! Thanks for opening a PR! We added some new docs pretty recently that you might find helpful, especially https://oss.anchore.com/docs/contributing/vunnel/ There are a couple things to check here:
Lastly, if none of those are working, please feel free to ping again (make sure you @willmurphyscode so that your message stands out in the ~200 github notifications I get per day) and I'll take a look. Also, we have a community meeting every two weeks: calendar link and I have appointment slots of vulnerability provider office hours: https://calendar.app.google/ws24p3SPhKiL2vjC9 |
|
@willmurphyscode , thank you for looking into this. 1st I'm indeed using Vunnel Development Shell (found it myself in docs). Hence I'm quite sure Vunnel, Grype and Grype-DB are custom built with my patches: 2nd, Grype seems not see the relevant data: I am strongly unsure about how Grype binds the Provider (which is Can you please take another look? Thanks! |
|
Hi @i-bs !
I still think this is the problem. Can you confirm that your local build of grype-db links to your local build of grype? (Just having them next to each other won't do that, nor will just using the vunnel dev shell. Maybe that's something we should fix about the dev shell). I usually run Please let me know whether that helps! |
|
@willmurphyscode , Did you look into the Vunnel and Grype patches? Does those seem ok to you? Thanks! |
|
Hi @i-bs ! I think the issue is that the records in the grype database don't line up with your aliases. So grype-db is going to call This: is telling grype, "when you see alpaquita or bellsoft-hardened-containers in /etc/os-release or the distro param, look under "bellsoft" instead. So that's the mismatch, grype-db is putting either alpaquita or "bellsoft hardened containers" and grype is always searching under "bellsoft". Does that help? If not maybe we can meet sometime? https://github.com/anchore/grype-db/blob/075c3c2183976d045ef5209c96bbb78e85dd3b44/pkg/process/v6/transformers/osv/transform.go#L515 does some ecosystem name normalization, might be relevant. |
|
You can also set I'm also not sure you want to mark every version of alpaquita "rolling" - that basically tells grype that this distro doesn't confine searches by distro version. I think instead you want to mark |
|
@willmurphyscode , thanks a lot, but..
unfortunately gives no extra debug info (log below). I suggest it doesn't correctly merge config values because I see two Also thanks for pointing to the Details[0000] INFO grype version: [not provided]
[0000] DEBUG config:
�[35m log:
quiet: false
level: debug
file: ""
dev:
profile: none
output: []
file: ""
pretty: false
distro: ""
add-cpes-if-none: false
output-template-file: ""
check-for-app-update: false
only-fixed: false
only-notfixed: false
ignore-states: ""
platform: ""
search:
scope: squashed
unindexed-archives: false
indexed-archives: true
ignore: []
exclude: []
external-sources:
enable: false
maven:
search-upstream: true
base-url: https://search.maven.org/solrsearch/select
rate-limit: 300ms
match:
java:
using-cpes: false
jvm:
using-cpes: true
dotnet:
using-cpes: false
golang:
using-cpes: false
always-use-cpe-for-stdlib: true
allow-main-module-pseudo-version-comparison: false
javascript:
using-cpes: false
python:
using-cpes: false
ruby:
using-cpes: false
rust:
using-cpes: false
stock:
using-cpes: true
fail-on-severity: ""
registry:
insecure-skip-tls-verify: false
insecure-use-http: false
auth: []
ca-cert: ""
show-suppressed: false
by-cve: false
sort-by: risk
name: ""
default-image-pull-source: ""
from: []
vex-documents: []
vex-add: []
match-upstream-kernel-headers: false
fix-channel:
redhat-eus:
apply: auto
versions: '>= 8.0'
timestamp: true
db:
cache-dir: /usr/src/vunnel/.cache/grype
update-url: https://grype.anchore.io/databases
ca-cert: ""
auto-update: false
validate-by-hash-on-start: true
validate-age: false
max-allowed-built-age: 120h0m0s
require-update-check: false
update-available-timeout: 30s
update-download-timeout: 5m0s
max-update-check-frequency: 2h0m0s
exp: {}
dev:
db:
debug: true�[0m
[0000] DEBUG gathering packages
[0000] DEBUG loading DB
[0000] DEBUG skipping db update
[0000] DEBUG no CPEs for package: Pkg(name="busybox" version="1.37.0-r30" type="apk" id="d1148f6b9191ae04")
[0000] INFO using distro: alpaquita
[0000] INFO gathered packages packages=1 time=14.384273ms
[0000] INFO loaded DB status=valid time=19.394129ms
[0000] DEBUG ├── schema=v6.1.3
[0000] DEBUG ├── built=2026-01-24T12:31:46Z
[0000] DEBUG ├── from=manual import
[0000] DEBUG └── path=/usr/src/vunnel/.cache/grype/6/vulnerability.db
[0000] DEBUG no OS found in the DB for the given criteria os=alpaquita
[0000] DEBUG failed to find CPE matches for package error=attempted CPE match against package with no CPEs package=busybox
[0000] DEBUG no OS found in the DB for the given criteria os=alpaquita
[0000] DEBUG no OS found in the DB for the given criteria os=alpaquita
[0000] INFO found 0 vulnerability matches across 1 packages
[0000] DEBUG ├── fixed: 0
[0000] DEBUG ├── ignored: 0 (due to user-provided rule)
[0000] DEBUG ├── dropped: 0 (due to hard-coded correction)
[0000] DEBUG └── matched: 0
[0000] DEBUG ├── unknown: 0
[0000] DEBUG ├── negligible: 0
[0000] DEBUG ├── low: 0
[0000] DEBUG ├── medium: 0
[0000] DEBUG ├── high: 0
[0000] DEBUG └── critical: 0
[0000] INFO found vulnerability matches time=1.656074ms
No vulnerabilities found
BTW, |
|
Hi @i-bs ! Thanks for taking this so far. I am happy to help get it in, but I need a test image that is publicly pullable for our quality gate. Is there a place I can pull an image? What many providers have done if the images are normally paid is push an old tag of a small image to some other registry and made it publicly pullable. If you can provide a test image, I can try to get this provider in. (You're also welcome to keep working on it - we need a test image either way.) Please let me know:
In case you want to keep working on this, here's what I've learned: SQL only shows up at [0000] TRACE sql: SELECT * FROM `operating_systems` WHERE (name = "alpaquita" collate nocase OR release_id = "alpaquita" collate nocase) AND (channel IS NULL OR channel = '') duration=24.667µs rows=3
[0000] TRACE sql: SELECT `affected_package_handles`.`id`,`affected_package_handles`.`vulnerability_id`,`affected_package_handles`.`operating_system_id`,`affected_package_handles`.`package_id`,`affected_package_handles`.`blob_id` FROM `affected_package_handles` JOIN packages ON affected_package_handles.package_id = packages.id WHERE packages.name = "busybox" collate nocase AND affected_package_handles.operating_system_id IN (1,2,3) ORDER BY `affected_package_handles`.`id` LIMIT 300 duration=102.25µs rows=22When I run Which means Grype is dropping vulnerabilities because there are bad version constraints in the database. I think maybe the grype-db osv transformer doesn't know how handle a the combination of introduced and fixed correctly for the new provider. Looking a I see in the json: "ranges": [
{
"version": {
"type": "ecosystem",
"constraint": ">= 1.36.1-r1 < 1.37.0-r31"
},
"fix": {
"version": "1.37.0-r31",
"state": "fixed"
}
}
]which I don't think is the correct way of making one. I think this is a grype-db bug that pre-exists your changes in the OSV transformer. Let me know how you'd like to proceed and what test image we can use for this! Thanks for sticking with it so far! |
|
Dear @willmurphyscode , (links are the same as above). Your findings are very good. Thanks.
looks quite right from a human POV. But not from the Constraint Parser's. So if the OSV transformer needs to be fixed, maybe the missing peace is the APK (Linux package) version parser. Indeed the scanner needs to know how to compare versions which is generally distro-specific. One advantage here is that Alpaquita versions obey the same rules as Alpine Linux APKs and it probably can be reused. I'll keep looking deeper here. If you have something please share or add your commits. |
That's fine OSV, but I believe grype, which does not use OSV internally, is expecting a comma. The log line is Grype saying, "someone put I think the issue is that at https://github.com/anchore/grype-db/blob/main/pkg/process/v6/transformers/osv/transform.go#L302-L307 only semver and bitnami type versions are being normalized. |
|
The change like this one? |
i-bs
force-pushed
the
main
branch
2 times, most recently
from
94b56cf to
b3bfcd4
Compare
for Alpaquita Linux and BellSoft Hardened Containers Signed-off-by: Ildar Mulyukov <ildar.mulyukov@bell-sw.com>
I believe I fixed this fault in anchore/grype-db#869 . Please take a look, it is short. I also enabled my desired Ecosystems in that PR. With that Grype sees vuln. info for the requested pkg. But I suspect that the package version ranges are not determined correctly. Will check it tomorrow. Thanks. |
|
Hi @i-bs ! Thanks for your patience. I am working to get this in, but I have one question: Typically, other APK / Apline based distros we scan use Alpine SecDB, which has basically the semantics: use NVD CPEs to do the initial search, and then use entries from Alpine SecDB to rule out findings that are fixed in distro packages. Is that the semantics we have here? To ask another way, do you publish vulnerabilities in your OSV feed or only fixes? |
|
@willmurphyscode , thanks for the feedback. Let me clarify:
|
|
@i-bs sorry, I wasn't trying to ask if we should literally use Alpine SecDB for matching. But most Alpine based distros we've seen have some variant of SecDB semantics, that is, they expect us to match against NVD CPEs and then filter out fixes they have published. It sounds like for Bellsoft / Alpaquita we should just assume that the OSV feed is complete for APK packages on that distro. Is that right? |
|
@willmurphyscode , sorry for the delay. BellSoft OSV feed is complete regarding the package set, applicable CVEs as well as severity data and needs no additional source of information(*). Moreover the feed is limited only to the packages/CVEs relevant to BellSoft Linux ecosystem and doesn't contain any info relevant to Alpine/Windows/etc. So, yes, BellSoft OSV feed should be enough to process vulnerability data in BellSoft Alpaquita Linux / Hardened Containers. (*) The feed is almost complete but it has Summary and Description fields empty. See google/osv-scanner#2476 for details. |
|
@i-bs thanks! One additional question. What's the relationship between "bellsoft-hardened-containers" and "alpaquita linux"? Are there two sets of vulnerability data here or just one? |
|
While in theory the two products share the same artifacts, those are built slightly differently and should be treated differently. Internally the APK comparison rules are the same. I see that Grype "ecosystems" and OSV ecosystems are a little different. BellSoft ecosystem in OSV has two Linux distros: Alpaquita and Hardened Containers. In Grype source code it seems to be hard having both in one "ecosystem". Is that right? |
|
@i-bs I'm just trying to understand what the matching behavior should be; implementation difficulty in Grype is something I think about after I know what I'm trying to accomplish. Here are some very concrete questions that may help:
|
|
@willmurphyscode , thanks for working on this.
Sure. It is
OSV data contains vulnerability data for both of the distros in the same records. So normally no guessing here. The match may and may not be, we cannot tell a-priory. I.e. if a BellSoft record has entries for Alpaquita packages and does not have entries for Bellsoft Hardened Containers this means that the vulnerability is not applicable to Bellsoft Hardened Containers. I hope this clarifies a bit. Also reminding the live images links: |
|
@willmurphyscode , |
2 participants
for Alpaquita Linux and BellSoft Hardened Containers