Skip to content

Commit 37c6380

Browse files
authored
CVE schema 5.2 upgrade (#234)
* Package updates Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com> Update to CVE schema 5.2 Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com> Add VDB_IGNORE_OS env var Signed-off-by: prabhu <7842+prabhu@users.noreply.github.com> purl based searches was converting %40 to @ and therefore broken Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com> move vers compare to the db layer Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com> * package updates Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com> --------- Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
1 parent 19d4dc3 commit 37c6380

File tree

21 files changed

+2467
-1566
lines changed

21 files changed

+2467
-1566
lines changed

.github/workflows/pythonapp.yml

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ jobs:
1010
strategy:
1111
matrix:
1212
os: [ubuntu-latest, windows-latest, macos-15]
13-
python-version: ['3.10', '3.11', '3.12', '3.13']
13+
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
1414
fail-fast: false
1515
steps:
1616
- uses: actions/checkout@v4
@@ -50,14 +50,12 @@ jobs:
5050
run: |
5151
npm install -g @cyclonedx/cdxgen
5252
cdxgen -t python -o bom.json . -p --profile research
53-
uv sync --all-extras --dev
54-
uv run vdb --cache --only-osv
55-
uv run vdb --bom bom.json
5653
if: ${{ matrix.python-version == '3.13' && matrix.os == 'ubuntu-latest' }}
57-
env:
58-
VDB_TEMP_DIR: ${{ runner.temp }}/vdb-temp
5954
- name: CLI tests
6055
run: |
56+
uv sync --all-extras --dev
57+
uv run vdb --cache --only-osv
58+
uv run vdb --bom bom.json
6159
uv run vdb --search "pkg:maven/org.springframework/spring-core@6.0.13"
6260
uv run vdb --search "pkg:maven/org.hibernate.orm/hibernate-core@6.2.9.Final"
6361
uv run vdb --search "pkg:nuget/Microsoft.Data.SqlClient@5.0.1"
@@ -66,4 +64,9 @@ jobs:
6664
uv run vdb --search "pkg:npm/eslint-config-prettier@9.1.0"
6765
uv run vdb --search "pkg:npm/eslint-config-prettier@9.1.1"
6866
uv run vdb --search "pkg:npm/eslint-config-prettier@10.1.7"
67+
uv run vdb --search "escape.tech:graphql-armor-max-depth:2.4.0"
68+
uv run vdb --search "pkg:npm/%40escape.tech/graphql-armor-max-depth@2.4.0"
6969
if: ${{ matrix.python-version == '3.13' && matrix.os == 'ubuntu-latest' }}
70+
env:
71+
VDB_IGNORE_OS: true
72+
VDB_TEMP_DIR: ${{ runner.temp }}/vdb-temp

.github/workflows/pythonpublish.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ on:
88
- master
99
- release/*
1010
tags:
11-
- 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10
11+
- 'v*'
1212
workflow_dispatch:
1313

1414
jobs:
15-
deploy:
15+
vdb-pypi:
1616
runs-on: ubuntu-latest
1717
permissions:
1818
contents: write
@@ -61,6 +61,7 @@ jobs:
6161

6262
mcp-container:
6363
runs-on: ubuntu-latest
64+
needs: vdb-pypi
6465
permissions:
6566
contents: write
6667
packages: write

INTEGRATION.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ When used as a Python library, the only dependency is Python >= 3.10. When using
1111
The vulnerability database comprises two SQLite database files.
1212

1313
- data.index.vdb6 - A smaller index database optimized for quick purl or cpe string searches and vers-based range comparisons.
14-
- data.vdb6 - Full CVE source database containing normalized data in CVE 5.1 specification formation and purl prefix.
14+
- data.vdb6 - Full CVE source database containing normalized data in CVE 5.2 specification formation and purl prefix.
1515

1616
### cve_index schema
1717

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ A good vulnerability database must have the following properties:
1010
- Easy to [download](#download-pre-built-database-recommended), [integrate](./INTEGRATION.md), and use
1111
- Performance
1212

13-
Multiple upstream sources are used by vdb to improve accuracy and reduce false negatives. SQLite database containing data in CVE 5.1 schema format is precompiled and distributed as files via ghcr to simplify download. With automatic purl prefix generation even for git repos, searches on the database can be performed with purl, cpe, or even http git url string. Every row in the database uses an open specification such as CVE 5.0 or Package URL (purl and vers) thus preventing the possibility of vendor lock-in.
13+
Multiple upstream sources are used by vdb to improve accuracy and reduce false negatives. SQLite database containing data in CVE 5.2 schema format is precompiled and distributed as files via ghcr to simplify download. With automatic purl prefix generation even for git repos, searches on the database can be performed with purl, cpe, or even http git url string. Every row in the database uses an open specification such as CVE 5.2 or Package URL (purl and vers) thus preventing the possibility of vendor lock-in.
1414

1515
## Vulnerability Data sources
1616

@@ -40,7 +40,7 @@ Multiple upstream sources are used by vdb to improve accuracy and reduce false n
4040
## Installation
4141

4242
```shell
43-
pip install appthreat-vulnerability-db>=6.2.0
43+
pip install appthreat-vulnerability-db>=6.5.0
4444
```
4545

4646
To install vdb with optional dependencies such as `oras` use the `[oras]` or `[all]` dependency group.
@@ -82,7 +82,7 @@ Use any sqlite browser or cli tools to load and query the two databases.
8282

8383
<img src="./docs/index-vdb6.png" alt="index" width="400">
8484

85-
**data.vdb6** - Contains source data in CVE 5.1 format stored as a jsonb blob.
85+
**data.vdb6** - Contains source data in CVE 5.2 format stored as a jsonb blob.
8686

8787
<img src="./docs/vdb6.png" alt="database" width="400">
8888

@@ -92,7 +92,7 @@ Using [ORAS cli](https://oras.land/) might be slightly faster.
9292

9393
```
9494
export VDB_HOME=$HOME/vdb
95-
oras pull ghcr.io/appthreat/vdbxz:v6.4.x -o $VDB_HOME
95+
oras pull ghcr.io/appthreat/vdbxz:v6.5.x -o $VDB_HOME
9696
tar -xvf *.tar.xz
9797
rm *.tar.xz
9898
```

data/CVE_Record_Format.json

Lines changed: 138 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
{
22
"$schema": "http://json-schema.org/draft-07/schema#",
33
"type": "object",
4+
"$id": "https://cveproject.github.io/cve-schema/schema/CVE_Record_Format.json",
45
"title": "CVE JSON record format",
5-
"description": "cve-schema specifies the CVE JSON record format. This is the blueprint for a rich set of JSON data that can be submitted by CVE Numbering Authorities (CNAs) and Authorized Data Publishers (ADPs) to describe a CVE Record. Some examples of CVE Record data include CVE ID number, affected product(s), affected version(s), and public references. While those specific items are required when assigning a CVE, there are many other optional data in the schema that can be used to enrich CVE Records for community benefit. Learn more about the CVE program at [the official website](https://cve.mitre.org). This CVE JSON record format is defined using JSON Schema. Learn more about JSON Schema [here](https://json-schema.org/).",
6+
"description": "cve-schema specifies the CVE JSON record format. This is the blueprint for a rich set of JSON data that can be submitted by CVE Numbering Authorities (CNAs) and Authorized Data Publishers (ADPs) to describe a CVE Record. Some examples of CVE Record data include CVE ID number, affected product(s), affected version(s), and public references. While those specific items are required when assigning a CVE, there are many other optional data in the schema that can be used to enrich CVE Records for community benefit. Learn more about the CVE program at [the official website](https://www.cve.org). This CVE JSON record format is defined using JSON Schema. Learn more about JSON Schema [here](https://json-schema.org/).",
67
"definitions": {
78
"uriType": {
89
"description": "A universal resource identifier (URI), according to [RFC 3986](https://tools.ietf.org/html/rfc3986).",
@@ -51,8 +52,23 @@
5152
},
5253
"cveId": {
5354
"type": "string",
55+
"description":"The official CVE identifier contains the string 'CVE', followed by the year, followed by a 4 to 19 digit number. Note that the year-part of the identifier should indicate either the year the vulnerability was discovered, or the year the CVE ID is published in. CVE IDs must be unique.",
5456
"pattern": "^CVE-[0-9]{4}-[0-9]{4,19}$"
5557
},
58+
"cpe22and23": {
59+
"type": "string",
60+
"description":"Common Platform Enumeration (CPE) Name in either 2.2 or 2.3 format",
61+
"pattern": "([c][pP][eE]:/[AHOaho]?(:[A-Za-z0-9._\\-~%]*){0,6})|(cpe:2\\.3:[aho*\\-](:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[*\\-]))(:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){4})",
62+
"minLength": 1,
63+
"maxLength": 2048
64+
},
65+
"cpe23": {
66+
"type": "string",
67+
"description":"Common Platform Enumeration (CPE) Name in 2.3 format",
68+
"pattern": "(cpe:2\\.3:[aho*\\-](:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[*\\-]))(:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){4})",
69+
"minLength": 1,
70+
"maxLength": 2048
71+
},
5672
"orgId": {
5773
"description": "A UUID for an organization participating in the CVE program. This UUID can be used to lookup the organization record in the user registry service.",
5874
"$ref": "#/definitions/uuidType"
@@ -106,6 +122,7 @@
106122
]
107123
}
108124
],
125+
"additionalProperties": false,
109126
"properties": {
110127
"vendor": {
111128
"type": "string",
@@ -196,15 +213,12 @@
196213
},
197214
"cpes": {
198215
"type": "array",
199-
"description": "Affected products defined by CPE. This is an array of CPE values (vulnerable and not), we use an array so that we can make multiple statements about the same version and they are separate (if we used a JSON object we'd essentially be keying on the CPE name and they would have to overlap). Also, this allows things like cveDataVersion or cveDescription to be applied directly to the product entry. This also allows more complex statements such as \"Product X between versions 10.2 and 10.8\" to be put in a machine-readable format. As well since multiple statements can be used multiple branches of the same product can be defined here.",
216+
"description": "Affected products defined by CPE. This is an array of CPE values (vulnerable and not), we use an array so that we can make multiple statements about the same version and they are separate (if we used a JSON object we'd essentially be keying on the CPE name and they would have to overlap). Also, this allows things like cveDataVersion or cveDescription to be applied directly to the product entry. This also allows more complex statements such as \"Product X between versions 10.2 and 10.8\" to be put in a machine-readable format. As well since multiple statements can be used multiple branches of the same product can be defined here. NOTE: Consider using the newer cpeApplicability block for defining CPE data using the CPE Applicability Language which includes more options for defining CPE Names.",
200217
"uniqueItems": true,
201218
"items": {
202219
"title": "CPE Name",
203-
"type": "string",
204220
"description":"Common Platform Enumeration (CPE) Name in either 2.2 or 2.3 format",
205-
"pattern": "([c][pP][eE]:/[AHOaho]?(:[A-Za-z0-9._\\-~%]*){0,6})|(cpe:2\\.3:[aho*\\-](:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[*\\-]))(:(((\\?*|\\*?)([a-zA-Z0-9\\-._]|(\\\\[\\\\*?!\"#$%&'()+,/:;<=>@\\[\\]\\^`{|}~]))+(\\?*|\\*?))|[*\\-])){4})",
206-
"minLength": 1,
207-
"maxLength": 2048
221+
"$ref": "#/definitions/cpe22and23"
208222
}
209223
},
210224
"modules": {
@@ -272,7 +286,7 @@
272286
},
273287
"versions": {
274288
"type": "array",
275-
"description": "Set of product versions or version ranges related to the vulnerability. The versions satisfy the CNA Rules [8.1.2 requirement](https://cve.mitre.org/cve/cna/rules.html#section_8-1_cve_entry_information_requirements). Versions or defaultStatus may be omitted, but not both.",
289+
"description": "Set of product versions or version ranges related to the vulnerability. The versions help satisfy the CNA Rules [5.1.3 requirement](https://www.cve.org/ResourcesSupport/AllResources/CNARules#section_5-1_Required_CVE_Record_Content). Versions or defaultStatus may be omitted, but not both.",
276290
"minItems": 1,
277291
"uniqueItems": true,
278292
"items": {
@@ -350,6 +364,28 @@
350364
},
351365
"additionalProperties": false
352366
}
367+
},
368+
"packageURL": {
369+
"description": "A Package URL, a unified URL specification for identifying packages hosted by known package hosts. The Package URL MUST NOT include a version.",
370+
"$ref": "#/definitions/uriType",
371+
"examples": [
372+
"pkg:bitbucket/birkenfeld/pygments-main",
373+
"pkg:deb/debian/curl?arch=i386&distro=jessie",
374+
"pkg:docker/cassandra",
375+
"pkg:docker/customer/dockerimage?repository_url=gcr.io",
376+
"pkg:gem/jruby-launcher?platform=java",
377+
"pkg:gem/ruby-advisory-db-check",
378+
"pkg:github/package-url/purl-spec",
379+
"pkg:golang/google.golang.org/genproto#googleapis/api/annotations",
380+
"pkg:maven/org.apache.xmlgraphics/batik-anim?packaging=sources",
381+
"pkg:maven/org.apache.xmlgraphics/batik-anim?repository_url=repo.spring.io/release",
382+
"pkg:npm/%40angular/animation",
383+
"pkg:npm/foobar",
384+
"pkg:nuget/EnterpriseLibrary.Common",
385+
"pkg:pypi/django",
386+
"pkg:rpm/fedora/curl?arch=i386&distro=fedora-25",
387+
"pkg:rpm/opensuse/curl?arch=i386&distro=opensuse-tumbleweed"
388+
]
353389
}
354390
}
355391
},
@@ -364,7 +400,7 @@
364400
"description": "The version of the CVE schema used for validating this record. Used to support multiple versions of this format.",
365401
"type": "string",
366402
"pattern": "^5\\.(0|[1-9][0-9]*)(\\.(0|[1-9][0-9]*))?$",
367-
"default": "5.1.0"
403+
"default": "5.2.0"
368404
},
369405
"cveMetadataPublished": {
370406
"description": "This is meta data about the CVE ID such as the CVE ID, who requested it, who assigned it, when it was requested, the current state (PUBLISHED, REJECTED, etc.) and so on. These fields are controlled by the CVE Services.",
@@ -487,6 +523,87 @@
487523
"required": ["orgId"],
488524
"additionalProperties": false
489525
},
526+
"cpeApplicabilityElement": {
527+
"description": "Affected products defined using an implementation of the CPE Applicability Language, mostly copied/forked from the NIST NVD CVE API v2.0 schema (optional). An operator property allows AND or OR logic between CPEs or combinations of CPEs. The negate and vulnerable Boolean properties allow CPEs to be inverted and/or defined as vulnerable or not. Multiple version fields are provided for capturing ranges of products when defining a CPE Match String Range. NOTE: When defining a cpeApplicability block, it is recommended that it align with (as much as possible) the product data provided within the affected block.",
528+
"properties": {
529+
"operator": {
530+
"type": "string",
531+
"enum": [
532+
"AND",
533+
"OR"
534+
]
535+
},
536+
"negate": {
537+
"type": "boolean"
538+
},
539+
"nodes": {
540+
"type": "array",
541+
"items": {
542+
"$ref": "#/definitions/cpe_node"
543+
}
544+
}
545+
},
546+
"required": [
547+
"nodes"
548+
]
549+
},
550+
"cpe_node": {
551+
"description": "Defines a CPE configuration node in an applicability statement.",
552+
"properties": {
553+
"operator": {
554+
"type": "string",
555+
"enum": [
556+
"AND",
557+
"OR"
558+
]
559+
},
560+
"negate": {
561+
"type": "boolean"
562+
},
563+
"cpeMatch": {
564+
"type": "array",
565+
"items": {
566+
"$ref": "#/definitions/cpe_match"
567+
}
568+
}
569+
},
570+
"required": [
571+
"operator",
572+
"cpeMatch"
573+
]
574+
},
575+
"cpe_match": {
576+
"description": "CPE match string or range",
577+
"type": "object",
578+
"properties": {
579+
"vulnerable": {
580+
"type": "boolean"
581+
},
582+
"criteria": {
583+
"$ref": "#/definitions/cpe23"
584+
},
585+
"matchCriteriaId": {
586+
"$ref": "#/definitions/uuidType"
587+
},
588+
"versionStartExcluding": {
589+
"$ref": "#/definitions/version"
590+
},
591+
"versionStartIncluding": {
592+
"$ref": "#/definitions/version"
593+
},
594+
"versionEndExcluding": {
595+
"$ref": "#/definitions/version"
596+
},
597+
"versionEndIncluding": {
598+
"$ref": "#/definitions/version"
599+
}
600+
},
601+
"required": [
602+
"vulnerable",
603+
"criteria"
604+
],
605+
"additionalProperties": false
606+
},
490607
"cnaPublishedContainer": {
491608
"description": "An object containing the vulnerability information provided by a CVE Numbering Authority (CNA) for a published CVE ID. There can only be one CNA container per CVE record since there can only be one assigning CNA. The CNA container must include the required information defined in the CVE Rules, which includes a product, version, problem type, prose description, and a reference.",
492609
"type": "object",
@@ -514,6 +631,12 @@
514631
"affected": {
515632
"$ref": "#/definitions/affected"
516633
},
634+
"cpeApplicability": {
635+
"type": "array",
636+
"items": {
637+
"$ref": "#/definitions/cpeApplicabilityElement"
638+
}
639+
},
517640
"problemTypes": {
518641
"$ref": "#/definitions/problemTypes"
519642
},
@@ -620,6 +743,12 @@
620743
"affected": {
621744
"$ref": "#/definitions/affected"
622745
},
746+
"cpeApplicability": {
747+
"type": "array",
748+
"items": {
749+
"$ref": "#/definitions/cpeApplicabilityElement"
750+
}
751+
},
623752
"problemTypes": {
624753
"$ref": "#/definitions/problemTypes"
625754
},
@@ -1231,4 +1360,4 @@
12311360
"additionalProperties": false
12321361
}
12331362
]
1234-
}
1363+
}

packages/mcp-server-vdb/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,12 @@ LABEL maintainer="Team AppThreat" \
2727
org.opencontainers.image.authors="Team AppThreat <cloud@appthreat.com>" \
2828
org.opencontainers.image.source="https://github.com/AppThreat/vulnerability-db" \
2929
org.opencontainers.image.url="https://github.com/AppThreat/vulnerability-db" \
30-
org.opencontainers.image.version="1.0.x" \
30+
org.opencontainers.image.version="1.1.x" \
3131
org.opencontainers.image.vendor="appthreat" \
3232
org.opencontainers.image.licenses="MIT" \
3333
org.opencontainers.image.title="vulnerability-db" \
3434
org.opencontainers.image.description="MCP server for AppThreat's vulnerability database and package search library." \
35-
org.opencontainers.docker.cmd="docker run -i --rm -e VDB_HOME=/db -v $HOME/db:/db:rw ghcr.io/appthreat/mcp-server-vdb:master"
35+
org.opencontainers.docker.cmd="docker run -i --rm -e VDB_HOME=/db -v $HOME/db:/db:rw ghcr.io/appthreat/mcp-server-vdb"
3636

3737
WORKDIR /app
3838

0 commit comments

Comments
 (0)