Skip to content

feat(suse): add cvrf CVE feed#423

Open
manojkrishna-nomula wants to merge 1 commit into
aquasecurity:mainfrom
manojkrishnanomula:feat/suse-cvrf-cve-feed
Open

feat(suse): add cvrf CVE feed#423
manojkrishna-nomula wants to merge 1 commit into
aquasecurity:mainfrom
manojkrishnanomula:feat/suse-cvrf-cve-feed

Conversation

@manojkrishna-nomula

@manojkrishna-nomula manojkrishna-nomula commented Mar 26, 2026

Copy link
Copy Markdown

The cvrf feed which contains the advisories are missing the scores of the cve's data related to the advisory, so to fetch the cve data we're using cvrf-cve feed provided by the suse to get the cve details.

The new directory is 221MB

The regular SUSE CVRF advisory feed at http://ftp.suse.com/pub/projects/security/cvrf/ (consumed by the existing suse-cvrf target) does not publish CVSS v3 / v4 scores — only the v2 vector, where present. SUSE exposes the v3/v4 scores in a separate per-CVE CVRF feed at http://ftp.suse.com/pub/projects/security/cvrf-cve/, and this PR adds a new suse-cvrf-cve target that mirrors those documents into cvrf/suse-cves/<year>/CVE-<id>.json so downstream consumers (e.g. trivy-db) can look up the scores.

  • New suse/cvrfarchive package that encapsulates the download / decompress / tar-walk logic. Both feeds (suse/cvrf and suse/cvrfcve) now share this code; the duplicated archive plumbing in suse/cvrf/cvrf.go has been removed.
  • New suse/cvrfcve package that registers as suse-cvrf-cve (directory renamed from suse/cvrf-cve to drop the hyphen, matching Go package conventions).
  • The persisted schema is trimmed to the fields needed for CVSS lookup: Title, Tracking.{ID,InitialReleaseDate,CurrentReleaseDate}, and per-vulnerability CVE / Threats / References / CVSSScoreSets. The bulky ProductTree, ProductStatuses, DocumentNotes and RevisionHistory blocks are dropped, which keeps the persisted JSON tree small (the upstream archive is ~250 MB compressed / ~10 GB uncompressed; trimming brings the resulting cvrf/suse-cves/ tree down by roughly two orders of magnitude).
  • Single archive download. Fetching the per-CVE files individually (≈80k requests at the previous 5/s budget) made the GitHub-hosted job exceed the 6h limit. The new code downloads cvrf-cve.tar.bz2 once and stream-extracts it, mirroring the fix applied to suse-cvrf in fix(suse): download CVRF archive instead of individual files #437.
  • main.go registers the new target and README.md documents it. update.yml gains a single SUSE CVE CVRF step.

@CLAassistant

CLAassistant commented Apr 9, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread suse/cvrf/types.go Outdated
References []Reference `xml:"References>Reference"`
ProductStatuses []Status `xml:"ProductStatuses>Status"`
CVSSScoreSets ScoreSet `xml:"CVSSScoreSets>ScoreSet" json:",omitempty"`
CVSSScoreSets []ScoreSet `xml:"CVSSScoreSets>ScoreSet" json:",omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed here — #401 (comment) — we cannot simply change the variable type.
Therefore #401 is a blocking PR.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DmitriyLewen yes, we agree this will fail if we don't use the latest commit, but we've got a workaround for it. We're good for merging this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to split the PRs (so that the changes for this field are made in #401) — could you describe in more detail in that PR (#401) what workaround you have for this change?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At premium side, we're not using the cvss field at all, so for our previous branches we want to copy the old tracker code and run it as premium tracker to support backward compatibility, but in later branches we'll consume the cvss scores.

Comment thread suse/cvrf/cve_cvrf.go Outdated
}
if sets, ok := cache[cveID]; ok {
if len(sets) > 0 {
cv.Vulnerabilities[i].CVSSScoreSets = sets

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You overwrite Scores, but should we merge scores from both sources?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get scores from the advisory feed, we'll capture only from the cve-cvrf feed.

Comment thread suse/cvrf/cve_cvrf.go Outdated
return scoreSetsFromCVE12(doc.Vuln.CVSSScoreSets), nil
}

func (c Config) mergeCVEDetailsFromCVEFeed(cv *Cvrf, cache map[string][]ScoreSet) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually for vuln-list-update we try not to modify the raw data from sources.
This is important so that users can view the original files, their history, etc.
After merging, it would be difficult to trace the source of a CVSS Score.

Perhaps we could save the CVSS files from http://ftp.suse.com/pub/projects/security/cvrf-cve/ separately and merge them in trivy-db instead?

@manojkrishnanomula manojkrishnanomula force-pushed the feat/suse-cvrf-cve-feed branch 3 times, most recently from 5785017 to a6bdcde Compare April 20, 2026 05:25
@manojkrishnanomula manojkrishnanomula force-pushed the feat/suse-cvrf-cve-feed branch 4 times, most recently from 01e7a62 to 321e998 Compare May 11, 2026 05:35

@DmitriyLewen DmitriyLewen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @manojkrishna-nomula
Thanks!
left a comments.

Can you also update PR description - write why we need to add cvrf CVE feed.

Comment thread suse/cvrf-cve/cvrf_cve_test.go Outdated
name: "positive test",
appFs: afero.NewMemMapFs(),
archiveDir: "testdata/cvrf-cve",
expectedFile: "/tmp/cvrf/suse-cves/2014/CVE-2014-6271.json",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you keep and compare golden files (like for cvrf tests)?

Comment thread main.go
var (
target = flag.String("target", "", "update target (nvd, alpine, alpine-unfixed, redhat, redhat-oval, "+
"redhat-csaf-vex, debian, ubuntu, amazon, oracle-oval, suse-cvrf, photon, arch-linux, glad, cwe, osvdev, mariner, kevc, wolfi, "+
"redhat-csaf-vex, debian, ubuntu, amazon, oracle-oval, suse-cvrf, suse-cvrf-cve, photon, arch-linux, glad, cwe, osvdev, mariner, kevc, wolfi, "+

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add suse-cvrf-cve into README.md?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread suse/cvrf-cve/types.go Outdated
Comment on lines +33 to +38
CVE string `xml:"CVE"`
Description string `xml:"Notes>Note"`
Threats []Threat `xml:"Threats>Threat"`
References []Reference `xml:"References>Reference"`
ProductStatuses []Status `xml:"ProductStatuses>Status"`
CVSSScoreSets CVSSScoreSets `xml:"CVSSScoreSets" json:",omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC we need to keep CVE advisories to see all CVSS scores (cvrf doesn't have V3 and V4 scores).
Do we need other fields? (we can reduce the size of the files to parse only the needed fields)

Comment thread suse/cvrf-cve/cvrf_cve.go Outdated
}
}

func (c Config) Update() error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can unify the logic.
Something like this:

diff --git a/main.go b/main.go
index 05102f5..3f372e9 100644
--- a/main.go
+++ b/main.go
@@ -32,7 +32,7 @@ import (
 	"github.com/aquasecurity/vuln-list-update/rootio"
 	"github.com/aquasecurity/vuln-list-update/seal"
 	susecvrf "github.com/aquasecurity/vuln-list-update/suse/cvrf"
-	susecvrfcve "github.com/aquasecurity/vuln-list-update/suse/cvrf-cve"
+	susecvrfcve "github.com/aquasecurity/vuln-list-update/suse/cvrfcve"
 	"github.com/aquasecurity/vuln-list-update/ubuntu"
 	"github.com/aquasecurity/vuln-list-update/utils"
 	"github.com/aquasecurity/vuln-list-update/wolfi"
diff --git a/suse/cvrf-cve/cvrf_cve.go b/suse/cvrf-cve/cvrf_cve.go
deleted file mode 100644
index dc51890..0000000
--- a/suse/cvrf-cve/cvrf_cve.go
+++ /dev/null
@@ -1,150 +0,0 @@
-package cvrfcve
-
-import (
-	"archive/tar"
-	"bytes"
-	"compress/bzip2"
-	"compress/gzip"
-	"encoding/xml"
-	"errors"
-	"fmt"
-	"io"
-	"log"
-	"path/filepath"
-	"regexp"
-	"strings"
-	"unicode/utf8"
-
-	"github.com/spf13/afero"
-	"golang.org/x/xerrors"
-
-	"github.com/aquasecurity/vuln-list-update/utils"
-)
-
-const (
-	cvrfCVEArchiveURL = "http://ftp.suse.com/pub/projects/security/cvrf-cve.tar.bz2"
-	cvrfDir           = "cvrf"
-	suseCVEDir        = "suse-cves"
-	retries           = 5
-)
-
-var fileRegexp = regexp.MustCompile(`^cvrf-(CVE-\d{4}-\d+)\.xml$`)
-
-type Config struct {
-	VulnListDir string
-	URL         string
-	AppFs       afero.Fs
-}
-
-func NewConfig() Config {
-	return Config{
-		VulnListDir: utils.VulnListDir(),
-		URL:         cvrfCVEArchiveURL,
-		AppFs:       afero.NewOsFs(),
-	}
-}
-
-func (c Config) Update() error {
-	log.Print("Fetching SUSE CVE CVRF archive...")
-
-	// The SUSE server is sometimes unstable, so download the whole archive into
-	// memory before processing. Streaming directly from the HTTP response would
-	// make it hard to distinguish a mid-transfer disconnection (which surfaces
-	// as a truncated tar) from a legitimate parse error. The archive is only a
-	// few hundred MB, which fits comfortably in memory on CI runners.
-	body, err := utils.FetchURL(c.URL, "", retries)
-	if err != nil {
-		return xerrors.Errorf("failed to download SUSE CVE CVRF archive: %w", err)
-	}
-
-	var decompressed io.Reader
-	switch {
-	case strings.HasSuffix(c.URL, ".tar.bz2"):
-		decompressed = bzip2.NewReader(bytes.NewReader(body))
-	case strings.HasSuffix(c.URL, ".tar.gz"):
-		// Go's compress/bzip2 lacks a Writer, so tests use .tar.gz instead.
-		gr, err := gzip.NewReader(bytes.NewReader(body))
-		if err != nil {
-			return xerrors.Errorf("failed to decompress gzip: %w", err)
-		}
-		defer gr.Close()
-		decompressed = gr
-	default:
-		return xerrors.Errorf("unsupported archive format: %s", c.URL)
-	}
-	tr := tar.NewReader(decompressed)
-
-	for {
-		hdr, err := tr.Next()
-		switch {
-		case errors.Is(err, io.EOF):
-			return nil
-		case err != nil:
-			return xerrors.Errorf("failed to read tar entry: %w", err)
-		case hdr.Typeflag != tar.TypeReg:
-			continue
-		}
-
-		filename := filepath.Base(hdr.Name)
-		if !strings.HasSuffix(filename, ".xml") {
-			continue
-		}
-		if fileRegexp.FindStringSubmatch(filename) == nil {
-			continue
-		}
-
-		data, err := io.ReadAll(tr)
-		if err != nil {
-			return xerrors.Errorf("failed to read tar entry data: %w", err)
-		}
-
-		if len(data) == 0 {
-			log.Printf("empty CVE CVRF xml: %s", filename)
-			continue
-		}
-
-		if !utf8.Valid(data) {
-			log.Printf("invalid UTF-8: %s", filename)
-			data = []byte(strings.ToValidUTF8(string(data), ""))
-		}
-
-		var cv Cvrf
-		if err = xml.Unmarshal(data, &cv); err != nil {
-			return xerrors.Errorf("failed to decode SUSE CVE CVRF XML (%s): %w", filename, err)
-		}
-
-		cveID := extractCVEID(cv)
-		if cveID == "" {
-			log.Printf("invalid CVE CVRF document ID: %s", cv.Tracking.ID)
-			continue
-		}
-
-		if err = c.saveCVEPerYear(cveID, cv); err != nil {
-			return xerrors.Errorf("failed to save SUSE CVE CVRF: %w", err)
-		}
-	}
-}
-
-func extractCVEID(cv Cvrf) string {
-	for _, v := range cv.Vulnerabilities {
-		cve := strings.TrimSpace(v.CVE)
-		if cve != "" {
-			return cve
-		}
-	}
-	return strings.TrimSpace(cv.Title)
-}
-
-func (c Config) saveCVEPerYear(cveID string, data Cvrf) error {
-	s := strings.Split(cveID, "-")
-	if len(s) < 3 {
-		return nil
-	}
-
-	yearDir := filepath.Join(c.VulnListDir, cvrfDir, suseCVEDir, s[1])
-	fileName := fmt.Sprintf("%s.json", cveID)
-	if err := utils.WriteJSON(c.AppFs, yearDir, fileName, data); err != nil {
-		return xerrors.Errorf("failed to write file: %w", err)
-	}
-	return nil
-}
diff --git a/suse/cvrf/cvrf.go b/suse/cvrf/cvrf.go
index 16e9325..87062cd 100644
--- a/suse/cvrf/cvrf.go
+++ b/suse/cvrf/cvrf.go
@@ -1,23 +1,17 @@
 package cvrf
 
 import (
-	"archive/tar"
-	"bytes"
-	"compress/bzip2"
-	"compress/gzip"
 	"encoding/xml"
-	"errors"
 	"fmt"
-	"io"
 	"log"
 	"path/filepath"
 	"regexp"
 	"strings"
-	"unicode/utf8"
 
 	"github.com/spf13/afero"
 	"golang.org/x/xerrors"
 
+	"github.com/aquasecurity/vuln-list-update/suse/cvrfarchive"
 	"github.com/aquasecurity/vuln-list-update/utils"
 )
 
@@ -47,81 +41,21 @@ func NewConfig() Config {
 func (c Config) Update() error {
 	log.Print("Fetching SUSE CVRF archive...")
 
-	// The SUSE server is sometimes unstable, so download the whole archive into
-	// memory before processing. Streaming directly from the HTTP response would
-	// make it hard to distinguish a mid-transfer disconnection (which surfaces
-	// as a truncated tar) from a legitimate parse error. The archive is only a
-	// few hundred MB, which fits comfortably in memory on CI runners.
-	body, err := utils.FetchURL(c.URL, "", retries)
-	if err != nil {
-		return xerrors.Errorf("failed to download CVRF archive: %w", err)
-	}
-
-	var decompressed io.Reader
-	switch {
-	case strings.HasSuffix(c.URL, ".tar.bz2"):
-		// The upstream archive is .tar.bz2, which is the only format used in production.
-		decompressed = bzip2.NewReader(bytes.NewReader(body))
-	case strings.HasSuffix(c.URL, ".tar.gz"):
-		// Go's compress/bzip2 lacks a Writer, so tests use .tar.gz instead.
-		gr, err := gzip.NewReader(bytes.NewReader(body))
-		if err != nil {
-			return xerrors.Errorf("failed to decompress gzip: %w", err)
-		}
-		defer gr.Close()
-		decompressed = gr
-	default:
-		return xerrors.Errorf("unsupported archive format: %s", c.URL)
-	}
-	tr := tar.NewReader(decompressed)
-
-	for {
-		hdr, err := tr.Next()
-		switch {
-		case errors.Is(err, io.EOF):
-			return nil
-		case err != nil:
-			return xerrors.Errorf("failed to read tar entry: %w", err)
-		case hdr.Typeflag != tar.TypeReg:
-			continue
-		}
-
-		filename := filepath.Base(hdr.Name)
-		// archive contains non-XML files (e.g. LICENSE), so skip them
-		if !strings.HasSuffix(filename, ".xml") {
-			continue
-		}
-		match := fileRegexp.FindStringSubmatch(filename)
-		if match == nil {
-			continue
-		}
+	return cvrfarchive.Walk(c.URL, retries, fileRegexp, func(e cvrfarchive.Entry) error {
+		match := fileRegexp.FindStringSubmatch(e.Filename)
 		osName := match[1]
 
-		data, err := io.ReadAll(tr)
-		if err != nil {
-			return xerrors.Errorf("failed to read tar entry data: %w", err)
-		}
-
-		if len(data) == 0 {
-			log.Printf("empty CVRF xml: %s", filename)
-			continue
-		}
-
-		if !utf8.Valid(data) {
-			log.Printf("invalid UTF-8: %s", filename)
-			data = []byte(strings.ToValidUTF8(string(data), ""))
-		}
-
 		var cv Cvrf
-		if err = xml.Unmarshal(data, &cv); err != nil {
-			return xerrors.Errorf("failed to decode SUSE XML (%s): %w", filename, err)
+		if err := xml.Unmarshal(e.Data, &cv); err != nil {
+			return xerrors.Errorf("failed to decode SUSE XML (%s): %w", e.Filename, err)
 		}
 
 		dir := filepath.Join(cvrfDir, suseDir, osName)
-		if err = c.saveCvrfPerYear(dir, cv.Tracking.ID, cv); err != nil {
+		if err := c.saveCvrfPerYear(dir, cv.Tracking.ID, cv); err != nil {
 			return xerrors.Errorf("failed to save CVRF: %w", err)
 		}
-	}
+		return nil
+	})
 }
 
 func (c Config) saveCvrfPerYear(dirName string, cvrfID string, data Cvrf) error {
diff --git a/suse/cvrfarchive/archive.go b/suse/cvrfarchive/archive.go
new file mode 100644
index 0000000..deb4758
--- /dev/null
+++ b/suse/cvrfarchive/archive.go
@@ -0,0 +1,103 @@
+// Package cvrfarchive walks SUSE CVRF tar archives published at
+// http://ftp.suse.com/pub/projects/security/. It hides the
+// download/decompress/tar plumbing so feed-specific code only needs
+// to handle XML decoding and persistence.
+package cvrfarchive
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/bzip2"
+	"compress/gzip"
+	"errors"
+	"io"
+	"log"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"unicode/utf8"
+
+	"golang.org/x/xerrors"
+
+	"github.com/aquasecurity/vuln-list-update/utils"
+)
+
+// Entry is a single XML document extracted from a SUSE CVRF archive.
+// Data is guaranteed to be valid UTF-8 (invalid bytes are stripped).
+type Entry struct {
+	Filename string
+	Data     []byte
+}
+
+// Walk downloads a SUSE CVRF archive from url, decompresses it
+// (bzip2 or gzip, detected by the URL suffix) and invokes handler
+// for every .xml entry whose base name matches nameRegexp.
+// Non-regular tar entries, empty files and non-XML files are skipped;
+// invalid UTF-8 byte sequences are stripped from the data.
+func Walk(url string, retries int, nameRegexp *regexp.Regexp, handler func(Entry) error) error {
+	// The SUSE server is sometimes unstable, so download the whole archive into
+	// memory before processing. Streaming directly from the HTTP response would
+	// make it hard to distinguish a mid-transfer disconnection (which surfaces
+	// as a truncated tar) from a legitimate parse error. The archive is only a
+	// few hundred MB, which fits comfortably in memory on CI runners.
+	body, err := utils.FetchURL(url, "", retries)
+	if err != nil {
+		return xerrors.Errorf("failed to download archive: %w", err)
+	}
+
+	decompressed, err := decompress(url, body)
+	if err != nil {
+		return err
+	}
+
+	tr := tar.NewReader(decompressed)
+	for {
+		hdr, err := tr.Next()
+		switch {
+		case errors.Is(err, io.EOF):
+			return nil
+		case err != nil:
+			return xerrors.Errorf("failed to read tar entry: %w", err)
+		case hdr.Typeflag != tar.TypeReg:
+			continue
+		}
+
+		filename := filepath.Base(hdr.Name)
+		if !strings.HasSuffix(filename, ".xml") {
+			continue
+		}
+		if nameRegexp != nil && !nameRegexp.MatchString(filename) {
+			continue
+		}
+
+		data, err := io.ReadAll(tr)
+		if err != nil {
+			return xerrors.Errorf("failed to read tar entry data: %w", err)
+		}
+		if len(data) == 0 {
+			log.Printf("empty xml: %s", filename)
+			continue
+		}
+		if !utf8.Valid(data) {
+			log.Printf("invalid UTF-8: %s", filename)
+			data = []byte(strings.ToValidUTF8(string(data), ""))
+		}
+
+		if err := handler(Entry{Filename: filename, Data: data}); err != nil {
+			return err
+		}
+	}
+}
+
+func decompress(url string, body []byte) (io.Reader, error) {
+	switch {
+	case strings.HasSuffix(url, ".tar.bz2"):
+		// The upstream archive is .tar.bz2, which is the only format used in production.
+		return bzip2.NewReader(bytes.NewReader(body)), nil
+	case strings.HasSuffix(url, ".tar.gz"):
+		// Go's compress/bzip2 lacks a Writer, so tests use .tar.gz instead.
+		return gzip.NewReader(bytes.NewReader(body))
+	default:
+		return nil, xerrors.Errorf("unsupported archive format: %s", url)
+	}
+}
diff --git a/suse/cvrfcve/cvrfcve.go b/suse/cvrfcve/cvrfcve.go
new file mode 100644
index 0000000..367100d
--- /dev/null
+++ b/suse/cvrfcve/cvrfcve.go
@@ -0,0 +1,65 @@
+package cvrfcve
+
+import (
+	"encoding/xml"
+	"fmt"
+	"log"
+	"path/filepath"
+	"regexp"
+	"strings"
+
+	"github.com/spf13/afero"
+	"golang.org/x/xerrors"
+
+	"github.com/aquasecurity/vuln-list-update/suse/cvrfarchive"
+	"github.com/aquasecurity/vuln-list-update/utils"
+)
+
+const (
+	cvrfCVEArchiveURL = "http://ftp.suse.com/pub/projects/security/cvrf-cve.tar.bz2"
+	cvrfDir           = "cvrf"
+	suseCVEDir        = "suse-cves"
+	retries           = 5
+)
+
+var fileRegexp = regexp.MustCompile(`^cvrf-(CVE-\d{4}-\d+)\.xml$`)
+
+type Config struct {
+	VulnListDir string
+	URL         string
+	AppFs       afero.Fs
+}
+
+func NewConfig() Config {
+	return Config{
+		VulnListDir: utils.VulnListDir(),
+		URL:         cvrfCVEArchiveURL,
+		AppFs:       afero.NewOsFs(),
+	}
+}
+
+func (c Config) Update() error {
+	log.Print("Fetching SUSE CVE CVRF archive...")
+
+	return cvrfarchive.Walk(c.URL, retries, fileRegexp, func(e cvrfarchive.Entry) error {
+		// ID is taken from the file name, which is already validated by fileRegexp.
+		cveID := fileRegexp.FindStringSubmatch(e.Filename)[1]
+
+		var cv Cvrf
+		if err := xml.Unmarshal(e.Data, &cv); err != nil {
+			return xerrors.Errorf("failed to decode SUSE CVE CVRF XML (%s): %w", e.Filename, err)
+		}
+
+		return c.saveCVEPerYear(cveID, cv)
+	})
+}
+
+func (c Config) saveCVEPerYear(cveID string, data Cvrf) error {
+	year := strings.Split(cveID, "-")[1]
+	yearDir := filepath.Join(c.VulnListDir, cvrfDir, suseCVEDir, year)
+	fileName := fmt.Sprintf("%s.json", cveID)
+	if err := utils.WriteJSON(c.AppFs, yearDir, fileName, data); err != nil {
+		return xerrors.Errorf("failed to write file: %w", err)
+	}
+	return nil
+}
diff --git a/suse/cvrf-cve/cvrf_cve_test.go b/suse/cvrfcve/cvrfcve_test.go
similarity index 93%
rename from suse/cvrf-cve/cvrf_cve_test.go
rename to suse/cvrfcve/cvrfcve_test.go
index cfdac90..56ba325 100644
--- a/suse/cvrf-cve/cvrf_cve_test.go
+++ b/suse/cvrfcve/cvrfcve_test.go
@@ -13,7 +13,7 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
-	cvrfcve "github.com/aquasecurity/vuln-list-update/suse/cvrf-cve"
+	"github.com/aquasecurity/vuln-list-update/suse/cvrfcve"
 )
 
 // createArchive creates a tar.gz archive from the given directory.
@@ -44,13 +44,13 @@ func TestConfig_Update(t *testing.T) {
 		{
 			name:         "positive test",
 			appFs:        afero.NewMemMapFs(),
-			archiveDir:   "testdata/cvrf-cve",
+			archiveDir:   "testdata/cvrfcve",
 			expectedFile: "/tmp/cvrf/suse-cves/2014/CVE-2014-6271.json",
 		},
 		{
 			name:             "broken XML",
 			appFs:            afero.NewMemMapFs(),
-			archiveDir:       "testdata/broken-cvrf-cve",
+			archiveDir:       "testdata/broken-cvrfcve",
 			expectedErrorMsg: "failed to decode SUSE CVE CVRF XML",
 		},
 	}
diff --git a/suse/cvrf-cve/testdata/broken-cvrf-cve/cvrf-CVE-2014-6271.xml b/suse/cvrfcve/testdata/broken-cvrfcve/cvrf-CVE-2014-6271.xml
similarity index 100%
rename from suse/cvrf-cve/testdata/broken-cvrf-cve/cvrf-CVE-2014-6271.xml
rename to suse/cvrfcve/testdata/broken-cvrfcve/cvrf-CVE-2014-6271.xml
diff --git a/suse/cvrf-cve/testdata/cvrf-cve/cvrf-CVE-1234-12345.xml b/suse/cvrfcve/testdata/cvrfcve/cvrf-CVE-1234-12345.xml
similarity index 100%
rename from suse/cvrf-cve/testdata/cvrf-cve/cvrf-CVE-1234-12345.xml
rename to suse/cvrfcve/testdata/cvrfcve/cvrf-CVE-1234-12345.xml
diff --git a/suse/cvrf-cve/testdata/cvrf-cve/cvrf-CVE-2014-6271.xml b/suse/cvrfcve/testdata/cvrfcve/cvrf-CVE-2014-6271.xml
similarity index 100%
rename from suse/cvrf-cve/testdata/cvrf-cve/cvrf-CVE-2014-6271.xml
rename to suse/cvrfcve/testdata/cvrfcve/cvrf-CVE-2014-6271.xml
diff --git a/suse/cvrf-cve/types.go b/suse/cvrfcve/types.go
similarity index 100%
rename from suse/cvrf-cve/types.go
rename to suse/cvrfcve/types.go

@manojkrishnanomula manojkrishnanomula force-pushed the feat/suse-cvrf-cve-feed branch 2 times, most recently from c8b6644 to 4bb48fa Compare May 13, 2026 09:12
Comment thread suse/cvrfcve/types.go Outdated
Comment on lines +7 to +9
// resulting JSON tree stays small (the upstream archive is ~250 MB
// compressed / ~10 GB uncompressed; trimming brings the persisted tree
// down by two orders of magnitude).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
I think we don't need to write upstream sizes to avoid confusion (also the size will increase).

Suggested change
// resulting JSON tree stays small (the upstream archive is ~250 MB
// compressed / ~10 GB uncompressed; trimming brings the persisted tree
// down by two orders of magnitude).
// resulting JSON tree stays small.

@DmitriyLewen

DmitriyLewen commented May 13, 2026

Copy link
Copy Markdown
Contributor

PR description:

The new directory is 1.5GB

221 MB after removing some fields:

➜  cvrf du -sh ./suse-cves/ 
221M	./suse-cves/

@DmitriyLewen DmitriyLewen changed the title Feat/suse cvrf CVE feed feat(suse): add cvrf CVE feed May 13, 2026
@manojkrishnanomula manojkrishnanomula force-pushed the feat/suse-cvrf-cve-feed branch from 4bb48fa to 0c165b2 Compare May 13, 2026 09:38
Add a new `suse-cvrf-cve` target that fetches the per-CVE CVRF documents
SUSE publishes at /pub/projects/security/cvrf-cve/ and writes them to
cvrf/suse-cves/<year>/CVE-<id>.json. The regular SUSE CVRF advisory feed
omits CVSS v3/v4 scores; the per-CVE feed is the only source that
exposes them, so we persist these documents to make the scores
available to downstream consumers (e.g. trivy-db).

To keep run time reasonable on GitHub-hosted runners, fetch the bundled
cvrf-cve.tar.bz2 archive once and stream-extract it, mirroring upstream
PR aquasecurity#437 for the regular SUSE CVRF feed. This avoids tens of thousands of
individual HTTP requests, which previously pushed the job past the 6h
GitHub Actions limit.

- New package suse/cvrfarchive that encapsulates the
  download/decompress/tar walking logic shared by the CVRF and CVE CVRF
  feeds; suse/cvrf is refactored to use it as well, eliminating the
  duplicated archive plumbing.
- New package suse/cvrfcve (the previous suse/cvrf-cve was renamed to
  drop the hyphen in line with Go package conventions). The persisted
  schema is trimmed to the fields needed for CVSS lookup
  (Title / Tracking ID + dates / CVE / Threats / References /
  CVSSScoreSets); ProductTree, ProductStatuses, DocumentNotes and
  RevisionHistory are dropped, which keeps the resulting JSON tree
  small.
- Tests build a tar.gz archive at runtime via tar.Writer.AddFS and
  compare the output against committed golden files, matching the
  pattern used by suse/cvrf.
- Register the suse-cvrf-cve target in main.go and document it in
  README.md.
- Add a SUSE CVE CVRF step to .github/workflows/update.yml.

Co-authored-by: Cursor <cursoragent@cursor.com>
@manojkrishnanomula manojkrishnanomula force-pushed the feat/suse-cvrf-cve-feed branch from 0c165b2 to 0e6ca8b Compare May 14, 2026 08:57
@knqyf263

Copy link
Copy Markdown
Collaborator

Sorry for the late review.

The premise is correct: the advisory CVRF feed (cvrf.tar.bz2) only carries CVSS v2, not v3.

But SUSE also publishes an advisory-level CSAF feed (https://ftp.suse.com/pub/projects/security/csaf.tar.bz2), which covers everything we need from a single feed:

  • Same advisory granularity: document.tracking.id is the SUSE-SU / openSUSE-SU ID, so the vulnerability IDs in trivy-db stay the same.
  • Product matching: product_tree.relationships carries product_reference / relates_to_product_reference with the same values as the current CVRF Relationship fields, so getOSVersion / splitPkgName in trivy-db work unchanged.
  • CVSS v3 is included. I grepped both full archives: CSAF has 190,342 cvss_v3 entries, cvrf-cve has 41,576 ScoreSetV3, and neither contains any v4. So it's v3 only, not v3/v4 as the description states.

So why maintain two SUSE sources (cvrf + cvrf-cve) instead of switching the existing one over to CSAF? The per-CVE feed adds a ~241MB mirror, drops the advisory linkage, and largely duplicates what we already fetch.

trivy-db already depends on gocsaf/csaf for Red Hat (which still relies on OVAL v2 as well), so the parser is already available. Did you consider CSAF?

@manojkrishna-nomula

manojkrishna-nomula commented May 28, 2026

Copy link
Copy Markdown
Author

@knqyf263 Thanks for the detailed review — this is very helpful.

You're right on the facts:

  • Advisory CVRF only exposes CVSS v2 on the score fields we persist today.
  • SUSE's CSAF archive (csaf.tar.bz2) is advisory-level with the same tracking.id (SUSE-SU / openSUSE-SU) and product_tree.relationships that match our current CVRF ProductTree.Relationship model, so trivy-db's advisory IDs and getOSVersion / splitPkgName paths can stay aligned.
  • CVSS v3 is available in CSAF at much higher coverage than cvrf-cve; I'll fix the PR description — it's v3 only, not v3/v4.

We did not evaluate switching the existing suse-cvrf target to CSAF before adding suse-cvrf-cve. The motivation was to unblock CVSS v3 without changing the advisory JSON layout trivy-db already consumes. In hindsight, maintaining cvrf + cvrf-cve duplicates data, adds ~241MB to vuln-list, and drops advisory linkage for anything that keys off SUSE-SU-… / openSUSE-SU-….

I'm happy to pivot this work: close or replace the cvrf-cve approach and prototype suse-cvrf (or a new suse-csaf target) against csaf.tar.bz2, reusing csaf_distribution like Red Hat CSAF VEX. Before doing that in this PR I'd like to confirm with trivy-db maintainers:

  1. Output path/shape: still cvrf/suse/... with CSAF JSON, or a new csaf/suse/... tree?
  2. Whether we deprecate CVRF entirely for SUSE or run both during a transition.

If you prefer, I can open a follow-up PR for CSAF and revert the cvrf-cve feed from this one — let me know what you'd like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants