fix(sbom,ci): pin verifier check (C) to bomsh-traced gitoid; codespell

`make sbom`'s libtool relink rewrites `src/.libs/lib*.so*` after bomsh
has already gitoid-ed it, breaking check (C) on the previous push.
Capture the gitoid in `make bomsh` BEFORE `make sbom`, persist
`<path>\t<gitoid>` to `_bomsh.artefact`, and have check (C) compare SPDX
vs the saved gitoid (NOTE if on-disk has diverged).  Plus two codespell
typo fixes (`unparseable` → `unparsable`).

Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>

Author: sameehj

Makefile.am

@@ -475,10 +475,16 @@ BOMSH_CONF             = $(abs_builddir)/_bomsh.conf
 BOMSH_OMNIBORDIR       = $(abs_builddir)/omnibor
 BOMSH_SPDX_OUT         = omnibor.wolfssl-$(PACKAGE_VERSION).spdx.json
 # Single-source-of-truth manifest of the library artefact bomtrace3
-# actually traced.  Written by the bomsh: recipe so downstream
-# verification (CI: `Bomsh provenance is end-to-end verifiable`) doesn't
-# have to re-derive the same WOLFSSL_LIB_DSO_BASENAMES priority order
-# in parallel and risk drift.
+# actually traced.  Format: one line, '<path>\t<gitoid>'.  Both fields
+# are captured by the bomsh: recipe right after bomtrace3 finishes, so
+# downstream verification (CI: `Bomsh provenance is end-to-end
+# verifiable`) compares the SPDX gitoid against the gitoid bomsh
+# itself recorded -- decoupling check (C) from the file's *current*
+# bytes, which `make sbom`'s subsequent `make install` step relinks
+# in place via libtool (RPATH fixup), changing the gitoid that would
+# be re-computed off the on-disk file.  The verifier still warns when
+# the on-disk gitoid disagrees, so the install-time relink remains
+# visible.
 BOMSH_ARTEFACT_MANIFEST = $(abs_builddir)/_bomsh.artefact
 bomshdir          = $(datadir)/doc/$(PACKAGE)
 
@@ -508,25 +514,39 @@ bomsh:
 	@printf 'raw_logfile=%s\n' '$(BOMSH_RAWLOG_BASE)' > '$(BOMSH_CONF)'
 	$(BOMTRACE3) -c '$(BOMSH_CONF)' $(MAKE)
 	$(BOMSH_CREATE_BOM) -r '$(BOMSH_RAWLOG)' -b '$(BOMSH_OMNIBORDIR)'
-	$(MAKE) sbom
-	@if test -z "$(BOMSH_SBOM)"; then \
-	    echo "NOTE: bomsh_sbom.py not in PATH; skipping SPDX enrichment."; \
-	    echo "      The OmniBOR graph in $(BOMSH_OMNIBORDIR) is still produced."; \
-	    exit 0; \
-	fi; \
-	bomsh_artifact=""; \
+	@# Capture the gitoid of the bomtrace3-traced library BEFORE the
+	@# `make sbom` below, which calls `make install DESTDIR=...` --
+	@# libtool's --mode=install relinks src/.libs/lib*.so* in place
+	@# to fix RPATH, mutating the bytes that bomsh recorded in the
+	@# ADG via bomsh_create_bom above.  Capturing here pins the
+	@# verifier's check (C) to the bomsh-traced gitoid (so SPDX <->
+	@# manifest agree even though the on-disk bytes diverge after
+	@# install).  The on-disk divergence is surfaced as a verifier
+	@# warning, not a failure.
+	@bomsh_artifact=""; \
 	for lib in \
 	    $(addprefix "$(abs_builddir)/src/.libs"/,$(WOLFSSL_LIB_DSO_BASENAMES)) \
 	    "$(abs_builddir)/src/.libs/libwolfssl.a" \
 	    "$(abs_builddir)/src/libwolfssl.a"; do \
 	    if test -f "$$lib"; then bomsh_artifact="$$lib"; break; fi; \
 	done; \
-	if test -z "$$bomsh_artifact"; then \
+	if test -n "$$bomsh_artifact"; then \
+	    bomsh_artifact_gid=`$(PYTHON3) -c 'import hashlib,sys;d=open(sys.argv[1],"rb").read();h=hashlib.sha1();h.update(("blob %d\0"%len(d)).encode());h.update(d);print(h.hexdigest())' "$$bomsh_artifact"`; \
+	    printf '%s\t%s\n' "$$bomsh_artifact" "$$bomsh_artifact_gid" \
+	        > '$(BOMSH_ARTEFACT_MANIFEST)'; \
+	fi
+	$(MAKE) sbom
+	@if test -z "$(BOMSH_SBOM)"; then \
+	    echo "NOTE: bomsh_sbom.py not in PATH; skipping SPDX enrichment."; \
+	    echo "      The OmniBOR graph in $(BOMSH_OMNIBORDIR) is still produced."; \
+	    exit 0; \
+	fi; \
+	if test ! -f '$(BOMSH_ARTEFACT_MANIFEST)'; then \
 	    echo "NOTE: no built libwolfssl artifact found in $(abs_builddir)/src/.libs/"; \
 	    echo "      OmniBOR graph produced; SPDX enrichment skipped."; \
 	    exit 0; \
 	fi; \
-	printf '%s\n' "$$bomsh_artifact" > '$(BOMSH_ARTEFACT_MANIFEST)'; \
+	bomsh_artifact=`awk 'NR==1 {print $$1}' '$(BOMSH_ARTEFACT_MANIFEST)'`; \
 	echo "Enriching SPDX with OmniBOR ExternalRefs (artifact: $$bomsh_artifact)..."; \
 	$(BOMSH_SBOM) \
 	    -b '$(BOMSH_OMNIBORDIR)' \

scripts/bomsh_verify.py

@@ -15,17 +15,26 @@
       a downstream verifier weeks later.
 
   (C) Artefact correspondence -- the gitoid recorded against the
-      wolfSSL package equals the git-blob hash of the actual library
-      artefact that `make bomsh` traced (read from the
-      `_bomsh.artefact` manifest written by the bomsh: Makefile
-      target).  This is what makes the SBOM a true attestation of
-      the binary that would ship.
+      wolfSSL package equals the gitoid bomsh itself recorded for the
+      library it traced (read from the `_bomsh.artefact` manifest the
+      bomsh: Makefile target writes as '<path>\\t<gitoid>' BEFORE
+      `make sbom` runs).  This is the strongest claim the bomsh
+      pipeline alone can make: the SPDX agrees with what bomsh saw.
+
+      Comparing against bomsh's own recorded gitoid (rather than
+      against the on-disk file's *current* bytes) is deliberate.
+      `make sbom`'s subsequent `make install` step relinks
+      src/.libs/lib*.so* in place via libtool to fix RPATH, mutating
+      the bytes after bomsh has already gitoid-ed them.  The verifier
+      still hashes the on-disk file and emits a NOTE if it has
+      diverged, so the install-time relink remains visible without
+      causing a false negative on the bomsh<->SPDX agreement.
 
 Without this, a future `bomsh_sbom.py` change that emits a
 plausibly-shaped but fictional gitoid (one that does not resolve in
-the ADG, or resolves but to the wrong artefact) would pass the
-existing PERSISTENT-ID assertion and ship a provenance bundle whose
-externalRef is a lie.
+the ADG, or resolves but to a different artefact than bomsh recorded)
+would pass the existing PERSISTENT-ID assertion and ship a provenance
+bundle whose externalRef is a lie.
 
 CLI form (used by `.github/workflows/sbom.yml`):
 
@@ -150,27 +159,51 @@ def check_object_store_integrity(omnibor_objects_dir):
     return obj_count, bad
 
 
-def check_artefact_correspondence(spdx_gitoids, artefact_path,
-                                  package_name_substr='wolfssl'):
-    """(C) The gitoid recorded against the wolfSSL package equals the
-    git-blob hash of the library artefact at <artefact_path>.
-
-    Returns (artefact_gid, wolfssl_gids).  Caller checks
-    `artefact_gid in wolfssl_gids`.  Raises FileNotFoundError if the
-    artefact does not exist; raises ValueError if no SPDX gitoid is
-    associated with a wolfSSL package."""
-    if not os.path.isfile(artefact_path):
+def parse_artefact_manifest(manifest_path):
+    """Parse the `_bomsh.artefact` manifest written by the bomsh:
+    recipe.  Format: a single line, `<absolute-path>\\t<gitoid-hex>`
+    -- both fields captured by the recipe AFTER bomtrace3 finishes
+    but BEFORE `make sbom` relinks the library.
+
+    Returns (path, recorded_gid).  Raises FileNotFoundError if the
+    manifest does not exist (bomsh: skipped artefact discovery, e.g.
+    no built library); raises ValueError if the line is malformed."""
+    if not os.path.isfile(manifest_path):
         raise FileNotFoundError(
-            f'artefact {artefact_path!r} does not exist')
-    artefact_gid = gitoid_sha1(artefact_path)
+            f'{manifest_path} not produced by `make bomsh`; cannot '
+            f'verify gitoid <-> artefact correspondence.  This usually '
+            f'means the bomsh enrichment step skipped the artefact-'
+            f'discovery loop (no built library).')
+    with open(manifest_path) as f:
+        line = f.readline().rstrip('\n')
+    if not line:
+        raise ValueError(
+            f'{manifest_path} is empty; bomsh: recipe wrote nothing')
+    parts = line.split('\t')
+    if len(parts) != 2 or not all(parts):
+        raise ValueError(
+            f'{manifest_path}: expected "<path>\\t<gitoid>", got {line!r}.  '
+            f'Re-run `make bomsh` against an up-to-date Makefile.am.')
+    return parts[0], parts[1]
+
+
+def check_artefact_correspondence(spdx_gitoids, recorded_gid,
+                                  package_name_substr='wolfssl'):
+    """(C) The gitoid bomsh recorded for the traced library matches a
+    gitoid externalRef on the wolfSSL SPDX package.  This is the
+    bomsh<->SPDX agreement check; it does NOT compare against the
+    on-disk file's current bytes (see module docstring).
+
+    Returns (matched, wolfssl_gids).  Raises ValueError if no SPDX
+    gitoid is associated with a wolfSSL-named package."""
     wolfssl_gids = [gid for name, gid in spdx_gitoids
                     if package_name_substr in name.lower()]
     if not wolfssl_gids:
         raise ValueError(
             f'no SPDX gitoid externalRef on a package whose name '
             f'contains {package_name_substr!r}; cannot verify '
             f'artefact correspondence')
-    return artefact_gid, wolfssl_gids
+    return recorded_gid in wolfssl_gids, wolfssl_gids
 
 
 def verify(spdx_glob, omnibor_dir, artefact_manifest,
@@ -214,39 +247,50 @@ def verify(spdx_glob, omnibor_dir, artefact_manifest,
             f'round-trip (object store is corrupt)')
         return False, messages
 
-    if not os.path.isfile(artefact_manifest):
-        messages.append(
-            f'{artefact_manifest} not produced by `make bomsh`; '
-            f'cannot verify gitoid <-> artefact correspondence.  '
-            f'This usually means the bomsh enrichment step skipped '
-            f'the artefact-discovery loop (no built library).')
-        return False, messages
-    with open(artefact_manifest) as f:
-        artefact = f.read().strip()
-    if not artefact:
-        messages.append(
-            f'{artefact_manifest} is empty; bomsh: recipe wrote a '
-            f'blank path')
+    try:
+        artefact, recorded_gid = parse_artefact_manifest(artefact_manifest)
+    except (FileNotFoundError, ValueError) as e:
+        messages.append(str(e))
         return False, messages
 
     try:
-        artefact_gid, wolfssl_gids = check_artefact_correspondence(
-            spdx_gitoids, artefact, package_name_substr)
-    except (FileNotFoundError, ValueError) as e:
+        matched, wolfssl_gids = check_artefact_correspondence(
+            spdx_gitoids, recorded_gid, package_name_substr)
+    except ValueError as e:
         messages.append(str(e))
         return False, messages
 
-    if artefact_gid not in wolfssl_gids:
+    if not matched:
         messages.append(
             f'wolfSSL package SPDX gitoids {wolfssl_gids} do not '
-            f'include the gitoid of the actual built artefact '
-            f'{artefact} ({artefact_gid}); the SBOM does not '
-            f'attest to the binary that would ship')
+            f'include the gitoid bomsh recorded for the traced '
+            f'artefact {artefact} ({recorded_gid}); the SBOM is '
+            f'inconsistent with what bomsh actually saw')
         return False, messages
 
     messages.append(f'OK: {len(spdx_gitoids)} gitoid(s) verified')
     messages.append(f'    objects round-trip: {obj_count} blobs')
-    messages.append(f'    artefact match: {artefact} -> {artefact_gid}')
+    messages.append(
+        f'    artefact match: {artefact} -> {recorded_gid} (bomsh-traced)')
+
+    # Diagnostic-only: the on-disk file may have been rewritten since
+    # bomsh saw it (the canonical case is `make sbom`'s `make install`
+    # step relinking via libtool to fix RPATH).  We do NOT fail on
+    # this -- the SBOM<->bomsh agreement above is what matters for
+    # the provenance proof -- but surfacing it as a NOTE keeps the
+    # divergence visible so it does not silently grow into a
+    # bigger gap (e.g. someone adds a strip step that goes unflagged).
+    if os.path.isfile(artefact):
+        on_disk = gitoid_sha1(artefact)
+        if on_disk != recorded_gid:
+            messages.append(
+                f'NOTE: on-disk {artefact} now has gitoid {on_disk}, '
+                f'but bomsh recorded {recorded_gid}.  This is expected '
+                f'when `make sbom` runs `make install` (libtool relinks '
+                f'src/.libs/lib*.so* in place to fix RPATH).  The SBOM '
+                f'attests to the bomsh-traced bytes; if you need it to '
+                f'attest to the *installed* bytes, the bomsh: recipe '
+                f'must trace `make install` too.')
     return True, messages
 
 

scripts/test_gen_sbom.py

@@ -400,7 +400,7 @@ def test_no_gpl_mention_returns_none_with_warning(self):
                 'Permission is hereby granted, free of charge, ...\n')
         self.assertIsNone(result)
         # Warning must mention the file path so an operator running
-        # `make sbom` can see which file was unparseable.
+        # `make sbom` can see which file was unparsable.
         self.assertIn('no GPL version found', stderr.getvalue())
 
     def test_missing_file_returns_none_with_warning(self):
@@ -1898,15 +1898,26 @@ def __init__(self, tmpdir):
         # check (B) actually exercises its loop).
         self.artefact_content = b'\x7fELF...wolfssl shared library content...'
         self.artefact_path.write_bytes(self.artefact_content)
-        self.manifest_path.write_text(str(self.artefact_path) + '\n')
         self.aux_blobs = [b'/* aes.c */\n', b'/* sha.c */\n']
         self.gitoids = {
             'wolfssl': self._stage_blob(self.artefact_content),
         }
         for i, content in enumerate(self.aux_blobs):
             self.gitoids[f'aux{i}'] = self._stage_blob(content)
+        # Manifest mirrors the new bomsh: recipe format: a single line,
+        # '<path>\t<gitoid>'.  The gitoid is captured BEFORE `make sbom`
+        # would rewrite the file (libtool relink), so it pins the
+        # bomsh-traced bytes rather than the on-disk current bytes --
+        # decoupling check (C) from libtool's install-time rewrite.
+        self.write_manifest(self.artefact_path, self.gitoids['wolfssl'])
         self._write_spdx()
 
+    def write_manifest(self, path, gid):
+        """Helper so individual tests can rewrite the manifest with a
+        deliberately-wrong path or gitoid without re-reading
+        the recipe's exact format."""
+        self.manifest_path.write_text(f'{path}\t{gid}\n')
+
     def _stage_blob(self, content):
         """Write `content` into omnibor/objects/<aa>/<rest> at the
         correct gitoid path; return the gitoid hex.  Uses
@@ -2034,26 +2045,67 @@ def test_artefact_manifest_missing_fails_check_C(self):
             self.assertIn('not produced by `make bomsh`', joined)
 
     def test_artefact_gid_mismatch_fails_check_C(self):
-        # Manifest points at a different file (e.g. the recipe ran the
-        # discovery loop on a stale build).  The verifier must reject
-        # and surface BOTH the SPDX gitoid set and the actual artefact
-        # gitoid so the operator can diff them.
+        # Manifest records a gitoid that does NOT match any wolfSSL
+        # SPDX externalRef.  This is the canonical "bomsh recorded X
+        # but bomsh_sbom enriched the SPDX with a different gitoid Y"
+        # bug -- exactly what check (C) is here to catch.  The failure
+        # message must surface both the SPDX gitoid set AND the
+        # bomsh-recorded gitoid so the operator can diff them.
+        with tempfile.TemporaryDirectory() as tmpdir:
+            fx = _BomshFixture(tmpdir)
+            fake_gid = 'f' * 40
+            fx.write_manifest(fx.artefact_path, fake_gid)
+            ok, messages = fx.verify()
+            self.assertFalse(ok)
+            joined = '\n'.join(messages)
+            self.assertIn('inconsistent with what bomsh actually saw',
+                          joined)
+            self.assertIn(fake_gid, joined)
+            self.assertIn(fx.gitoids['wolfssl'], joined)
+
+    def test_on_disk_divergence_emits_note_but_passes(self):
+        # The on-disk artefact bytes have changed since bomsh recorded
+        # them (the canonical libtool-relink case).  Check (C) compares
+        # against the manifest's bomsh-recorded gitoid (which still
+        # matches the SPDX), so the verifier must PASS, but it must
+        # also emit a NOTE so the divergence is not silently hidden.
+        # Pinning this is the contract that makes the install-time
+        # relink visible without breaking CI.
+        with tempfile.TemporaryDirectory() as tmpdir:
+            fx = _BomshFixture(tmpdir)
+            # Rewrite the on-disk artefact AFTER the fixture pinned
+            # its gitoid in the manifest -- simulates `make sbom`'s
+            # `make install` relink.
+            fx.artefact_path.write_bytes(b'post-relink RPATH-fixed bytes')
+            ok, messages = fx.verify()
+            self.assertTrue(ok, f'verifier failed despite agreement: {messages}')
+            joined = '\n'.join(messages)
+            self.assertIn('NOTE:', joined)
+            self.assertIn('libtool relinks', joined)
+            # Both gitoids surfaced so triage doesn't need a second pass.
+            self.assertIn(fx.gitoids['wolfssl'], joined)
+
+    def test_manifest_path_only_legacy_format_rejected(self):
+        # A manifest containing only the path (the pre-fix legacy
+        # format) must be rejected explicitly, with a message that
+        # tells the operator to re-run `make bomsh` against an
+        # up-to-date Makefile.am.  Silent acceptance would re-introduce
+        # the false-positive failure mode the new format was designed
+        # to prevent.
         with tempfile.TemporaryDirectory() as tmpdir:
             fx = _BomshFixture(tmpdir)
-            wrong = pathlib.Path(tmpdir) / 'wrong-artefact.so'
-            wrong.write_bytes(b'totally different bytes')
-            fx.manifest_path.write_text(str(wrong) + '\n')
+            fx.manifest_path.write_text(str(fx.artefact_path) + '\n')
             ok, messages = fx.verify()
             self.assertFalse(ok)
             joined = '\n'.join(messages)
-            self.assertIn('does not attest to the binary', joined)
-            self.assertIn('wolfssl', joined.lower())
+            self.assertIn('expected "<path>\\t<gitoid>"', joined)
+            self.assertIn('up-to-date Makefile.am', joined)
 
     def test_unexpected_gitoid_locator_format_rejected(self):
         # bomsh upstream switching from sha1 to sha256 would change
         # the locator prefix.  load_spdx_gitoids must raise so the
         # maintainer is forced to update the verifier in lockstep,
-        # rather than silently accepting an unparseable value.
+        # rather than silently accepting an unparsable value.
         with tempfile.TemporaryDirectory() as tmpdir:
             fx = _BomshFixture(tmpdir)
             spdx = json.loads(fx.spdx_path.read_text())