SBOM report ane 2277 (#1534)

* Add a force-sbom option.

* Fetch SBOMs using fossa report.

* Implement tests,do error reporting better.

* Make report intelligently pick between bases.

* Just use the fallback method if we ever get a directory.

* Fix tests

* Try to make fixtures work on windows too.

* cleanup

* Update docs

* Update changelog.

* Delete redundant import.

* Parse fix.

* Specify that you need to run `fossa sbom analyze` first.

Author: csasarak

Changelog.md

@@ -1,5 +1,9 @@
 # FOSSA CLI Changelog
 
+## 3.10.7
+
+- Report: Allow generating SBOMs attribution reports using fossa-cli. ([#1534](https://github.com/fossas/fossa-cli/pull/1534))
+
 ## 3.10.6
 - Licensing: Fix a bug where the scikit-learn had an incorrect license detected ([#1527](https://github.com/fossas/fossa-cli/pull/1527))
 - Licensing: Adds support for the NREL disclaimer

docs/references/subcommands/report.md

@@ -24,6 +24,41 @@ fossa report attribution --timeout 60
 
 Where `60` is the maximum number of seconds to wait for the report to be downloaded.
 
+### Valid Report Targets
+
+#### Project Directory
+
+You can specify a project directly like you would with `fossa analyze` to generate a report.
+For example:
+
+```
+fossa report attribution --format json ~/my-project
+```
+
+With no final path, FOSSA will try to fetch a report for the current directory's project.
+
+#### SBOM Files
+
+After using [`fossa sbom analyze`](./sbom.md), you can specify an SBOM that you would like to generate a report for using its file:
+
+```
+fossa report attribution --format json ~/my-project-sbom.txt
+```
+
+#### Project Arguments
+
+All `fossa` commands support the following FOSSA-project-related flags:
+
+| Name                               | Short | Description                                                                                                                                            |
+| ---------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `--project 'some project'`         | `-p`  | Override the detected project name                                                                                                                     |
+| `--revision 'some revision'`       | `-r`  | -Override the detected project revision                                                                                                                |
+| `--fossa-api-key 'my-api-key'`     |       | An alternative to using the `FOSSA_API_KEY` environment variable to specify a FOSSA API key                                                            |
+| `--endpoint 'https://example.com'` | `-e`  | Override the FOSSA API server base URL                                                                                                                 |
+| `--config /path/to/file`           | `-c`  | Path to a [configuration file](../files/fossa-yml.md) including filename. By default we look for `.fossa.yml` in base working directory. |
+
+In this case, FOSSA will attempt to fetch any report it can find matching the `project` and `revision` criteria.
+
 ### Specifying a report format
 
 `fossa report` supports customizing the format used to render a report via the `--format` flag.
@@ -59,15 +94,3 @@ To use this compatibility script:
 2. Run `fossa report attribution --format json`, piping its output to `compat-attribution`.
    For example, `fossa report attribution --format json | compat-attribution`
 3. Parse the resulting output as you would have from FOSSAv1.
-
-## Common FOSSA Project Flags
-
-All `fossa` commands support the following FOSSA-project-related flags:
-
-| Name                               | Short | Description                                                                                                                                            |
-| ---------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| `--project 'some project'`         | `-p`  | Override the detected project name                                                                                                                     |
-| `--revision 'some revision'`       | `-r`  | -Override the detected project revision                                                                                                                |
-| `--fossa-api-key 'my-api-key'`     |       | An alternative to using the `FOSSA_API_KEY` environment variable to specify a FOSSA API key                                                            |
-| `--endpoint 'https://example.com'` | `-e`  | Override the FOSSA API server base URL                                                                                                                 |
-| `--config /path/to/file`           | `-c`  | Path to a [configuration file](../files/fossa-yml.md) including filename. By default we look for `.fossa.yml` in base working directory. |

src/App/Fossa/API/BuildWait.hs

@@ -153,13 +153,14 @@ waitForReportReadiness ::
   ) =>
   ProjectRevision ->
   Cancel ->
+  LocatorType ->
   m ()
-waitForReportReadiness revision cancelFlag = do
-  void $ waitForIssues revision Nothing LocatorTypeCustom cancelFlag
+waitForReportReadiness revision cancelFlag locatorType = do
+  void $ waitForIssues revision Nothing locatorType cancelFlag
 
   supportsDepCacheReadinessPolling <- orgSupportsDependenciesCachePolling <$> getOrganization
   when supportsDepCacheReadinessPolling $
-    waitForValidDependenciesCache revision cancelFlag
+    waitForValidDependenciesCache revision cancelFlag locatorType
 
 waitForValidDependenciesCache ::
   ( Has Diagnostics sig m
@@ -170,15 +171,16 @@ waitForValidDependenciesCache ::
   ) =>
   ProjectRevision ->
   Cancel ->
+  LocatorType ->
   m ()
-waitForValidDependenciesCache revision cancelFlag = do
+waitForValidDependenciesCache revision cancelFlag locatorType = do
   checkForTimeout cancelFlag
-  cacheStatus <- getRevisionDependencyCacheStatus revision
+  cacheStatus <- getRevisionDependencyCacheStatus revision locatorType
 
   case status cacheStatus of
     Ready -> pure ()
     Waiting -> do
       logSticky' $ "[ Waiting for revision's dependency cache... last status: " <> viaShow Waiting <> " ]"
       pauseForRetry
-      waitForValidDependenciesCache revision cancelFlag
+      waitForValidDependenciesCache revision cancelFlag locatorType
     UnknownDependencyCacheStatus status -> fatalText $ "unknown status of " <> status <> " received for revision's dependency cache"

src/App/Fossa/Config/Common.hs

@@ -48,6 +48,7 @@ module App.Fossa.Config.Common (
   titleHelp,
   -- Deprecation
   applyReleaseGroupDeprecationWarning,
+  collectBaseFile,
 ) where
 
 import App.Fossa.Config.ConfigFile (
@@ -259,6 +260,8 @@ pathOpt = first show . parseRelDir
 targetOpt :: String -> Either String TargetFilter
 targetOpt = first errorBundlePretty . runParser targetFilterParser "(Command-line arguments)" . toText
 
+-- | Argument for the base dir for FOSSA to operate out of.
+-- Defaults to the current directory, ".".
 baseDirArg :: Parser String
 baseDirArg = argument str (applyFossaStyle <> metavar "DIR" <> helpDoc baseDirDoc <> value ".")
   where
@@ -278,6 +281,14 @@ collectBaseDir ::
   m BaseDir
 collectBaseDir = fmap BaseDir . validateDir
 
+collectBaseFile ::
+  ( Has Diagnostics sig m
+  , Has (Lift IO) sig m
+  , Has ReadFS sig m
+  ) =>
+  FilePath -> m (Path Abs File)
+collectBaseFile = validateFile
+
 validateDir ::
   ( Has Diagnostics sig m
   , Has (Lift IO) sig m

src/App/Fossa/Config/Report.hs

@@ -5,6 +5,7 @@ module App.Fossa.Config.Report (
   ReportCliOptions,
   ReportOutputFormat (..),
   ReportType (..),
+  ReportBase (..),
   mkSubCommand,
   -- Exported for testing
   parseReportOutputFormat,
@@ -16,21 +17,27 @@ import App.Fossa.Config.Common (
   baseDirArg,
   collectApiOpts,
   collectBaseDir,
+  collectBaseFile,
   collectRevisionData',
   commonOpts,
   defaultTimeoutDuration,
  )
 import App.Fossa.Config.ConfigFile (ConfigFile, resolveLocalConfigFile)
 import App.Fossa.Config.EnvironmentVars (EnvVars)
+import App.Fossa.Config.SBOM.Common qualified as SBOMCfg
 import App.Fossa.Subcommand (EffStack, GetCommonOpts (getCommonOpts), GetSeverity (getSeverity), SubCommand (SubCommand))
-import App.Types (BaseDir, OverrideProject (OverrideProject), ProjectRevision)
+import App.Types (BaseDir (..), OverrideProject (OverrideProject), ProjectRevision)
+import Control.Applicative ((<|>))
 import Control.Effect.Diagnostics (Diagnostics, ToDiagnostic (renderDiagnostic), errHelp, fatal, fromMaybe)
+import Control.Effect.Diagnostics qualified as Diag
 import Control.Effect.Lift (Has, Lift)
 import Control.Timeout (Duration (Seconds))
 import Data.Aeson (ToJSON (toEncoding), defaultOptions, genericToEncoding)
 import Data.Error (SourceLocation, createEmptyBlock, getSourceLocation)
 import Data.List (intercalate)
 import Data.String.Conversion (ToText, toText)
+import Data.String.Conversion qualified as Conv
+import Data.Text (Text)
 import Effect.Exec (Exec)
 import Effect.Logger (Logger, Severity (..), vsep)
 import Effect.ReadFS (ReadFS)
@@ -52,6 +59,7 @@ import Options.Applicative (
   switch,
  )
 import Options.Applicative.Builder (helpDoc)
+import Path
 import Prettyprinter (Doc, comma, hardline, punctuate, viaShow)
 import Prettyprinter.Render.Terminal (AnsiStyle, Color (Green, Red))
 import Style (applyFossaStyle, boldItalicized, coloredBoldItalicized, formatDoc, stringToHelpDoc, styledDivider)
@@ -146,8 +154,11 @@ parser =
     <*> optional (strOption (applyFossaStyle <> long "format" <> helpDoc formatHelp))
     <*> optional (option auto (applyFossaStyle <> long "timeout" <> stringToHelpDoc "Duration to wait for build completion (in seconds)"))
     <*> reportTypeArg
-    <*> baseDirArg
+    <*> basePath
   where
+    basePath :: Parser Text
+    basePath = (Conv.toText <$> baseDirArg) <|> (SBOMCfg.unSBOMFile <$> SBOMCfg.sbomFileArg)
+
     jsonHelp :: Maybe (Doc AnsiStyle)
     jsonHelp =
       Just . formatDoc $
@@ -178,7 +189,7 @@ data ReportCliOptions = ReportCliOptions
   , cliReportOutputFormat :: Maybe String
   , cliReportTimeout :: Maybe Int
   , cliReportType :: ReportType
-  , cliReportBaseDir :: FilePath
+  , cliReportBase :: Text
   }
   deriving (Eq, Ord, Show)
 
@@ -211,19 +222,43 @@ mergeOpts ::
   m ReportConfig
 mergeOpts cfgfile envvars ReportCliOptions{..} = do
   let apiOpts = collectApiOpts cfgfile envvars commons
-      basedir = collectBaseDir cliReportBaseDir
       outputformat = validateOutputFormat cliReportJsonOutput cliReportOutputFormat
       timeoutduration = maybe defaultTimeoutDuration Seconds cliReportTimeout
-      revision =
-        collectRevisionData' basedir cfgfile ReadOnly $
-          OverrideProject (optProjectName commons) (optProjectRevision commons) Nothing
+      projectOverride = OverrideProject (optProjectName commons) (optProjectRevision commons) Nothing
+
+  (revision, reportBase) <- generateDirOrSBOMBase cliReportBase projectOverride
+
   ReportConfig
     <$> apiOpts
-    <*> basedir
+    <*> pure reportBase
     <*> outputformat
     <*> pure timeoutduration
     <*> pure cliReportType
-    <*> revision
+    <*> pure revision
+  where
+    generateDirOrSBOMBase ::
+      ( Has Exec sig m
+      , Has Logger sig m
+      , Has (Lift IO) sig m
+      , Has ReadFS sig m
+      , Has Diagnostics sig m
+      ) =>
+      Text -> OverrideProject -> m (ProjectRevision, ReportBase)
+    generateDirOrSBOMBase path projectOverride = do
+      basedir <- Diag.recover $ collectBaseDir (Conv.toString path)
+      case basedir of
+        Just dir ->
+          (,)
+            <$> collectRevisionData' (pure dir) cfgfile ReadOnly projectOverride
+            <*> pure (DirectoryBase . unBaseDir $ dir)
+        Nothing -> do
+          baseFile <- Diag.recover $ collectBaseFile (Conv.toString path)
+          case baseFile of
+            Just file ->
+              (,)
+                <$> SBOMCfg.getProjectRevision (SBOMCfg.SBOMFile (toText file)) projectOverride ReadOnly
+                <*> pure (SBOMBase file)
+            Nothing -> Diag.fatalText $ "No such file or directory " <> path
 
 newtype NoFormatProvided = NoFormatProvided SourceLocation
 instance ToDiagnostic NoFormatProvided where
@@ -252,13 +287,21 @@ validateOutputFormat False (Just format) = errHelp ReportErrorHelp $ fromMaybe (
 
 data ReportConfig = ReportConfig
   { apiOpts :: ApiOpts
-  , baseDir :: BaseDir
+  , reportBase :: ReportBase
   , outputFormat :: ReportOutputFormat
   , timeoutDuration :: Duration
   , reportType :: ReportType
   , revision :: ProjectRevision
   }
   deriving (Eq, Ord, Show, Generic)
 
+data ReportBase
+  = SBOMBase (Path Abs File)
+  | DirectoryBase (Path Abs Dir)
+  deriving (Eq, Ord, Show, Generic)
+
+instance ToJSON ReportBase where
+  toEncoding = genericToEncoding defaultOptions
+
 instance ToJSON ReportConfig where
   toEncoding = genericToEncoding defaultOptions

src/App/Fossa/Config/SBOM/Common.hs

@@ -66,4 +66,4 @@ getProjectRevision sbomPath override cacheStrategy = do
   let version = fromMaybe inferredVersion $ overrideRevision override
   let revision = ProjectRevision name version Nothing
   when (cacheStrategy == WriteOnly) $ saveRevision revision
-  pure $ ProjectRevision name version Nothing
+  pure revision

src/App/Fossa/Container/Scan.hs

@@ -16,7 +16,7 @@ import App.Fossa.Container.Sources.DockerArchive (analyzeFromDockerArchive, revi
 import App.Fossa.Container.Sources.DockerEngine (analyzeFromDockerEngine, revisionFromDockerEngine)
 import App.Fossa.Container.Sources.Podman (analyzeFromPodman, podmanInspectImage, revisionFromPodman)
 import App.Fossa.Container.Sources.Registry (analyzeFromRegistry, revisionFromRegistry, runWithCirceReexport)
-import App.Types (OverrideProject (..), ProjectRevision (ProjectRevision))
+import App.Types (OverrideProject (..), ProjectRevision (..))
 import Container.Docker.SourceParser (RegistryImageSource (..), parseImageUrl)
 import Container.Types (ContainerScan (..))
 import Control.Carrier.DockerEngineApi (runDockerEngineApi)

src/App/Fossa/Report.hs

@@ -10,18 +10,19 @@ import App.Fossa.API.BuildWait (
   waitForReportReadiness,
   waitForScanCompletion,
  )
-import App.Fossa.Config.Report (ReportCliOptions, ReportConfig (..), ReportOutputFormat (ReportJson), mkSubCommand)
+import App.Fossa.Config.Report (ReportBase (..), ReportCliOptions, ReportConfig (..), ReportOutputFormat (ReportJson), mkSubCommand)
 import App.Fossa.PreflightChecks (PreflightCommandChecks (ReportChecks), guardWithPreflightChecks)
 import App.Fossa.Subcommand (SubCommand)
 import App.Types (LocatorType (..), ProjectRevision (..))
 import Control.Carrier.Debug (ignoreDebug)
 import Control.Carrier.FossaApiClient (runFossaApiClient)
 import Control.Carrier.StickyLogger (StickyLogger, logSticky, runStickyLogger)
-import Control.Effect.Diagnostics (Diagnostics)
+import Control.Effect.Diagnostics (Diagnostics, (<||>))
 import Control.Effect.FossaApiClient (FossaApiClient, getAttribution)
 import Control.Effect.Lift (Has, Lift)
 import Control.Monad (void, when)
 import Control.Timeout (timeout')
+import Data.Functor (($>))
 import Data.String.Conversion (toText)
 import Data.Text.Extra (showT)
 import Effect.Logger (
@@ -79,13 +80,20 @@ fetchReport ReportConfig{..} =
       when (outputFormat /= ReportJson) $
         logWarn (pretty $ "\"" <> toText outputFormat <> "\" format may change independent of CLI version: it is sourced from the FOSSA service.")
 
+      let waitForSbomCompletion = waitForScanCompletion revision LocatorTypeSBOM cancelToken $> LocatorTypeSBOM
+          waitForCustomCompletion = waitForScanCompletion revision LocatorTypeCustom cancelToken $> LocatorTypeCustom
+
       logSticky "[ Waiting for build completion... ]"
+      locatorType <-
+        case reportBase of
+          SBOMBase _ -> waitForSbomCompletion
+          DirectoryBase _ -> waitForCustomCompletion <||> waitForSbomCompletion
 
-      waitForScanCompletion revision LocatorTypeCustom cancelToken
       logSticky "[ Waiting for scan completion... ]"
 
-      waitForReportReadiness revision cancelToken
+      waitForReportReadiness revision cancelToken locatorType
+
       logSticky $ "[ Fetching " <> showT reportType <> " report... ]"
 
-      renderedReport <- getAttribution revision outputFormat
+      renderedReport <- getAttribution revision outputFormat locatorType
       logStdout renderedReport

src/App/Types.hs

@@ -19,11 +19,13 @@ module App.Types (
   ComponentUploadFileType (..),
   Mode (..),
   uploadFileTypeToFetcherName,
+  locatorAPIText,
 ) where
 
 import Data.Aeson (FromJSON (parseJSON), ToJSON (toEncoding), defaultOptions, genericToEncoding, object, withObject, (.:), (.=))
 import Data.Aeson.Types (toJSON)
 import Data.Map (Map)
+import Data.String (IsString)
 import Data.String.Conversion (ToText (..), showText)
 import Data.Text (Text)
 import DepTypes (DepType)
@@ -92,13 +94,16 @@ data ProjectRevision = ProjectRevision
 data LocatorType = LocatorTypeCustom | LocatorTypeSBOM
   deriving (Eq, Ord, Show, Generic)
 
+-- | How to output a 'LocatorType' when interacting with FOSSA APIs.
+locatorAPIText :: IsString a => LocatorType -> a
+locatorAPIText LocatorTypeCustom = "custom"
+locatorAPIText LocatorTypeSBOM = "sbom"
+
 instance ToText LocatorType where
-  toText LocatorTypeCustom = "custom"
-  toText LocatorTypeSBOM = "sbom"
+  toText = locatorAPIText
 
 instance ToJSON LocatorType where
-  toEncoding LocatorTypeCustom = "custom"
-  toEncoding LocatorTypeSBOM = "sbom"
+  toEncoding = locatorAPIText
 
 instance ToJSON ProjectRevision where
   toEncoding = genericToEncoding defaultOptions

src/Control/Carrier/FossaApiClient.hs

@@ -66,11 +66,11 @@ runFossaApiClient apiOpts action = do
             FinalizeLicenseScan components -> LicenseScanning.finalizeLicenseScan components
             FinalizeLicenseScanForPathDependency locators forceRebuild -> LicenseScanning.finalizePathDependencyScan locators forceRebuild
             GetApiOpts -> pure apiOpts
-            GetAttribution rev format -> Core.getAttribution rev format
+            GetAttribution rev format locType -> Core.getAttribution rev format locType
             GetIssues rev diffRev locatorType -> Core.getIssues rev diffRev locatorType
             GetEndpointVersion -> Core.getEndpointVersion
             GetLatestBuild rev locatorType -> Core.getLatestBuild rev locatorType
-            GetRevisionDependencyCacheStatus rev -> Core.getRevisionDependencyCacheStatus rev
+            GetRevisionDependencyCacheStatus rev locatorType -> Core.getRevisionDependencyCacheStatus rev locatorType
             GetOrganization -> Core.getOrganization
             GetPolicies -> Core.getPolicies
             GetProject rev locatorType -> Core.getProject rev locatorType