Parallelize the scan pr flow #1028

eyalk007 · 2026-01-13T13:15:58Z

All tests passed. If this feature is not already covered by the tests, I added new tests.
This pull request is on the dev branch.
I used gofmt for formatting the code before submitting the pull request.
Update documentation about new features / new supported technologies

- Run SCA and JAS scans in parallel for better performance - Use LogCollector for isolated log capture per scan - Logs dumped sequentially after scans complete (no interleaving) - Add SetLogCollector to ScanDetails for passing collector to CLI - Export FrogbotUploadRtRepoPath for result upload

- Add JF_PARALLEL_PR_SCAN env var (default: true, set to 'false' for sequential mode) - Remove verbose/redundant comments - Remove gradle cache files from tracking

…ing, support FailUponAnyScannerError

…t, remove comments

- scanPullRequestBranches: scanning + partial results filtering - auditPullRequestAndReport: output flags + issue conversion

eyalk007 · 2026-01-14T09:51:23Z

frogbot-pr-info.txt - regular log mode

frogbot-pr-debug.txt - debug mode

utils/scandetails.go

go.mod

…ctives

scanpullrequest/scanpullrequest.go

orto17 · 2026-01-15T10:06:16Z

scanpullrequest/scanpullrequest.go

+	var err error
+
+	if isParallelScanEnabled() {
+		scanResults, err = auditBranchesInParallel(scanDetails, sourceBranchWd, targetBranchWd)


which type of errors can be returned here? isn't it make sense to check here the FailUponAnyScannerError() and continue to the filter function in case its false?

Idk, i kept it consistent with the original logic

scanpullrequest/scanpullrequest.go

orto17 · 2026-01-15T10:13:36Z

scanpullrequest/scanpullrequest.go

+	startTime := time.Now()
+
+	// Pre-download Analyzer Manager to avoid parallel download contention
+	if err := securityjas.DownloadAnalyzerManagerIfNeeded(0); err != nil {


i am not sure we want to have this analyzer manager logic in frogbot. the AM is a "cli problem", and in general we want to simplify frogbot and not giving it more responsibility. is there a way to avoid it?

not really
you are triggering cli in parallel

meaning jas-target and jas-source will try to download it if it doesn't exist

and sca-source will also try to install it for ca if CA is needed

worst case you are installing AM 3 times to scan

adding @attiasas to the disscussion
what do you think about the following solution: create a new public function in cli-security: PrepareEnvForScan() which will be called by frogbot once and will downloiad AM (and new-sca engine?)

We have similar thing for our tests in the reposiotry. can you access this?

https://github.com/jfrog/jfrog-cli-security/blob/main/tests/utils/integration/test_integrationutils.go#L534

adding @attiasas to the disscussion what do you think about the following solution: create a new public function in cli-security: PrepareEnvForScan() which will be called by frogbot once and will downloiad AM (and new-sca engine?)

new sca engine its fine it will be downloaded once anyway by cli , not our responsibility

orto17 · 2026-01-15T10:17:03Z

scanpullrequest/scanpullrequest.go

+	diffResults := results.UnifyScaAndJasResults(scaResult.source, jasDiff)
+
+	log.Info("Uploading results to platform...")
+	if _, err := output.UploadCommandResults(scanDetails.ServerDetails, utils.FrogbotUploadRtRepoPath, diffResults); err != nil {


was this operation exist in frogbot before ? (uploading the results to RT) or was it cli responsibility before?

cli
it uploaded after the second scan finished, and it knew it was in a diff context

now scans in parallel and it doesnt know its in diff really (other than sca)

so if i understand - it means that for scan-pr -> frogbot uploads results, but for scan-repo - cli uploads the results

orto17 · 2026-01-15T10:20:07Z

scanpullrequest/scanpullrequest.go

+	targetLogCollector := audit.NewLogCollector(log.GetLogger().GetLogLevel())
+	targetScanDetails.SetLogCollector(targetLogCollector)
+
+	targetResults = targetScanDetails.Audit(targetDir)


why are the Audit commands in runScaScans are not running in parallel but in run JasScans they are executed as a go routine?

because in sca , for diff:
We calculate the target sca sbom
delta it with source, during source scan
and then what is left we do sca scan and contextual when we scan the source

running the sca on target takes milliseconds becasue sbom calculation is almost immediate with the new engine

orto17 · 2026-01-15T10:22:30Z

scanpullrequest/scanpullrequest.go

-	if e != nil {
-		err = errors.Join(err, fmt.Errorf("failed to get issues for pull request. Error: %s", e.Error()))
+	log.Debug("Scanning source branch code...")
+	scanDetails.SetUploadCdxResults(true)


can you explain to me what is the purpose of this line scanDetails.SetUploadCdxResults(true) ? and in general why did you add SetUploadCdxResults(upload bool) ?

https://github.com/jfrog/frogbot/pull/1028/changes#r2693788773

evolved from the issue in this thread

in the past we sent results in scan repo always

and in sca pr only on the second scan, and we knew if we were the second scan because of value of DiffScan and , resultsToComapre see this:
SetUploadCdxResults(!sc.diffScan || sc.ResultsToCompare != nil)

now because scanning logic changed for scan pr,it became a lot more trivial to just send true or false to whether we need to upload results, than to let the function figure it out on its own

scanpullrequest/scanpullrequest.go

orto17 · 2026-01-15T10:29:13Z

scanpullrequest/scanpullrequest.go

+	scaResult := <-scaChan
+	jasResult := <-jasChan
+
+	if scaResult.target != nil && scaResult.target.GetStatusCodes().ScaScanStatusCode != nil {


there is no possible scenario where for example scaResult.target==nil and also err==nil ? because in this case there is no log at all

only case is maybe if sca is turned off , which is basically what im checking here
will validate

i updated the condition for this and sca considering all the edge cases (turned off by config, entitlements make jas not run)

eyalk007 · 2026-01-18T08:07:51Z

utils/utils.go

 	dependencySubmissionFrogbotDetector = "JFrog Frogbot"
 	frogbotUrl                          = "https://github.com/jfrog/frogbot"
-	frogbotUploadRtRepoPath             = "frogbot"
+	FrogbotUploadRtRepoPath             = "frogbot"


will move it up

eyalk007 · 2026-01-18T08:08:10Z

utils/scandetails.go

+	}
+}
+
+// CloneForBranchScan creates a clone configured for branch scanning with isolated logging.


ill proably remove comment say if its clear

eyalk007 added 4 commits January 11, 2026 20:55

Update go.mod/go.sum

6db8794

Update to use ReplayTo() API for log capture

566a025

Remove duplicate 'Starting parallel' log

3ae1cd5

eyalk007 requested a deployment to frogbot January 13, 2026 13:16 — with GitHub Actions Waiting

eyalk007 added the new feature Automatically generated release notes label Jan 13, 2026

eyalk007 self-assigned this Jan 13, 2026

Revert unnecessary utils.go changes (keep only parallel logging)

70c6a7d

eyalk007 requested a deployment to frogbot January 13, 2026 13:21 — with GitHub Actions Waiting

Add JF_PARALLEL_PR_SCAN flag and clean up comments

ec13559

- Add JF_PARALLEL_PR_SCAN env var (default: true, set to 'false' for sequential mode) - Remove verbose/redundant comments - Remove gradle cache files from tracking

eyalk007 requested a deployment to frogbot January 13, 2026 13:30 — with GitHub Actions Waiting

Revert .gitignore change

5bdc988

eyalk007 requested a deployment to frogbot January 13, 2026 13:32 — with GitHub Actions Waiting

Clean up parallel scan flag check

6b3ac77

eyalk007 requested a deployment to frogbot January 13, 2026 13:34 — with GitHub Actions Waiting

Refactor parallel scan: return both target/source, unify post-process…

166093a

…ing, support FailUponAnyScannerError

eyalk007 requested a deployment to frogbot January 13, 2026 13:57 — with GitHub Actions Waiting

Restore debug logs

e1f2480

eyalk007 requested a deployment to frogbot January 13, 2026 13:59 — with GitHub Actions Waiting

Remove outdated comment - both modes now set ResultsToCompare

9d3ad04

eyalk007 requested a deployment to frogbot January 13, 2026 14:01 — with GitHub Actions Waiting

Rename unifiedSource to diffResults, use FrogbotUploadRtRepoPath cons…

349f070

…t, remove comments

eyalk007 requested a deployment to frogbot January 13, 2026 14:09 — with GitHub Actions Waiting

Update log message: uploading results to platform

4dc8f5f

eyalk007 requested a deployment to frogbot January 13, 2026 14:11 — with GitHub Actions Waiting

Simplify uploadCdxResults: explicit bool instead of inference logic

18687dd

eyalk007 requested a deployment to frogbot January 13, 2026 14:18 — with GitHub Actions Waiting

Refactor: separate scanning from issue conversion

acbc0a3

- scanPullRequestBranches: scanning + partial results filtering - auditPullRequestAndReport: output flags + issue conversion

eyalk007 requested a deployment to frogbot January 14, 2026 08:46 — with GitHub Actions Waiting

orto17 reviewed Jan 15, 2026

View reviewed changes

utils/scandetails.go Show resolved Hide resolved

orto17 reviewed Jan 15, 2026

View reviewed changes

utils/scandetails.go Show resolved Hide resolved

orto17 reviewed Jan 15, 2026

View reviewed changes

go.mod Outdated Show resolved Hide resolved

CR fixes - remove unused RootDir constant, update go.mod replace dire…

c8e9cb2

…ctives

orto17 reviewed Jan 15, 2026

View reviewed changes

scanpullrequest/scanpullrequest.go Outdated Show resolved Hide resolved

eyalk007 requested a deployment to frogbot January 15, 2026 10:03 — with GitHub Actions Waiting

orto17 reviewed Jan 15, 2026

View reviewed changes

scanpullrequest/scanpullrequest.go Outdated Show resolved Hide resolved

orto17 reviewed Jan 15, 2026

View reviewed changes

CR fixes

3f8c785

eyalk007 requested a deployment to frogbot January 15, 2026 10:15 — with GitHub Actions Waiting

orto17 reviewed Jan 15, 2026

View reviewed changes

scanpullrequest/scanpullrequest.go Show resolved Hide resolved

orto17 reviewed Jan 15, 2026

View reviewed changes

CR fixes - add CloneForBranchScan, rename to parallelJasResult

4a719b3

eyalk007 requested a deployment to frogbot January 15, 2026 11:26 — with GitHub Actions Waiting

CR fixes - improve JAS execution check, remove unnecessary nil checks

e63c667

eyalk007 requested a deployment to frogbot January 15, 2026 13:12 — with GitHub Actions Waiting

eyalk007 commented Jan 18, 2026

View reviewed changes

added some logs for error cases of no logs for scans in cli

e9ed44b

eyalk007 requested a deployment to frogbot January 18, 2026 08:54 — with GitHub Actions Waiting

flag product fixes

d892c5d

eyalk007 requested a deployment to frogbot January 18, 2026 09:41 — with GitHub Actions Waiting

CR fixes: rename UnifyScaAndJasResults to MergeScaAndJasResults

b438258

eyalk007 requested a deployment to frogbot January 18, 2026 10:19 — with GitHub Actions Waiting

temp fix for env var

7bc63fe

eyalk007 requested a deployment to frogbot January 19, 2026 10:16 — with GitHub Actions Waiting

Parallelize the scan pr flow #1028

Are you sure you want to change the base?

Parallelize the scan pr flow #1028

Conversation

eyalk007 commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eyalk007 commented Jan 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

orto17 Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eyalk007 Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eyalk007 Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

eyalk007 commented Jan 13, 2026 •

edited

Loading

orto17 Jan 15, 2026 •

edited

Loading

eyalk007 Jan 15, 2026 •

edited

Loading

eyalk007 Jan 15, 2026 •

edited

Loading