Skip to content

Commit 2ee51c4

Browse files
authored
fix(socket-export-sbom): run the SPDX export in JS to avoid bot-protection 403s (#2258)
* fix(socket-export-sbom): run the SPDX export in JS to avoid bot-protection 403s The export step was a `curl` loop that could not tell a 403 from a 404: it retried every non-200 for the full timeout and then reported the failure as "not ready". `curl` also sends no User-Agent, which we suspect trips the bot protection in front of the Socket API. Replace it with export-sbom.js, loaded by actions/github-script: - Retry only 404, 408, 429 and 5xx. Fail fast on 400/401/403 with a hint to check that the Socket API token carries the `report:read` scope. - Require a 200 body to parse as JSON before writing it, so a challenge page can never reach a later step as an SBOM. - Join the base URL and the endpoint path with exactly one separator, so a `socket_base_url` with or without a trailing slash resolves the same way. - Express the polling backoff in seconds, matching the `export_timeout_seconds` input. No input or output changed, so callers need no migration. * adds body snippet to error when invalid JSON is returned
1 parent c45c3eb commit 2ee51c4

3 files changed

Lines changed: 255 additions & 50 deletions

File tree

actions/socket-export-sbom/README.md

Lines changed: 23 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,29 @@ A good use case is including this sbom as part of a public repo's release artifa
88
99
## Inputs
1010

11-
| Name | Type | Description | Default Value | Required |
12-
| ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------- | -------- |
13-
| `socket_api_token` | `string` | API Key used to authenticate to socket.dev, requires the `full-scans:create` (for `socket scan create`) and `report:read` (for the SPDX export) scopes | `none` | true |
14-
| `socket_base_url` | `string` | Base URL of the socket api endpoint. | `"https://api.socket.dev/v0"` | false |
15-
| `socket_org` | `string` | Name of the socket org. | `"grafana"` | true |
16-
| `branch` | `string` | Branch to scan and export the SBOM for. The caller must have already checked out this branch's source tree before invoking this action, since the Socket CLI scans the local manifest files rather than reading a pre-existing scan. | `none` | true |
17-
| `output_file` | `string` | Name of the file to save the socket sbom on the runner. Defaults to `<repo>-<branch>.spdx.json`, e.g. `grafana-v1.2.3.spdx.json`. | `none` | false |
18-
| `export_timeout_seconds` | `string` | Max seconds to wait, with backoff, for the SBOM export to become available after scan creation. The full scan report is generated lazily by Socket, so raise this for repos larger than grafana/grafana. | `"180"` | false |
11+
| Name | Type | Description | Default Value | Required |
12+
| ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------ | -------- |
13+
| `socket_api_token` | `string` | API Key used to authenticate to socket.dev, requires the `full-scans:create` (for `socket scan create`) and `report:read` (for the SPDX export) scopes | `none` | true |
14+
| `socket_base_url` | `string` | Base URL of the socket api endpoint. Must end in a trailing slash: the Socket CLI appends endpoint paths to it without adding a separator. | `"https://api.socket.dev/v0/"` | false |
15+
| `socket_org` | `string` | Name of the socket org. | `"grafana"` | true |
16+
| `branch` | `string` | Branch to scan and export the SBOM for. The caller must have already checked out this branch's source tree before invoking this action, since the Socket CLI scans the local manifest files rather than reading a pre-existing scan. | `none` | true |
17+
| `output_file` | `string` | Name of the file to save the socket sbom on the runner. Defaults to `<repo>-<branch>.spdx.json`, e.g. `grafana-v1.2.3.spdx.json`. | `none` | false |
18+
| `export_timeout_seconds` | `string` | Max seconds to wait, with backoff, for the SBOM export to become available after scan creation. The full scan report is generated lazily by Socket, so raise this for repos larger than grafana/grafana. | `"180"` | false |
19+
20+
## Export behaviour
21+
22+
Socket generates the full scan report lazily, so the SPDX export endpoint returns a 404 for a while after `socket scan create` finishes. The export step ([`export-sbom.js`](./export-sbom.js)) polls with a 5s backoff, doubling to a 30s cap, until the report is ready or `export_timeout_seconds` elapses.
23+
24+
Not every failure is worth retrying, so the step separates the two cases:
25+
26+
| Response | Behaviour |
27+
| ------------------------------------------------ | --------------------------------------------------------------------- |
28+
| `200` with a parseable JSON body | Written to `output_file`, path published as the `path` output |
29+
| `200` with a body that does not parse | Retried — Socket can serve a partial report while the scan is running |
30+
| `404`, `408`, `429`, any `5xx`, network failures | Retried until the timeout |
31+
| `400`, `401`, `403`, any other `4xx` | Fails immediately — no amount of retrying adds a missing scope |
32+
33+
Only a body that parses as JSON is written to `output_file`, so a rejection page can never be mistaken for an SBOM by a later step.
1934

2035
## Examples
2136

actions/socket-export-sbom/action.yml

Lines changed: 12 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ inputs:
66
description: "Socket API token for authentication"
77
required: true
88
socket_base_url:
9-
description: "Socket base url"
9+
description: "Socket base url. Must end in a trailing slash: the Socket CLI appends endpoint paths to it without adding a separator, so a base of `https://api.socket.dev/v0` requests `/v0report/supported` and 404s."
1010
required: false
1111
default: "https://api.socket.dev/v0/"
1212
socket_org:
@@ -69,9 +69,13 @@ runs:
6969
echo "full_scan_id=$FULL_SCAN_ID" >> "$GITHUB_OUTPUT"
7070
echo "repo_name=$REPO_NAME" >> "$GITHUB_OUTPUT"
7171
72+
# The export runs in JS rather than curl: curl sends no User-Agent, which we
73+
# suspect trips the bot protection in front of the Socket API, and the shell
74+
# loop could not tell a 403 from a 404 so it retried both for the full
75+
# timeout. See export-sbom.js.
7276
- name: Export SPDX SBOM from Socket.dev
7377
id: export-sbom
74-
shell: bash
78+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
7579
env:
7680
SOCKET_API_TOKEN: ${{ inputs.socket_api_token }}
7781
SOCKET_BASE_URL: ${{ inputs.socket_base_url }}
@@ -82,43 +86,9 @@ runs:
8286
OUTPUT_FILE: ${{ inputs.output_file }}
8387
ACTION_PATH: ${{ github.action_path }}
8488
TIMEOUT_SECONDS: ${{ inputs.export_timeout_seconds }}
85-
run: |
86-
if [[ -n "$OUTPUT_FILE" ]]; then
87-
# Extract basename in case a path was provided
88-
OUTPUT_FILE=$(basename "$OUTPUT_FILE")
89-
else
90-
# e.g. "grafana" + "feature/foo" -> "grafana-feature-foo.spdx.json"
91-
OUTPUT_FILE="${REPO_NAME}-${BRANCH//\//-}.spdx.json"
92-
fi
93-
DEST="$ACTION_PATH/$OUTPUT_FILE"
94-
URL="$SOCKET_BASE_URL/orgs/$SOCKET_ORG/export/spdx/$FULL_SCAN_ID"
95-
96-
# The full scan report is generated lazily by Socket, so this endpoint
97-
# can 404/empty for a while after `socket scan create` returns. Poll
98-
# with backoff until it's ready or TIMEOUT_SECONDS elapses.
99-
START=$(date +%s)
100-
DELAY=5
101-
ATTEMPT=0
102-
while true; do
103-
ATTEMPT=$((ATTEMPT + 1))
104-
HTTP_CODE=$(curl --silent --max-time 30 --output "$DEST" --write-out "%{http_code}" \
105-
--header "Authorization: Bearer $SOCKET_API_TOKEN" "$URL") || HTTP_CODE="curl_error"
106-
107-
if [[ "$HTTP_CODE" == "200" ]] && jq -e . "$DEST" >/dev/null 2>&1; then
108-
echo "SBOM ready after ${ATTEMPT} attempt(s)"
109-
break
110-
fi
111-
112-
ELAPSED=$(($(date +%s) - START))
113-
if ((ELAPSED >= TIMEOUT_SECONDS)); then
114-
echo "SBOM export not ready after ${ELAPSED}s (last HTTP $HTTP_CODE):" >&2
115-
cat "$DEST" >&2
116-
exit 1
117-
fi
118-
119-
echo "SBOM not ready yet (HTTP $HTTP_CODE), retrying in ${DELAY}s... (${ELAPSED}s/${TIMEOUT_SECONDS}s elapsed)"
120-
sleep "$DELAY"
121-
DELAY=$((DELAY * 2 > 30 ? 30 : DELAY * 2))
122-
done
123-
124-
echo "path=$DEST" >> "$GITHUB_OUTPUT"
89+
with:
90+
# ACTION_PATH, not GITHUB_ACTION_PATH: inside a `uses:` step the latter points
91+
# at github-script's own directory rather than this action's.
92+
script: |
93+
const exportSBOM = require(`${process.env.ACTION_PATH}/export-sbom.js`);
94+
await exportSBOM({ core });
Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
// Socket generates the full scan report lazily, so the SPDX export endpoint 404s for a
2+
// while after `socket scan create` returns. This polls until the report is ready.
3+
//
4+
// This replaces a bare `curl` loop that could not tell a 403 from a 404: it retried
5+
// every non-200 for the full timeout and then reported the failure as "not ready".
6+
// curl also sends no User-Agent, which we suspect trips bot protection in front of the
7+
// Socket API. Both are addressed here.
8+
9+
// `actions/github-script` loads this file with require(), so it has to stay CommonJS.
10+
/* eslint-disable @typescript-eslint/no-require-imports */
11+
const fsPromises = require("node:fs/promises");
12+
const path = require("node:path");
13+
/* eslint-enable @typescript-eslint/no-require-imports */
14+
15+
const REQUEST_TIMEOUT_SECONDS = 30;
16+
const INITIAL_DELAY_SECONDS = 5;
17+
const MAX_DELAY_SECONDS = 30;
18+
19+
// Enough of an error body to identify the failure without flooding the log.
20+
const BODY_SNIPPET_LIMIT = 500;
21+
22+
const USER_AGENT =
23+
"grafana-shared-workflows-socket-export-sbom (+https://github.com/grafana/shared-workflows)";
24+
25+
const REQUIRED_ENV = [
26+
"SOCKET_API_TOKEN",
27+
"SOCKET_BASE_URL",
28+
"SOCKET_ORG",
29+
"FULL_SCAN_ID",
30+
"REPO_NAME",
31+
"BRANCH",
32+
"ACTION_PATH",
33+
];
34+
35+
// Socket has not finished generating the report (404), is rate limiting us (429), or is
36+
// briefly unhealthy (5xx). Every other status -- 400, 401, 403 -- means the request
37+
// itself is wrong, and retrying only delays the failure until the timeout expires.
38+
const RETRYABLE_STATUSES = new Set([404, 408, 429]);
39+
40+
// Retrying cannot add a missing scope, and a bot challenge will not clear on its own.
41+
const AUTH_HINT =
42+
"Check that the Socket API token carries the `report:read` scope and that the request is not being blocked by bot protection.";
43+
44+
// Every duration in this file is in seconds, matching the `export_timeout_seconds`
45+
// input. setTimeout is the only place that needs milliseconds.
46+
function delay(seconds) {
47+
return new Promise((resolve) => setTimeout(resolve, seconds * 1000));
48+
}
49+
50+
function isRetryableStatus(status) {
51+
return RETRYABLE_STATUSES.has(status) || status >= 500;
52+
}
53+
54+
// exportUrl joins the base URL and the endpoint path with exactly one separator.
55+
// `socket_base_url` ends in a trailing slash -- the Socket CLI needs it, since it
56+
// appends endpoint paths without adding a separator of its own -- so the slash is
57+
// stripped here rather than assumed away: `https://api.socket.dev/v0//orgs/...` is
58+
// not the same path to Socket.
59+
function exportUrl({ baseUrl, org, scanId }) {
60+
const base = baseUrl.replace(/\/+$/, "");
61+
const scan = encodeURIComponent(scanId);
62+
63+
return `${base}/orgs/${encodeURIComponent(org)}/export/spdx/${scan}`;
64+
}
65+
66+
// outputFileName is the caller's file name, or `<repo>-<branch>.spdx.json` with slashes
67+
// in the branch flattened: "grafana" + "feature/foo" -> "grafana-feature-foo.spdx.json".
68+
function outputFileName({ outputFile, repo, branch }) {
69+
// basename, because the caller may pass a path but the file goes in the action dir.
70+
if (outputFile) {
71+
return path.basename(outputFile);
72+
}
73+
74+
return `${repo}-${branch.replaceAll("/", "-")}.spdx.json`;
75+
}
76+
77+
function bodySnippet(body) {
78+
const text = body.trim();
79+
if (!text) {
80+
return "";
81+
}
82+
83+
const shown =
84+
text.length > BODY_SNIPPET_LIMIT
85+
? `${text.slice(0, BODY_SNIPPET_LIMIT)}...`
86+
: text;
87+
88+
return `: ${shown}`;
89+
}
90+
91+
// fetchSBOM makes one attempt at the export. It returns the SBOM body on success, and
92+
// otherwise the reason the attempt failed and whether another attempt could succeed.
93+
async function fetchSBOM({ fetch, url, token }) {
94+
let response;
95+
96+
try {
97+
response = await fetch(url, {
98+
headers: {
99+
authorization: `Bearer ${token}`,
100+
accept: "application/json",
101+
"user-agent": USER_AGENT,
102+
},
103+
signal: AbortSignal.timeout(REQUEST_TIMEOUT_SECONDS * 1000),
104+
});
105+
} catch (error) {
106+
// A stalled, refused or reset connection is worth another attempt.
107+
return {
108+
retryable: true,
109+
reason:
110+
error.name === "TimeoutError"
111+
? `the request did not complete within ${REQUEST_TIMEOUT_SECONDS}s`
112+
: `the request failed: ${error.message}`,
113+
};
114+
}
115+
116+
const body = await response.text();
117+
118+
if (!response.ok) {
119+
return {
120+
retryable: isRetryableStatus(response.status),
121+
reason: `Socket returned ${response.status} ${response.statusText}${bodySnippet(body)}`,
122+
};
123+
}
124+
125+
// Socket has served a 200 holding a partial report while the scan is still being
126+
// generated, so the body has to parse before the export counts as done.
127+
try {
128+
JSON.parse(body);
129+
} catch (error) {
130+
return {
131+
retryable: true,
132+
reason: `Socket returned 200 with a body that is not valid JSON (${error.message})${bodySnippet(body)}`,
133+
};
134+
}
135+
136+
return { body };
137+
}
138+
139+
// exportSBOM writes the SPDX SBOM for FULL_SCAN_ID into the action directory and
140+
// publishes its location as the `path` output. It marks the step as failed when the
141+
// export cannot be retrieved.
142+
//
143+
// env, fetch, sleep, now and writeFile exist so tests can drive the polling loop.
144+
module.exports = async function exportSBOM({
145+
core,
146+
env = process.env,
147+
fetch = globalThis.fetch,
148+
sleep = delay,
149+
now = Date.now,
150+
writeFile = fsPromises.writeFile,
151+
}) {
152+
const missing = REQUIRED_ENV.filter((name) => !env[name]);
153+
if (missing.length > 0) {
154+
core.setFailed(`Missing required environment: ${missing.join(", ")}`);
155+
return;
156+
}
157+
158+
const timeoutSeconds = Number(env.TIMEOUT_SECONDS);
159+
if (!Number.isFinite(timeoutSeconds) || timeoutSeconds < 0) {
160+
core.setFailed(
161+
`export_timeout_seconds must be a non-negative number, got "${env.TIMEOUT_SECONDS}"`,
162+
);
163+
return;
164+
}
165+
166+
const url = exportUrl({
167+
baseUrl: env.SOCKET_BASE_URL,
168+
org: env.SOCKET_ORG,
169+
scanId: env.FULL_SCAN_ID,
170+
});
171+
172+
const dest = path.join(
173+
env.ACTION_PATH,
174+
outputFileName({
175+
outputFile: env.OUTPUT_FILE,
176+
repo: env.REPO_NAME,
177+
branch: env.BRANCH,
178+
}),
179+
);
180+
181+
// now() stays a millisecond clock (Date.now); the deadline it feeds is seconds.
182+
const deadlineSeconds = now() / 1000 + timeoutSeconds;
183+
let delaySeconds = INITIAL_DELAY_SECONDS;
184+
185+
for (let attempt = 1; ; attempt++) {
186+
const { body, retryable, reason } = await fetchSBOM({
187+
fetch,
188+
url,
189+
token: env.SOCKET_API_TOKEN,
190+
});
191+
192+
if (body !== undefined) {
193+
// Only a validated body reaches the destination, so a rejection page can never
194+
// be mistaken for an SBOM by a later step.
195+
await writeFile(dest, body);
196+
core.info(`SBOM ready after ${attempt} attempt(s)`);
197+
core.setOutput("path", dest);
198+
return;
199+
}
200+
201+
if (!retryable) {
202+
core.setFailed(`Failed to export the SBOM: ${reason}. ${AUTH_HINT}`);
203+
return;
204+
}
205+
206+
const remainingSeconds = deadlineSeconds - now() / 1000;
207+
if (remainingSeconds <= 0) {
208+
core.setFailed(
209+
`SBOM export not ready after ${attempt} attempt(s) and ${timeoutSeconds}s: ${reason}`,
210+
);
211+
return;
212+
}
213+
214+
// Never sleep past the deadline; the next check would only report the timeout.
215+
const waitSeconds = Math.min(delaySeconds, remainingSeconds);
216+
core.info(`SBOM not ready yet (${reason}); retrying in ${waitSeconds}s`);
217+
await sleep(waitSeconds);
218+
delaySeconds = Math.min(delaySeconds * 2, MAX_DELAY_SECONDS);
219+
}
220+
};

0 commit comments

Comments
 (0)