Description
Is there an existing issue for this?
- I have searched the existing issues.
Current Behavior
Variables set in flow protocol aren't visible most of the time from headless protocol.
Expected Behavior
Variables set in flow protocol should be visible from headless protocol every time.
Steps To Reproduce
Running nuclei template below in debug mode:
id: test
info:
name: test
author: test
variables:
varFromVariables: "works"
flow: |
set("varFromFlow", "works");
log("Running http protocol...")
http();
log("Running headless protocol...")
headless();
http:
- method: GET
path:
- "{{BaseURL}}?varFromFlow={{varFromFlow}}&varFromVariables={{varFromVariables}}"
headless:
- steps:
- action: navigate
args:
url: "{{BaseURL}}?varFromFlow={{varFromFlow}}&varFromVariables={{varFromVariables}}"
Returns this output:
❯ ./nuclei -u 'http://localhost:3000/' -t ./test.yaml -duc --headless --debug
__ _
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v3.3.8
projectdiscovery.io
[WRN] The current platform and privileged user will run the browser without sandbox
[WRN] Could not load template /home/r14s/d/nuclei/bin/test.yaml: field 'severity' is missing
[WRN] Found 1 templates with syntax warning (use -validate flag for further examination)
[INF] Current nuclei version: v3.3.8 (outdated)
[INF] Current nuclei-templates version: v9.9.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 46
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[JS] Running http protocol...
[INF] [test] Dumped HTTP request for http://localhost:3000/?varFromFlow=works&varFromVariables=works
GET /?varFromFlow=works&varFromVariables=works HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip
[DBG] [test] Dumped HTTP response http://localhost:3000/?varFromFlow=works&varFromVariables=works
HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 21
Cache-Control: no-store, must-revalidate
Content-Type: text/html; charset=utf-8
Date: Wed, 22 Jan 2025 23:29:05 GMT
Etag: "15-/6VXivhc2MKdLfIkLcUE47K6aH0"
Vary: Accept-Encoding
X-Powered-By: Next.js
Internal Server Error
[JS] Running headless protocol...
[INF] [test] Dumped Headless request for http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works
[DBG] navigate => http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works
[DBG] [test] Dumped Headless response for http://localhost:3000/
<html><head></head><body>Internal Server Error</body></html>
[INF] No results found. Better luck next time!
You can notice that http protocol has no problem with populating the correct variable values:
http://localhost:3000/?varFromFlow=works&varFromVariables=works
But the headless protocol encounters a problem with populating variables set in flow protocol:
http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works
I've also noticed while testing on turned off test target server that it works most of the time and populates the values correctly.
I say most of the time because it sometimes fails as for the case with turned on test target server.
Here is an example of two runs one after another:
❯ ./nuclei -u 'http://localhost:3000/' -t ./test.yaml -duc --headless --debug
__ _
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v3.3.8
projectdiscovery.io
[WRN] The current platform and privileged user will run the browser without sandbox
[WRN] Could not load template /home/r14s/d/nuclei/bin/test.yaml: field 'severity' is missing
[WRN] Found 1 templates with syntax warning (use -validate flag for further examination)
[INF] Current nuclei version: v3.3.8 (outdated)
[INF] Current nuclei-templates version: v9.9.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 46
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[JS] Running http protocol...
[INF] [test] Dumped HTTP request for http://localhost:3000/?varFromFlow=works&varFromVariables=works
GET /?varFromFlow=works&varFromVariables=works HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip
[JS] Running headless protocol...
[WRN] [test] Could not execute step: [:RUNTIME] got following errors while executing flow <- [:RUNTIME] failed to execute headless:1 protocol <- could not get html element: error occurred executing action: [:RUNTIME] could not navigate to url http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works <- navigation failed: net::ERR_CONNECTION_REFUSED; [:RUNTIME] failed to execute http:1 protocol <- [:RUNTIME] got err while executing http://localhost:3000/?varFromFlow=works&varFromVariables=works <- Get "http://localhost:3000/?varFromFlow=works&varFromVariables=works": cause="port closed or filtered" address=localhost:3000 chain="connection refused"
[INF] No results found. Better luck next time!
❯ ./nuclei -u 'http://localhost:3000/' -t ./test.yaml -duc --headless --debug
__ _
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v3.3.8
projectdiscovery.io
[WRN] The current platform and privileged user will run the browser without sandbox
[WRN] Could not load template /home/r14s/d/nuclei/bin/test.yaml: field 'severity' is missing
[WRN] Found 1 templates with syntax warning (use -validate flag for further examination)
[INF] Current nuclei version: v3.3.8 (outdated)
[INF] Current nuclei-templates version: v9.9.1 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 46
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[JS] Running http protocol...
[INF] [test] Dumped HTTP request for http://localhost:3000/?varFromFlow=works&varFromVariables=works
GET /?varFromFlow=works&varFromVariables=works HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.34
Connection: close
Accept: */*
Accept-Language: en
Accept-Encoding: gzip
[JS] Running headless protocol...
[WRN] [test] Could not execute step: [:RUNTIME] got following errors while executing flow <- [:RUNTIME] failed to execute http:1 protocol <- [:RUNTIME] got err while executing http://localhost:3000/?varFromFlow=works&varFromVariables=works <- Get "http://localhost:3000/?varFromFlow=works&varFromVariables=works": cause="port closed or filtered" address=localhost:3000 chain="connection refused"; [:RUNTIME] failed to execute headless:1 protocol <- could not get html element: error occurred executing action: [:RUNTIME] could not navigate to url http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works <- navigation failed: net::ERR_CONNECTION_REFUSED
[INF] No results found. Better luck next time!
First run:
http://localhost:3000/?varFromFlow={{varFromFlow}}&varFromVariables=works
Second run:
http://localhost:3000/?varFromFlow=works&varFromVariables=works
Smells like race condition.
Relevant log output
Environment
- OS: Arch Linux
- Nuclei: v3.3.8 stable/dev
- Go: go version go1.23.5 linux/amd64Anything else?
No response