Skip to content

feat(http): respect annotations in unsafe mode#7044

Open
dwisiswant0 wants to merge 1 commit intodevfrom
dwisiswant0/feat/http/respect-raw-annotations-in-unsafe-mode
Open

feat(http): respect annotations in unsafe mode#7044
dwisiswant0 wants to merge 1 commit intodevfrom
dwisiswant0/feat/http/respect-raw-annotations-in-unsafe-mode

Conversation

@dwisiswant0
Copy link
Member

@dwisiswant0 dwisiswant0 commented Feb 26, 2026

Proposed changes

Apply request annotations during unsafe raw
request generation so @Host/@timeout/etc affect
the effective target and execution context.

Strip leading annotation directives from unsafe
wire bytes before sending, since rawhttp
transmits raw payload verbatim and annotation
lines can produce malformed HTTP requests.

Close #6747

Proof

$ go test -v -run "^Test(MakeUnsafeRequestFromRawWithHostAnnotation|ParseUnsafeRequestStripsLeadingAnnotations|UnsafeWithFullURLAndQueryParams|RequestParseAnnotations(Timeout|SNI)|UnsafeWith(FullURLAndPath|HTTPSFullURL)|UnsafeWithFullURL)$" ./pkg/protocols/http ./pkg/protocols/http/raw
=== RUN   TestMakeUnsafeRequestFromRawWithHostAnnotation
--- PASS: TestMakeUnsafeRequestFromRawWithHostAnnotation (0.04s)
=== RUN   TestRequestParseAnnotationsSNI
=== RUN   TestRequestParseAnnotationsSNI/compliant-SNI-value
=== RUN   TestRequestParseAnnotationsSNI/non-compliant-SNI-value
--- PASS: TestRequestParseAnnotationsSNI (0.00s)
    --- PASS: TestRequestParseAnnotationsSNI/compliant-SNI-value (0.00s)
    --- PASS: TestRequestParseAnnotationsSNI/non-compliant-SNI-value (0.00s)
=== RUN   TestRequestParseAnnotationsTimeout
=== RUN   TestRequestParseAnnotationsTimeout/positive
=== RUN   TestRequestParseAnnotationsTimeout/large-timeout
=== RUN   TestRequestParseAnnotationsTimeout/below-cap-timeout
=== RUN   TestRequestParseAnnotationsTimeout/negative
--- PASS: TestRequestParseAnnotationsTimeout (0.00s)
    --- PASS: TestRequestParseAnnotationsTimeout/positive (0.00s)
    --- PASS: TestRequestParseAnnotationsTimeout/large-timeout (0.00s)
    --- PASS: TestRequestParseAnnotationsTimeout/below-cap-timeout (0.00s)
    --- PASS: TestRequestParseAnnotationsTimeout/negative (0.00s)
PASS
ok  	github.com/projectdiscovery/nuclei/v3/pkg/protocols/http	0.207s
=== RUN   TestParseUnsafeRequestStripsLeadingAnnotations
--- PASS: TestParseUnsafeRequestStripsLeadingAnnotations (0.00s)
=== RUN   TestUnsafeWithFullURL
--- PASS: TestUnsafeWithFullURL (0.00s)
=== RUN   TestUnsafeWithFullURLAndPath
--- PASS: TestUnsafeWithFullURLAndPath (0.00s)
=== RUN   TestUnsafeWithFullURLAndQueryParams
--- PASS: TestUnsafeWithFullURLAndQueryParams (0.00s)
=== RUN   TestUnsafeWithHTTPSFullURL
--- PASS: TestUnsafeWithHTTPSFullURL (0.00s)
PASS
ok  	github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/raw	0.140s

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Summary by CodeRabbit

Release Notes

  • New Features
    • Raw HTTP requests now support annotations for per-request configuration, including host overrides, request cancellation control, and Interactsh URL management.

Apply request annotations during `unsafe` raw
request generation so @Host/@timeout/etc affect
the effective target and execution context.

Strip leading annotation directives from `unsafe`
wire bytes before sending, since `rawhttp`
transmits raw payload verbatim and annotation
lines can produce malformed HTTP requests.

Close #6747

Signed-off-by: Dwi Siswanto <git@dw1.io>
@auto-assign auto-assign bot requested a review from Mzack9999 February 26, 2026 00:58
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 26, 2026

Walkthrough

The pull request adds support for request annotations in unsafe/raw HTTP mode. Changes include fetching annotation overrides, applying host and header adjustments, stripping leading annotation lines from raw requests, and merging annotation-provided interactsh URLs into the final unsafe request object.

Changes

Cohort / File(s) Summary
Unsafe Request Annotation Processing
pkg/protocols/http/build_request.go
Implements annotation override fetching for unsafe raw requests. Derives annotationURL from parsed raw request or base URL, fetches overrides via context-bound request, applies host/header adjustments, attaches cancellation functions, and merges interactsh URLs.
Raw Request Parsing
pkg/protocols/http/raw/raw.go
Introduces stripLeadingAnnotations helper to remove leading annotation lines (prefixed with @) from unsafe raw request bytes, ensuring annotations are processed separately from the actual HTTP request.
URL Resolution Logic
pkg/protocols/http/request.go
Updates unsafe/raw HTTP path to prioritize generatedRequest.rawRequest.FullURL over input URL when available, allowing annotation-derived URLs to override the original input URL.
Test Coverage
pkg/protocols/http/build_request_test.go, pkg/protocols/http/raw/raw_test.go
Adds tests verifying Host annotation override behavior and annotation line stripping in unsafe request parsing. Introduces urlutil import for URL handling.

Sequence Diagram

sequenceDiagram
    participant Client
    participant RawParser as Raw Parser
    participant AnnotationFetcher as Annotation Handler
    participant RequestBuilder as Request Builder
    participant Server

    Client->>RawParser: Parse unsafe raw request with annotations
    RawParser->>RawParser: Strip leading `@annotations`
    RawParser-->>Client: rawRequest (without annotations)
    
    Client->>RequestBuilder: Generate unsafe request
    RequestBuilder->>AnnotationFetcher: Fetch annotation overrides<br/>(if annotationURL available)
    AnnotationFetcher->>AnnotationFetcher: Apply Host header<br/>from original request
    AnnotationFetcher->>Server: Context-bound request
    Server-->>AnnotationFetcher: Annotation metadata
    AnnotationFetcher->>AnnotationFetcher: Extract cancelFunc<br/>& interactsh URLs
    AnnotationFetcher-->>RequestBuilder: Overrides
    
    RequestBuilder->>RequestBuilder: Apply host overrides
    RequestBuilder->>RequestBuilder: Merge interactsh URLs
    RequestBuilder->>RequestBuilder: Attach cancelFunc
    RequestBuilder-->>Client: Final unsafe request
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Annotations leap free,
In unsafe mode's decree,
Hosts override, URLs dance,
Raw requests now stand a chance!
~🔗✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(http): respect annotations in unsafe mode' directly and clearly summarizes the main change: enabling proper handling of annotations in unsafe HTTP request mode.
Linked Issues check ✅ Passed The PR successfully addresses issue #6747 by stripping annotation lines from unsafe raw bytes and applying per-request annotations including Host overrides, cancellation functions, and interactsh URLs.
Out of Scope Changes check ✅ Passed All changes directly support the stated objectives: annotation stripping, annotation-driven URL/Host overrides, timeout handling, and test coverage for unsafe mode with annotations.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dwisiswant0/feat/http/respect-raw-annotations-in-unsafe-mode

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/protocols/http/raw/raw.go`:
- Around line 268-283: stripLeadingAnnotations fails to detect annotation
directives that are indented (e.g., "  `@Host`: ..."); update the annotation check
in stripLeadingAnnotations to ignore leading whitespace before testing for "@"
so indented annotation lines are treated the same as non-indented ones.
Specifically, change the condition that calls stringsutil.HasPrefixAny on line
to instead call it on a left-trimmed version of line (trim spaces and tabs) so
lines starting with optional indentation followed by "@" are recognized and
skipped; keep the rest of the logic (requestLineFound, buffer writes, and
bufferPool handling) unchanged and only alter the annotation-detection
expression in stripLeadingAnnotations.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 72b6919 and 1cb3076.

📒 Files selected for processing (5)
  • pkg/protocols/http/build_request.go
  • pkg/protocols/http/build_request_test.go
  • pkg/protocols/http/raw/raw.go
  • pkg/protocols/http/raw/raw_test.go
  • pkg/protocols/http/request.go

Comment on lines +268 to +283
func stripLeadingAnnotations(request string) []byte {
reader := bufio.NewReader(strings.NewReader(request))
buffer := bufferPool.Get().(*bytes.Buffer)
buffer.Reset()
defer func() {
buffer.Reset()
bufferPool.Put(buffer)
}()

requestLineFound := false
for {
line, err := reader.ReadString('\n')
if len(line) > 0 {
if requestLineFound || !stringsutil.HasPrefixAny(line, "@") {
requestLineFound = true
_, _ = buffer.WriteString(line)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Handle indented annotation directives when stripping.

Annotation detection is currently prefix-only on the raw line. Inputs like " @host: ..." are retained, so unsafe raw bytes can still carry annotation text.

🔧 Proposed fix
diff --git a/pkg/protocols/http/raw/raw.go b/pkg/protocols/http/raw/raw.go
@@
-	if stringsutil.HasPrefixAny(s, "@") {
+	if stringsutil.HasPrefixAny(strings.TrimLeft(s, " \t"), "@") {
 		goto read_line
 	}
@@
-			if requestLineFound || !stringsutil.HasPrefixAny(line, "@") {
+			trimmed := strings.TrimLeft(line, " \t")
+			if requestLineFound || !stringsutil.HasPrefixAny(trimmed, "@") {
 				requestLineFound = true
 				_, _ = buffer.WriteString(line)
 			}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/protocols/http/raw/raw.go` around lines 268 - 283,
stripLeadingAnnotations fails to detect annotation directives that are indented
(e.g., "  `@Host`: ..."); update the annotation check in stripLeadingAnnotations
to ignore leading whitespace before testing for "@" so indented annotation lines
are treated the same as non-indented ones. Specifically, change the condition
that calls stringsutil.HasPrefixAny on line to instead call it on a left-trimmed
version of line (trim spaces and tabs) so lines starting with optional
indentation followed by "@" are recognized and skipped; keep the rest of the
logic (requestLineFound, buffer writes, and bufferPool handling) unchanged and
only alter the annotation-detection expression in stripLeadingAnnotations.

@neo-by-projectdiscovery-dev
Copy link

neo-by-projectdiscovery-dev bot commented Feb 26, 2026

Neo - PR Security Review

No security issues found

Highlights

  • Adds @host, @timeout, and @sni annotation support to unsafe mode HTTP requests
  • Implements annotation stripping from raw request bytes to prevent malformed HTTP
  • Includes timeout cap (5 minutes) to prevent DoS via excessive timeout values
Hardening Notes
  • Consider documenting that @host annotation is intended for trusted templates only, as it allows arbitrary target redirection (similar to self-contained requests)
  • The stripLeadingAnnotations function in raw/raw.go:268-296 correctly strips only leading annotations; consider adding a comment explaining why inline @ characters are preserved
  • In request_annotations.go:88-96, the @host parsing uses isHostPort validation; consider adding explicit documentation that this is intentional and not SSRF protection, as templates are trusted code

Comment @neo help for available commands. · Open in Neo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Support request annotations in unsafe mode

1 participant