[FEATURE REQUEST] Auto-detection of 404-like responses

[ocervell]

I know we have a --filter-similar-to flag that allows to do this, but is there a way to do it automatically ?

A.k.a make a request to http://example.com/<impossible_path> --> grab the sim hash and then filter out similar urls automatically later on ?

NB: maybe it should do the same for all subpaths lke http://example.com//<impossible_path> to detect all potential 404-like pages.

[epi052]

howdy!

thanks for the great suggestion! thankfully, this is already implemented 😁

if you're interested, check out this function.

What basically happens is this:

For each HTTP method and file extension, feroxbuster makes 6 requests with impossible/random paths:

Empty prefix with 1 UUID (32 chars): /{uuid}
Empty prefix with 3 UUIDs (96 chars): /{uuid}{uuid}{uuid}
.htaccess prefix with 1 UUID: /.htaccess{uuid}
.htaccess prefix with 3 UUIDs: /.htaccess{uuid}{uuid}{uuid}
admin prefix with 1 UUID: /admin{uuid}
admin prefix with 3 UUIDs: /admin{uuid}{uuid}{uuid}

The responses are grouped by status code and analyzed for common patterns (content-length, word count, line count)

When patterns are detected:

• WildcardFilter is created based on size/words/lines matching
• SimilarityFilter is created as a fallback using simhash
• Both filters are automatically applied to all subsequent responses

The detection runs for every directory, not just the base path, so it catches custom 404 pages at different path depths