HTTP error handling in w3af

Exceptions

• Sub-classes of urllib2.URLError are raised by w3af when one HTTP request fails

• HTTPRequestException is raised by w3af when one HTTP request fails

• ScanMustStopException is raised by the extended_urllib.py when multiple HTTP requests fail in a row, potentially indicating that the remote server is unreachable.

• ScanMustStopByUserRequest is a subclass of ScanMustStopException which is raised when we want to stop the scan. This exception is raised by the extended_urllib.py only when the user clicks "stop" in the UI.

Code sections where exceptions are raised

Code sections where exceptions are handled

Retry

All urllib2 handlers (which are used by the extended_urllib.py module) raise exceptions and might have errors, but the only place where we retry to send an HTTP request is in ExtendedUrllib._retry.

Avoid retries in any other code section, since that might lead to "multiple retries":

• Your wants to send an HTTP request using ExtendedUrllib.send
• For some reason that request fails
• ExtendedUrllib._retry is called three times to retry sending the request
• The request still fails and a HTTPRequestException is raised
• Your code catches HTTPRequestException and re-sends the request. It does this in a loop, three times.
• The result is that the request was sent (or at least w3af tried to) 9 times.