Takes a significant amount of time check if URL is reachable

[souvikinator]

Before performing any fuzzing queries, gofuzz checks whether URL is reachable? Current state: It takes a significant amount of time to perform this function.

[caioeverest]

Hey, @souvikinator! Can you confirm to me that you are talking about the http.Head check?

gofuzz/pkg/data/options.go

Line 109 in 0e62517

_, err = http.Head(host)

I perform a few tests and this operation takes on average 3μs to execute. Are you sure this operation is the one that is taking too long to execute?

I did try a few alternatives (other methods and using a TCP dial directly to the address), but the Head execution seems to me to be the way to go. Maybe we could think into another strategy for parsing the url, it could take a few milliseconds from the runtime

[souvikinator]

Hey, @caioeverest yes I'm talking about http.Head in parseUrl() at least that was my initial suspect. I tried using different URLs now and it seems to works fine so I'm not sure what really took so much time back then, may be the host took long to respond. My bad I should have tested and then opened the issue.

[caioeverest]

No problem at all. I'm going to make a few more tests to try to see if I found where it is getting slow. Can you share with me the host that you used in your testings and notice this behavior?

[souvikinator]

sure, there you go: https://tyup-analytics.herokuapp.com/resource/get/default-profile-photo?key=<@>
It's from one of my projects and is hosted on Heroku , it takes a little bit of time on using this URL (probably because Heroku puts idle server to sleep).

However, I encountered another issue as well. When I query using -n=1000 I get an error (on n=650) but weird it worked on another try (can be seen below)

[caioeverest]

I opened a pull request (pr-#8) that will handle this kind of error and retry the request when it is necessary. Also, I took the liberty to add a time mark for us to know how long the process takes and a more fit transport configuration that helps a little bit in performance.