Replace HTTP client with imroc/req/v3

[Marv963]

Problem
Colly’s default net/http client is easy for bot-protections to fingerprint (predictable TLS handshake, header order, etc.).

Proposal
Replace the default HTTP client in Colly with imroc/req v3. This gives users realistic browser fingerprints out of the box (via ImpersonateChrome()), full control over headers, HTTP/2/3 support, and other stealth features—without forking.

Under the hood, every call to the standard net/http client would be replaced by the req.Client. Example usage:

c := colly.NewCollector().ImpersonateChrome()

Proof-of-Concept
I’ve implemented this in a fork and have been scraping reliably for several weeks:
https://github.com/M-Horizon-GmbH/colly-with-req

Question
Would you be interested in merging this as a new feature-branch (or similar)? If so, I can prepare a clean PR

[lakpahana]

I don't think replacing Colly's default HTTP client is viable. I can think of two options:

1. Maintain a separate fork of Colly that uses req, and update it frequently from the main Colly repository.
2. Add the ability to Colly to change the HTTP client as needed.

[Marv963]

Thank you for your response, @lakpahana!
I’ll start by maintaining a separate fork for now.

[asciimoo]

While I agree with @lakpahana that replacing the default HTTP client could lead to numerous issues/questions, I also think that an ability to use other HTTP clients would be super useful. (A similar topic: #844)
As @lakpahana suggested, a more generic solution could work where the HTTP clients were interchangeable by communicating with Colly via an interface. This way anyone could plug-in their preferred HTTP client by implementing a small interface. Also, these "client connectors" could be independent repositories/packages.
What do you think about it @Marv963 ?

[Marv963]

Hi @asciimoo,

I like that idea to make HTTP clients swappable via a simple interface. It keeps Colly core clean and lets anyone drop in their preferred client

[asciimoo]

@Marv963 would you be interested to work on this?
HTTPDriver interface in https://github.com/gocolly/colly/pull/844/files#diff-404d2f8bb2274f4e2d29ffffbba2098cb65b86bc0cac2f02259483bcf04bcf9aR42 follows a similar idea.

[Marv963]

@asciimoo sure, I’d be happy to take this on and will try to implement it. Thank you!

[asciimoo]

@Marv963 awesome, thank you! Let me know if I can help.

[lakpahana]

@Marv963 Great! Let me know if you need any help.

[Bide-UK]

I've also actually a branch with these changes by chance!

[vit0-9]

I'm also offering my help. For either implementing a more advanced http client or using a headless browser!

[xetrics]

Any updates on this?