[Feature]: Optional Rotating Tor/HaProxy Backend?

[patricksthannon]

What is the problem this feature would solve?

Rate limiting is among one of the top issues for searxng/degoog users so I'm curious if a rotating multi tor proxy array would function as something that would help this.

What is the feature you are proposing to solve the problem?

Possibly include a custom built tor( w/ or without Haproxy) array as one of the store's transports or even built in as an option.

What alternatives have you considered?

https://github.com/trimstray/multitor

https://hub.docker.com/r/peterdavehello/tor-socks-proxy/

https://github.com/mfasanya/docker-tor-rotator

https://blog.databigbang.com/running-your-own-anonymous-rotating-proxies

[fccview]

Hey @patricksthannon
You can already using rotating proxies (in your list tor-socks-proxy for example would already work).

Settings -> server -> proxy

You can even set specific proxies per engine in case, if you configure any engine and toggle the advanced settings, this would allow for example to set google/startpage on residential vpns whilst all other engines on normal vpns to avoid being detected as bot instantly (startpage especially is REALLY aggressive banning ip ranges from famous vpns).

But yes, generally it'd be pretty easy to create a transport with rotating tor proxies in case that's not enough, are you up for trying to use the CLI to do it or would you rather me do it? The latter may take a while as there's quite a backlog of fairly pressing things to do ❤️

[patricksthannon]

Oh of course, yeah I tested using a single socks5 proxy via https://hub.docker.com/r/peterdavehello/tor-socks-proxy/ last night (which is incredibly light) and did have some minor issues with speed but it worked well. I'm going to test out a rotating single entry point and see how it goes.

No need to implement it yourself, totally understand the backlog 0_0, it was just a minor suggestion for ease of other users who have difficulties setting up multiple proxies.

Thanks for your response! Stoked for the new release with the searxng integration!