Amazon has set its AI to overload websites

[addison74]

For a few days Amazon flooded my webservers logs. Here are a part of the actions added products to the cart, comparison, trying to create accounts, checkout process without finishing, search a long list of words and phrases. At this moment there are 40 established connections in a server. The behavior is different from a crawler it looks like a DoS.

If someone is facing with something like this, the fast solution is to block the following ranges of IP addresses. I am using Fail2Ban and IPset to catch every IP automatically then I use IPRange to join individual IP's in ranges.

47.128.16.0/24 .. 47.128.31.0/24

[Flyingmana]

thanks for the hint, might be worth a try to report it with some of the log entries
https://repost.aws/knowledge-center/report-aws-abuse

[addison74]

I followed your advice and reported the abuse. 24/7 the VPS have been accessed from IP addresses in the Amazon range, after our logs filled up quickly similar to a crawler, it was similar to a DoS, being 30 - 50 IPs simultaneously. I received a confirmation message that it is working on the case, but I had to block again between 47.128.32.0/24 - 47.128.63.0/24. It is actually a spider called Bytespider. I will block it the webserver too.

For the moment we blocked this range 47.128.0.0/18 and set in .htaccess BrowserMatchNoCase "Bytespider(.*) BAD_BOT. I I will make a rule in HAProxy to send all requests from these IPs to black hole.

[Flyingmana]

seems like blocking the user agent is the best one can do here :/
https://wordpress.org/support/topic/psa-bytedance-and-bytespider-bots-recommend-blocking/

And yeah, AWS might block the user, but they will then likely create new ones. Still always good to report it.

[addison74]

Besides the fact that it generates high traffic and uses hardware resources, this bot behaves like a human visitor. It adds products in the shopping cart, makes comparisons, filters, chooses the correct option of a configurable product.

[addison74]

I received a reply from Amazon, but it is embarrassing. First of all, they admit that there is a crawling activity carried out on their IP addresses. Secondly, the ByteSpider crawler has an email address mentioned in the User-Agent. If the crawler skips the robots.txt file then I should contact the developer, this is really funny. Thirdly, on the servers we created a long list of bots that we don't allow , including Bytedance and Byspider, but all requests don't take this file into account.

In conclusion, Amazon has customers who do real daily crawling, causing DoS and high bandwidth usage, but it appears to be fine for them.

[addison74]

Amazon allows abuse. For 24 hours I have a server that is subject to a tornado of accesses from IPs owned by Amazon. The same pattern 2 - 3 accesses then it stops. We have up to this moment 4050 IPs caught and blocked in the ISP's router and the number is increasing. Most likely we will permanently block entire ranges of addresses. I will complain to Amazon again, but I don't have high expectations. It is obvious that those who manage the products they sell are incompetent. In the present case there is a moron who has created an AWS network to abuse websites.

[kiatng]

Incompetent or corporate greed? Not sure how much one IP Amazon can profit, is tens of thousands of IPs a significant sum? Reminds me of casinos allowing loan sharks to operate in their premises, more profit for them.

[addison74]

If in the past they had to infect tens, hundreds or thousands of computers to take control, now with a few dollars anyone can create virtual machines with which they can launch attacks. Amazon is not interested to solve my problems, they have to earn from those who use their services. I was talking to an official and he was telling me that the competition has ended up using DDoS services to create problems for important online stores. As many depend on online commerce, a drop of a few days can seriously affect a business.

[addison74]

Today we changed the strategy. We redirect all requests coming from Amazon IP addresses to ... Amazon.com and to the senders. If they want to harvest a domain it would be best to do it at the source. The intensity of the attack decreased significantly.