Issue with anti-bot detection #1905
Unanswered
thomas-doerr-lexware
asked this question in
Forums - Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I updated today from version 0.6.2 to the newest 0.8.5. A lot of my test pages are working as before, but I noticed that for some URLs I get the error message "Blocked by anti-bot protection: Structural: no tag".
Pages that from my perspective are not running an anti-bot protection.
I noticed, that the problem started with version 0.7.0.
And after a few checks, my assumption is, that Wordpress pages building with the theme Avada are facing this problem.
Since there are a lot of changes beween 0.6.2 and 0.8.5, can someone with a running system, crawl4ai 0.8.5, double check if the following websites can be scraped without any anti-bot protection problem?
I get for all of them the error message "Blocked by anti-bot protection: Structural: no tag" and I'm not sure, if I should create an issue since it is a bug or if I need to search in my code. I already tried to change my configuration, but without any change.
Thanks in advance!
Thomas Dörr
Beta Was this translation helpful? Give feedback.
All reactions