Scraping Crawlers in Python (Bs4, request) for HTML and VTEX
Technologies: Python 3, requests, BeautifulSoup (bs4)
What can i do with the current script?
The script recieve the URL of a product or a category in the market's website, detects if it is an individual product or a category of products.
Gets: Product name and price
Category
Gets: Name of visible products, price, image, description even if the picked url cames into a category (if the product has it), image url
Crawlers getting whole page with info.
Refreshes DataBase when using it.