Modular Web Scraping Spider with Python

V 0.01

Modules

• Initialization

  • Imports
  • Browser selection

• Scrape module

  • Has to be written separately for each website

• Multi threading

  • Toggleable

• (Tkinter) GUI

  • Continue from last (auto fetch ID from file)
  • Settings
  • Displays and manipulates data

• File handling module

  • Easy saving to file

Features:

• Check if x amounts of queries have been made with current ip (ipgetter)