Skip to content

Latest commit

 

History

33 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Puppeteer Based Website Crawler

Building Project

  1. Install NodeJS
  2. Run $ npm install to install dependencies
  3. Add a base url and DOM selector that both match your use case to ./config/default.js. Base url is a url for a webpage that contains a list of urls you would like to crawl
  4. Run $ npm run start to develop the project on a local machine
  5. Run $ npm run build to build the project and $ npm run serve to serve it

About

Website Crawler

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages