Puppeteer Based Website Crawler
Building Project
- Install NodeJS
- Run
$ npm installto install dependencies - Add a base url and DOM selector that both match your use case to ./config/default.js. Base url is a url for a webpage that contains a list of urls you would like to crawl
- Run
$ npm run startto develop the project on a local machine - Run
$ npm run buildto build the project and$ npm run serveto serve it