NSC - Nested Sitemap Crawler
Recursively Crawls the Nested Sitemap information and generates the TXT file for all the important information in all sitemaps.
- update the valid sitemap on line no. 71
start_sitemap = "SITEMAP_URL"
- run the nsc.py file -
python nsc.py
- if sitemap url is valid then it will generate the domain.txt file in the same directory