Write-up on how to index websites

[kishorenc]

[maciejmackowiak]

Any updates on those instructions? ;)

[jasonbosco]

Hi @maciejmackowiak! We actually just published a site scraper here.

Now it's based on Algolia's docsearch-scraper which was built for documentation sites, but I think you might be able to use it to scrape and index any website.

Would you be able to run it on your site and let me know if you notice any issues?