Improve `robots.txt`

We currently allow everything and we should think what we really want to enable, since we use amplitude we should come up with a better robots.txt to discourage crawlers and AI scrapers