Prevent search engines to index v1-* domains of our site

[ItalyPaleAle]

For example, see this search result:

Google is recommending results from v1-7.docs.dapr.io and even v1-1.docs.dapr.io. We should add a robots.txt file to non-prod sites to prevent search engines to index them.

Alternatively, we should add a canonical URL that always points to the prod page, so search engines know that the content in v1-* is duplicate.