Skip to content

Make robots.txt part of the boilerplate #344

@fe-lix-

Description

@fe-lix-

Is your feature request related to a problem? Please describe.
When creating a new website all pages under .page and .live have default robots.txt which is great because you don't want to have your development website indexed. But at the moment you create your BYOCDN integration, the default robots.txt is not served anymore if you have not added one until that point. This can have side effects where the website you are working on start being indexed by search engines.

Steps to reproduce:

  1. Start a new website
  2. Configure your CDN (example.com)
  3. Now on https://example.com you have no robots.txt

Describe the solution you'd like

  1. Have a robots.txt in the boilerplate with the same content
  2. Add Modifying the robots.txt as part of the launch checks

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions