Commit 16234b5
authored
chore(robots): noindex for tiles (#1348)
## Problem
We have a rule setup on Cloudflare to prevent robots from indexing Tiles
links.
To reduce the reliance on Cloudflare we want to move the rule dependency
to our app.
## Solution
This is a fallback in case we need to turn off orange cloud in the event
of a Cloudflare outage.
**Frontend:**
Added `<meta name="robots" content="noindex,nofollow" />` to:
* Main tile viewer page
* Invalid/expired tile links
* Missing tile error page
**Backend:**
Created `robots-header.ts` - Express middleware that sets `X-Robots-Tag:
noindex, nofollow` header for all `/tiles/*` routes
Note: the meta tag should be present even if the route is invalid so
that all routes to `tiles/*` are not indexed.
## Tests
- [x] Build and run locally: npm run build && npm start
- [ ] Verify header on tiles routes: curl -I
http://localhost:8080/tiles/123 shows X-Robots-Tag: noindex, nofollow
- [ ] Verify no header on non-tiles routes: curl -I
http://localhost:8080/flows has no X-Robots-Tag
## Screenshots
**Tile**
<img width="1920" height="970" alt="Screenshot 2025-11-28 at 3 51 56 PM"
src="https://github.com/user-attachments/assets/636c406d-9b18-4b7a-aa8a-f397e365c5f3"
/>
**Invalid Tile link**
<img width="1920" height="1013" alt="Screenshot 2025-11-28 at 3 52
27 PM"
src="https://github.com/user-attachments/assets/f81f9446-8bca-4d45-ba28-91299efee06e"
/>
**View only link**
<img width="1920" height="968" alt="Screenshot 2025-11-28 at 3 54 50 PM"
src="https://github.com/user-attachments/assets/a995384e-529c-498e-b891-2ca6d8e4566d"
/>
**Revoked view only link**
<img width="1920" height="965" alt="Screenshot 2025-11-28 at 3 56 45 PM"
src="https://github.com/user-attachments/assets/07acea2f-5e2e-4f74-9615-4de9da488089"
/>1 parent cdf9f85 commit 16234b5
File tree
5 files changed
+67
-42
lines changed- packages
- backend/src
- helpers
- frontend/src/pages
- Tile
- UnauthorizedTile
5 files changed
+67
-42
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
53 | | - | |
| 54 | + | |
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| 102 | + | |
102 | 103 | | |
103 | 104 | | |
104 | 105 | | |
| |||
Lines changed: 48 additions & 41 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | 3 | | |
3 | 4 | | |
| |||
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
32 | 32 | | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
38 | 36 | | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
52 | 59 | | |
53 | 60 | | |
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
0 commit comments