-
Notifications
You must be signed in to change notification settings - Fork 60
Expand file tree
/
Copy path.lycheeignore
More file actions
31 lines (26 loc) · 1004 Bytes
/
Copy path.lycheeignore
File metadata and controls
31 lines (26 loc) · 1004 Bytes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# lychee URL exclusions for sites that block HEAD/cloud egress but work in real browsers.
# These are deliberately allowed in the README and not considered broken.
# OpenAI / X / Perplexity / Midjourney aggressively block automated requests
^https?://(www\.)?openai\.com($|/)
^https?://x\.ai($|/)
^https?://(www\.)?perplexity\.ai($|/)
^https?://(www\.)?midjourney\.com($|/)
^https?://(www\.)?ideogram\.ai($|/)
^https?://(www\.)?genspark\.ai($|/)
# Sites that 401/405 on HEAD even with a real UA
^https?://qwen\.ai
^https?://(www\.)?reuters\.com($|/)
^https?://aibusiness\.com
^https?://claude\.ai/security
^https?://(www\.)?researchgate\.net
# Geo-restricted but valid
^https?://(www\.)?sz\.gov\.cn
^https?://(www\.)?limitless\.ai
# Vendor / consumer sites that bot-block
^https?://(www\.)?samsung\.com
^https?://(www\.)?tesla\.com
^https?://(www\.)?microsoft\.com/en-us/microsoft-copilot
^https?://(www\.)?servicenow\.com
# Shields badges — they are images, not docs
shields\.io
img\.shields\.io