[BUG] Social media link preview not working

[Kefff]

Describe the bug

When using a shortened link in a social media post (Facebook, Twitter, Bluesky, ...), the link preview feature is unable to access the website the link is pointing to.

To Reproduce

Steps to reproduce the behavior:

1. Create a shortened URL pointing to a website ("https://www.reuters.com/" for example : https://spoo.me/i6mvkG) without "Block bots" using the API or website
2. Create a new post in Facebook / Twitter / Bluesky / ... and paste the URL
3. The link preview does not display SEO metadata for the original website.

When using the shortened URL in the Facebook debugger, the debugger gets a 403 error. https://developers.facebook.com/tools/debug/?q=https%3A%2F%2Fspoo.me%2Fi6mvkG
Using the a TinyURL link in the Facebook debugger works well and displays a 301 redirection to the origin website. https://developers.facebook.com/tools/debug/?q=https%253A%252F%252Ftinyurl.com%252F4ch2b8ec

Expected behavior

The link preview should display SEO metadata the same way it is displayed when using the long URL.

Screenshots

Example in BlueSky with a long URL:

Example in Bluesky with a shortened URL:

[Zingzy]

@Kefff Fixed in v2 #94

[Kefff]

Thx ! Looks good on Discord, Whatsapp and Bluesky. Not so much on X but I think its their cache. However in the Facebook debugger it gets a 403 error from spoo.me that indicates that facebookexternalhit should be whitelisted in robots.txt.

[Zingzy]

Thanks for testing this out, re-opening this. Will aim on fixing it completely by this week!

[Zingzy]

@Kefff I think this issue is more related to robots.txt being cached by facebook, we updated the robots.txt to allow all bots, and according to my knowledge, it takes facebook ~24 to 48 hours to crawl the new robots.txt... Platforms like discord never respected robots.txt, so they work flawless. Lets check on this again after a few days and see if this is automatically solved