Skip to content

Infrastructure: Update link-checker User-Agent string#3264

Merged
howard-e merged 2 commits into
mainfrom
link-checker-update-user-agent-string
May 20, 2025
Merged

Infrastructure: Update link-checker User-Agent string#3264
howard-e merged 2 commits into
mainfrom
link-checker-update-user-agent-string

Conversation

@howard-e

@howard-e howard-e commented May 5, 2025

Copy link
Copy Markdown
Contributor

Fix #3263


WAI Preview Link (Last built on Tue, 20 May 2025 13:59:09 GMT).

@howard-e howard-e changed the title Update link-checker User-Agent string Update link-checker User-Agent string May 5, 2025
@howard-e howard-e added the Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation label May 5, 2025
@howard-e howard-e changed the title Update link-checker User-Agent string Infrastructure: Update link-checker User-Agent string May 5, 2025

@gnarf gnarf left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good enough, and will fix the bug in the short term, but will probably break again in a few months.

I did a little research and ran into https://www.npmjs.com/package/user-agents - a package that someone has automated to collect the most popular user agents used on their own websites and it can generate "always up to date" user agents. We may want to look into using this package instead.

After a bit more research and a comment from @howard-e - https://github.com/DavideViolante/useragents-me-api/ looks like a much better choice

@mcking65

Copy link
Copy Markdown
Contributor

@howard-e

Given this is infra, I was waiting on you to merge.

@howard-e

howard-e commented May 20, 2025

Copy link
Copy Markdown
Contributor Author

This seems good enough, and will fix the bug in the short term, but will probably break again in a few months.

Certainly true.

I did a little research and ran into https://www.npmjs.com/package/user-agents - a package that someone has automated to collect the most popular user agents used on their own websites and it can generate "always up to date" user agents. We may want to look into using this package instead.

After a bit more research and a comment from @howard-e - https://github.com/DavideViolante/useragents-me-api/ looks like a much better choice

Not so much a fan for adding an extra lib for this small use case. Agreed on making this more future-proof, with the hardcoded string as a fallback. Fetching from the source site of that lib (specifically https://www.useragents.me/#most-common-desktop-useragents) is a good enough option to do in the future, to move this PR along.

Edit: tracking this in #3270

@howard-e howard-e merged commit 7b7d576 into main May 20, 2025
7 checks passed
@howard-e howard-e deleted the link-checker-update-user-agent-string branch May 20, 2025 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Infrastructure Related to maintaining task force and repo operations, processes, systems, documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

link-checker unexpectedly failing on paciellogroup.com

3 participants