feat: add support for AI crawl bots

[naorpeled]

Summary

• add descriptors for common AI crawlers (OpenAI, Anthropic, etc.)
• expose new bot names in constants maps for API consumers
• cover each bot with acceptance UA fixtures

[Copilot]

Pull request overview

This PR adds support for detecting AI crawler bots from major AI companies, enabling developers to identify and handle traffic from various AI web crawlers. The changes include adding regex patterns for bot detection, exposing new bot names through the constants API, and providing comprehensive test coverage.

• Added detection for 10 new AI crawler bots: GPTBot, ChatGPT-User, OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), Omgilibot, Diffbot, PerplexityBot, Perplexity-User, YouBot, and Meta-WebIndexer
• Renamed FacebookExternalHit from the previous "MetaWebCrawler" name for clarity
• Added acceptance test fixtures for all new bots with real User-Agent strings

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 14 comments.

File	Description
test/acceptance/useragentstrings.yml	Added test fixtures for 10 new AI crawler bots with expected parsing results; renamed MetaWebCrawler to FacebookExternalHit
src/parser-browsers.js	Added browser detection logic and version extraction for each new AI bot; renamed MetaWebCrawler to FacebookExternalHit
src/parser-platforms.js	Added platform detection patterns identifying bots and their vendor companies
src/constants.js	Added bot names to BROWSER_ALIASES_MAP and BROWSER_MAP for API consumers; updated FacebookExternalHit naming

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.