Added support for detecting bots through client-hints

[sanchezzzhak]

for matomo, you will need to add a new HTTP_X_CLIENT key
https://github.com/matomo-org/matomo/blob/d1eaaca1b7abdddea15ff8d1d8e2075b6f92c672/core/Http.php#L996-L1012

[!] this PR is worth viewing about when we have more bots through the xClient header.

[liviuconcioiu]

@sanchezzzhak I have something to add:

1. Can you also include the HTTP_FROM header? According to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/From this one is a must for any crawler. So, if this header is present it should be detected as Generic bot for example.
2. Here are some HTTP_FROM values that should be added, to be detected as specific bots:

googlebot(at)googlebot.com
bingbot(at)microsoft.com
[email protected]
[email protected]
[email protected]

[sanchezzzhak]

@liviuconcioiu Perhaps the implementation of HTTP_FROM should be added separately, since this PR is in limbo.
It would be good to have approximate statistics before implementing new features.

[sanchezzzhak]

ChatGPT support header HTTP_FROM;

From: gptbot(at)openai.com
User-Agent" Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)

[liviuconcioiu]

Here is a complete list of what I have so far:

bingbot(at)microsoft.com
googlebot(at)googlebot.com
gptbot(at)openai.com
[email protected]
[email protected]
[email protected]
TGVnaXRpbWF0ZSBsaW5rIHRyYWNrZXI=
[email protected]
[email protected]
[email protected]
pigafetta-bot(at)visual-seo.com
[email protected]
"<?=print(9347655345-4954366);?>"
[email protected]