Improve User-Agent to avoid rate limiting

[brightbyte]

Hi, I'm Daniel Kinzler, Principal Engineer at the Wikimedia Foundation. Your framework is showing up as one of the top clients for our APIs - nice work!

We are in the process of rolling out rate limiting for our APIs this week. When we do, requests from your framework are likely to get severely rate limited, because the User-Agent header it is sending is not compliant with the API usage guidelines. While it matches the format specified in the user agent policy, the problem is that the contact information in the User-Agent header points to the framework's Github page, instead of providing a way to contact the operator of the script. This means that all users of your framework will share a rate limit counter.

Please provide a mechanism that allows (ideally, requires) users of your framework to provider their own contact info, for inclusion in the User-Agent header. Alternatively, provide a way for users to authenticate with a wiki account so it can benefit from higher limits associated with that account.

If you have question, you can find my contact information at https://www.mediawiki.org/wiki/User:DKinzler_(WMF).

[martin-majlis]

Hi @brightbyte,
thanks for letting me know.
User agent is mandatory and required parameter provided by the user - https://github.com/martin-majlis/Wikipedia-API#importing

It was implemented as requested to fix this raised issue - #63.

In the past, package was downloaded few times per day, but then something has changed and it was downloaded 6M per day, and now it's back to 600k per day - https://clickpy.clickhouse.com/dashboard/wikipedia-api?min_date=2026-01-01&max_date=2026-04-27. So my hunch is, that this is caused by AI agents. Why I try different LLMs - https://chatgpt.com/share/69f0a558-0e30-8397-ab03-abcb1199dfb3 or https://gemini.google.com/share/3388f49484ec - they generate compliant user agent.

[brightbyte]

Ah, I see what is happening...

The format of the user-agent header is not quite right, so the wrong part of it is extracted and used as the contact information and rate limit key. We are seeing this:

So, e.g. WikipediaMCPServer/0.1.0 (https://github.com/rudra-ravi/wikipedia-mcp) (Wikipedia-API/0.14.1; https://github.com/martin-majlis/Wikipedia-API/). The problem is the second set of parentheses. The library information should not be in parentheses. The policy says:

The generic format is <client name>/<version> (<contact information>) <library/framework name>/<version> [<library name>/<version> ...]. Parts that are not applicable can be omitted.

So the correct user agent string would be WikipediaMCPServer/0.1.0 (https://github.com/rudra-ravi/wikipedia-mcp) Wikipedia-API/0.14.1; https://github.com/martin-majlis/Wikipedia-API/. That would allow us to extract the correct part as the contact info, namely https://github.com/rudra-ravi/wikipedia-mcp.

Is the second part of the user-agent header appended by your code? Can you change it so it doesn't put it into parentheses?

[stiermid]

Hi,

It seems like parentheses are added in:

Wikipedia-API/wikipediaapi/_http_client/base_http_client.py

Line 119 in c841584

default_headers["User-Agent"] = used_user_agent + " (" + USER_AGENT + ")"

which USER_AGENT is:

Wikipedia-API/wikipediaapi/_http_client/base_http_client.py

Lines 25 to 27 in c841584

	USER_AGENT = (
	"Wikipedia-API/" + __version_str__ + "; https://github.com/martin-majlis/Wikipedia-API/"
	)

[stiermid]

It turns out @martin-majlis made the corresponding changes in deb4dd6, but reverted in d271106