PyPI downloads shield showing "rate limited by upstream service"

[nightlark]

The PyPI Downloads badge seems to fairly often be showing up like this:

I think for other services the cache time was increased to address the issue.

It might also be nice to have an option to tell shields.io to return a 0 width image in case of an error -- since I'm using it in a place where it would be "nice to have" but if it frequently shows an error message than I'd rather just remove it entirely than make users see an error message.

[domdfcoding]

As well as bumping up the cache time, would returning stale data, rather than the error message, be better if upstream is rate limited? Does the user care that it's from yesterday, particularly for the monthly downloads?

[PyvesB]

Hey there 👋🏻

Thanks for raising.

would returning stale data, rather than the error message, be better if upstream is rate limited?

Shields.io is stateless, we don't have said cached data. Our caches are downstream, at the browser and CDN level, i.e. HTTP cache controls. If a request makes it through to our backend servers, we have no choice but to call the upstream pypistats API to get data.

I can increase the length of this caching, it will help a little. But ultimately, this won't resolve the problem, the rate limiting is very low, 5 per second and 30 per minute (source). If a Shields.io server gets 6 incoming requests at the same time for 6 different PyPI badges, one of them will fail. We can't control our user behaviours, and given this has become a popular badge with typically over a million of daily requests, statistically-speaking there will be failures no matter what caching we put in place.

Even our daily tests are regularly failing because of this strict rate limiting: https://github.com/badges/shields/actions/runs/20671953761

Worth noting that the rate-limiting is IP-based according to the API documentation. One thing that would help more than cache tweaks would be to scale Shields.io horizontally, so that the load is spread across more IPs. But we're not running on a pot of gold, and I was on the contrary considering scaling down our infra in the coming weeks to reduce our costs. The PyPI problem would likely get worse.

Many upstream services we integrate with allow for much higher API rate limits via an authenticated token. It may be worth opening an issue to see if the same thing could be put in place on the pypistats side.

[PyvesB]

Significantly increasing caching has helped, but as anticipated, some requests are still failing, typically a few hundred per day:

I've opened psf/pypistats.org#100 upstream.