Fake ratelimit?

[ButteredCats]

Been running into an issue where all my accounts suddenly get ratelimited with something like Fetch error, API: /i/api/graphql/bshMIjqDk8LTXTq4w91WKw/SearchTimeline, errors: (errors: @[(code: rateLimited, message: "Rate limit exceeded.")]), except the logs still show I have requests left and restarting will temporarily make all accounts work again.

Sometimes changing the value of disableTid fixes this, and other times it doesn't.

Seeing that restarting kept temporarily working I decided to just comment out session.limited = true and session.limitedAt = epochTime().int in src/auth.nim (line 181 and 182) to see if that'd let the accounts continue to work.

Doing so still gives me a few occasional failures but it does keep the instance usable, so it seems like most of the accounts aren't actually ratelimited despite the error messages?

This would effect both my servers at the exact same time until I started using a different set of accounts on each. Different IPs, one in the US one in Germany. This was also happening significantly more often when both servers shared the same set of accounts.

Using a spare account I also noticed that it'd work fine from my home network, and it would then stop working at home shortly after I added it into either of the servers when they were having this issue.

[zedeus]

This is related to the completely opaque 24 hour rate limit introduced within the last year or two. The behavior you're seeing is mostly known about, except the difference in behavior when sharing vs not sharing across two servers, and home vs non-home network note at the end. If you could do some more experiments like that, that would be greatly appreciated!

disableTid authenticates in a different less officially supported way, and only really counts as a temporary workaround. I can explain further if you DM me on Matrix.

[ButteredCats]

I'll do some more testing next time it happens! For now I set both servers to share only a few accounts. My guess is that when one server goes under, those shared accounts will be ratelimited on the other server too but the unshared accounts will be fine.

If you can show me what a curl command to use the API would look like or what headers I need to send I can do some testing with that too. It strikes me as odd that removing Nitter's ratelimit check makes things mostly continue to work despite all the errors it still returns, so I'm wondering if the API returns a non 200 status code but still contains the actual content. I'm not very familiar with Nitter's codebase though so maybe that isn't even a possible scenario.

Some more info that might be worth noting:

• Using a proxy didn't fix it once the ratelimit had occurred.
• Signing into the accounts continues to return posts, I don't get the message about having viewed too much.

[ButteredCats]

Already managed to happen again 😭.

All accounts on the US server got ratelimited, including the ones being shared between the US and DE servers. The DE server does show that those accounts aren't working. All non shared accounts on the DE server are fine.

Trying to retest with the spare account, I noticed that it isn't being marked as ratelimited, nor am I getting any rate limit messages for it. Just a no sessions available for API message. Running Nitter at home with only this account shows I can make a single request before this starts happening.Restart, one request that works, all others fail. Restart and the cycle continues. The behavior with the restarts seems similar to the rest of the accounts. In the middle of writing this it just started behaving normally and working every single request, don't know why. These failed requests are instant, devtools shows only a 1ms wait time.

All my other accounts will show a ratelimited error message and appear as such in .health though. The spare account is a much newer account, maybe that has something to do with why that behaves differently?

After the initial ratelimiting on the US server I grabbed another account from the working DE server and put it into the US server, where it immediately began returning rate limited by api messages despite not doing so on the DE server. It didn't seem to get ratelimited on the DE server until I restarted Nitter.

[ButteredCats]

Thanks to @cmj and this script he made I was able to do some testing with curl!

So I did a few requests in a row with one account, and got this:

• Ratelimited
• Ratelimited
• Working
• Ratelimited
• Working

And doing more requests continued this random mix of working and ratelimited requests. During this the request limit was not at zero and with each request, both working and ratelimited, it would go down by one.

The ratelimited requests do indeed return HTTP 429 with nothing but {"errors":[{"code":88,"message":"Rate limit exceeded."}]} so my thought about it still returning the content is wrong. However, this does explain why removing Nitter's ratelimit checks makes my instance (mostly) work again!

Even after the limit reset this mix of requests continued. Running out the limit didn't result in anything interesting.

Doing this from a different IP had no change either, both on an account that was working normally and one that was giving a mix of working and ratelimited responses.

Running out the ratelimit between two different IPs didn't create any issues either, and when the ratelimit reset on that account it still behaved normally afterwards.