Reduce GitHub API page request load by 20%

[chrisvanrun]

The GitHub api is hitting timeouts on requests. Reduce the requested page size from 100 to 80 to reduce the chance of hitting this.

[jmsmkn]

Have you checked that this would actually solve anything? I don't imagine that the user has admin access to more than 80 repos. If they do, this could then add in pagination which would slow it down further. I think this needs a more careful fix. Our view needs to return within 30s (maybe less). Might need a redesign.

[chrisvanrun]

Have you checked that this would actually solve anything? I don't imagine that the user has admin access to more than 80 repos. If they do, this could then add in pagination which would slow it down further. I think this needs a more careful fix. Our view needs to return within 30s (maybe less). Might need a redesign.

It is a bit hard to trace AFAIK since this requires 'fresh' tokens. From the docs on the GitHub API I deduced there are really only two ways to reduce the timeout chances: increase the timeout on our side to 10 seconds or reduce the number of queries. However, the number of page requests re-think things.

Having taken a second look, I think something else might be going on than simply being unlucky and hitting some bad rows in a query. The relevant sentry issue reports 10 (!) page requests. So >1.000 repositories are being queried and searched upon. Spot checking other events things failed at page 6, page 7, and page 9.

I find >1000 repo's hard to believe but not per-see impossible. Can't readily trace the specific installation and hence GitHub org / user.

[jmsmkn]

Okay - then as we're already paginating so much this won't do anything to help. The user will get a timeout before the error is hit. The whole thing needs a redesign.