Commit 6a942b2
committed
Prioritize review fetching in cron scraper to prevent missing approvals
PROBLEM: PR #25042 and 115 other PRs were missing from the database because
the scraper was timing out before completing review fetching. The old flow was:
1. Fetch all open PRs from GitHub
2. Create/update PR records
3. Scrape CI checks for each PR (SLOW - times out here)
4. Fetch reviews for all PRs (never reached if timeout)
This meant new PRs had no approval data even though they existed on GitHub.
SOLUTION:
- Increased timeout from 12 to 13 minutes for more processing time
- Moved review fetching INSIDE the PR processing loop
- Reviews now fetched immediately after CI checks for each PR
- Even if scraper times out, all processed PRs will have approval data
IMPACT: Prevents "missing approval" issues like PR #25042 which had an
approval from PhilipDeFraties 3 days ago but showed "None" in dashboard.
Changes:
- scripts/render_cron_scraper_fixed.rb:245: max_runtime 12min -> 13min
- scripts/render_cron_scraper_fixed.rb:303-327: Added inline review fetching
- scripts/render_cron_scraper_fixed.rb:347-377: Removed redundant Step 51 parent 0dd2cf7 commit 6a942b2
1 file changed
Lines changed: 30 additions & 35 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
245 | | - | |
| 245 | + | |
246 | 246 | | |
247 | 247 | | |
248 | 248 | | |
| |||
300 | 300 | | |
301 | 301 | | |
302 | 302 | | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
303 | 329 | | |
304 | 330 | | |
305 | 331 | | |
| |||
317 | 343 | | |
318 | 344 | | |
319 | 345 | | |
| 346 | + | |
320 | 347 | | |
321 | | - | |
322 | | - | |
323 | | - | |
324 | | - | |
325 | | - | |
326 | | - | |
327 | | - | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
343 | | - | |
344 | | - | |
345 | | - | |
346 | | - | |
347 | | - | |
348 | | - | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | | - | |
353 | | - | |
| 348 | + | |
354 | 349 | | |
355 | 350 | | |
356 | 351 | | |
| |||
369 | 364 | | |
370 | 365 | | |
371 | 366 | | |
372 | | - | |
| 367 | + | |
373 | 368 | | |
374 | 369 | | |
375 | 370 | | |
| |||
0 commit comments