Skip to content

Avoid race-condition when scheduling throttled jobs #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 5, 2025

Conversation

pezra
Copy link
Contributor

@pezra pezra commented Mar 18, 2025

When a job was throttled and the queue of pending jobs was fully drained between the throttle? decision and the delay calculation, the retry code would fail with "NoMethodError undefined method `last' for nil:NilClass". I don't see a way to write a test for this since it is very timing based and highly intermittent.

fixes #206

PS: I changed the appraisals file because install_if does all of the bundling work regardless of whether the condition evaluates to true or false. In fact, it fetches the packages list from gems.contribsys.com before it even checks the condition. If BUNDLE_GEMS__CONTRIBSYS__COM isn't set that fetch fails with an auth error.

Copy link

codecov bot commented Mar 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.31%. Comparing base (9483b5a) to head (40699db).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #208   +/-   ##
=======================================
  Coverage   98.30%   98.31%           
=======================================
  Files          19       19           
  Lines         473      474    +1     
  Branches       79       80    +1     
=======================================
+ Hits          465      466    +1     
  Misses          8        8           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pezra pezra force-pushed the schedule-retry-race-cond branch from 4f40ee6 to efdad7c Compare March 18, 2025 15:54
pezra added 2 commits March 18, 2025 10:04
install_if does all of the bundling work even if the condition is
false. That means it fetches the packages list from
gems.contribsys.com before it even checks the condition. If
BUNDLE_GEMS__CONTRIBSYS__COM isn't set it that fetch fails with an
auth error.
When a job was throttled and the queue of pending jobs was fully drained
between the throttle? decision and the delay calculation, the retry code
would fail with "NoMethodError undefined method `last' for nil:NilClass"
@pezra pezra force-pushed the schedule-retry-race-cond branch from efdad7c to 41ee01c Compare March 18, 2025 16:04
@ixti
Copy link
Owner

ixti commented May 5, 2025

Sorry took a while to get to this. Thanks for the fix.

@ixti ixti merged commit b865d40 into ixti:main May 5, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NoMethodError: undefined method 'last' for nil:NilClass
2 participants