Skip to content

Commit 69182c1

Browse files
committed
we want to run scheduled jobs and delayed jobs in separate processes. we have observed major throughput issues when jobs are dynamically scheduled adversly impacting throughput of delayed jobs
1 parent 6d7c205 commit 69182c1

File tree

1 file changed

+72
-0
lines changed

1 file changed

+72
-0
lines changed

lib/resque/scheduler.rb

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,78 @@ class << self
3131
# allow user to set an additional failure handler
3232
attr_writer :failure_handler
3333

34+
# run with RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=delayed
35+
def run_delayed_only
36+
procline 'Starting Delayed'
37+
38+
# trap signals
39+
register_signal_handlers
40+
41+
# Quote from the resque/worker.
42+
# Fix buffering so we can `rake resque:scheduler > scheduler.log` and
43+
# get output from the child in there.
44+
$stdout.sync = true
45+
$stderr.sync = true
46+
47+
begin
48+
@th = Thread.current
49+
50+
# Now start the scheduling part of the loop.
51+
loop do
52+
begin
53+
handle_delayed_items if master?
54+
rescue Errno::EAGAIN, Errno::ECONNRESET, Redis::CannotConnectError => e
55+
log! e.message
56+
release_master_lock
57+
end
58+
poll_sleep
59+
end
60+
61+
rescue Interrupt
62+
log 'Exiting'
63+
end
64+
ensure
65+
release_master_lock
66+
end
67+
68+
# run with RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=scheduler
69+
def run_scheduled_only
70+
procline 'Starting Scheduler'
71+
72+
# trap signals
73+
register_signal_handlers
74+
75+
# Quote from the resque/worker.
76+
# Fix buffering so we can `rake resque:scheduler > scheduler.log` and
77+
# get output from the child in there.
78+
$stdout.sync = true
79+
$stderr.sync = true
80+
81+
# Load the schedule into rufus
82+
# If dynamic is set, load that schedule otherwise use normal load
83+
reload_schedule!
84+
85+
begin
86+
@th = Thread.current
87+
88+
# Now start the scheduling part of the loop.
89+
loop do
90+
begin
91+
update_schedule if master?
92+
rescue Errno::EAGAIN, Errno::ECONNRESET, Redis::CannotConnectError => e
93+
log! e.message
94+
release_master_lock
95+
end
96+
poll_sleep
97+
end
98+
99+
rescue Interrupt
100+
log 'Exiting'
101+
end
102+
ensure
103+
release_master_lock
104+
end
105+
34106
# Schedule all jobs and continually look for delayed jobs (never returns)
35107
def run
36108
procline 'Starting'

0 commit comments

Comments
 (0)