Skip to content

Commit

Permalink
we want to run scheduled jobs and delayed jobs in separate processes.…
Browse files Browse the repository at this point in the history
… we have observed major throughput issues when jobs are dynamically scheduled adversly impacting throughput of delayed jobs
  • Loading branch information
taf2 committed Mar 19, 2017
1 parent 6d7c205 commit 69182c1
Showing 1 changed file with 72 additions and 0 deletions.
72 changes: 72 additions & 0 deletions lib/resque/scheduler.rb
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,78 @@ class << self
# allow user to set an additional failure handler
attr_writer :failure_handler

# run with RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=delayed
def run_delayed_only
procline 'Starting Delayed'

# trap signals
register_signal_handlers

# Quote from the resque/worker.
# Fix buffering so we can `rake resque:scheduler > scheduler.log` and
# get output from the child in there.
$stdout.sync = true
$stderr.sync = true

begin
@th = Thread.current

# Now start the scheduling part of the loop.
loop do
begin
handle_delayed_items if master?
rescue Errno::EAGAIN, Errno::ECONNRESET, Redis::CannotConnectError => e
log! e.message
release_master_lock
end
poll_sleep
end

rescue Interrupt
log 'Exiting'
end
ensure
release_master_lock
end

# run with RESQUE_SCHEDULER_MASTER_LOCK_PREFIX=scheduler
def run_scheduled_only
procline 'Starting Scheduler'

# trap signals
register_signal_handlers

# Quote from the resque/worker.
# Fix buffering so we can `rake resque:scheduler > scheduler.log` and
# get output from the child in there.
$stdout.sync = true
$stderr.sync = true

# Load the schedule into rufus
# If dynamic is set, load that schedule otherwise use normal load
reload_schedule!

begin
@th = Thread.current

# Now start the scheduling part of the loop.
loop do
begin
update_schedule if master?
rescue Errno::EAGAIN, Errno::ECONNRESET, Redis::CannotConnectError => e
log! e.message
release_master_lock
end
poll_sleep
end

rescue Interrupt
log 'Exiting'
end
ensure
release_master_lock
end

# Schedule all jobs and continually look for delayed jobs (never returns)
def run
procline 'Starting'
Expand Down

0 comments on commit 69182c1

Please sign in to comment.