When enable_rake_performance_instrumentation is on and a background-job
worker is started through a Rake task, the jobs it processes are recorded as
Rake tasks instead of background jobs. They are emitted under the rake
namespace rather than background_job, with the wrong action grouping, and in
collector mode they lose their distributed-tracing link back to the code that
enqueued them. Worse: because the worker reuses the Rake task's long-lived
transaction, every job shares that same transaction — same id and rake
namespace — instead of getting its own background_job transaction.
Why this matters
Two of the most common background-job libraries are started through Rake:
- Resque —
rake resque:work
- Delayed Job —
rake jobs:work
This is the normal, documented way to run these workers, so any app that runs
Resque or Delayed Job this way and has enable_rake_performance_instrumentation
enabled is affected. It's an easy combination to end up with — you turn on Rake
instrumentation to see your db:migrate and other tasks, and it quietly changes
how every job on those workers is reported.
The newer libraries (Sidekiq, Que, Shoryuken, Good Job, Solid Queue) ship their
own executables and are not affected.
What you see
Observed on a real app by running 20 Resque jobs through rake resque:work:
- Jobs appear in the Rake namespace instead of Background jobs — the
action is the job class (PerformanceJob#perform), grouped under Rake.
- Throughput is counted there — 20 occurrences under
rake — so the jobs
aren't lost, they're just filed under the wrong namespace (nothing under
Background jobs).
- But all 20 collapse to a single sample trace: because they reuse the Rake
task's one transaction, they share its id, so the backend keeps one trace for
all of them.
- Durations are meaningless — the reused transaction's clock spans far beyond a
single job (the kept sample showed a ~97s "job", ~57s mean) rather than the
jobs' real ~1s.
- In collector mode the job span is a
SERVER span rather than a CONSUMER
span, and for integrations that propagate trace context (e.g. Resque) the
link back to the enqueueing request is dropped — so distributed traces don't
connect across the enqueue → perform boundary.
What's happening (briefly)
Running a worker through Rake means the Rake task stays running for the whole
life of the worker. With Rake instrumentation on, AppSignal opens one
transaction for that task. Each job then tries to open its own background-job
transaction, but a transaction is already active, so it reuses the Rake one
instead — inheriting the rake namespace and discarding the job's own trace
context. The integration closes that transaction after each job (so it is sent
to the agent), but every job reuses the same underlying transaction, so the
agent receives one rake transaction id completed over and over rather than a
distinct background_job transaction per job. (Resque forks a process per job,
and each fork inherits the still-open Rake transaction, so this repeats for
every job it processes.)
Reproduction
Two failing specs are on the
rake-bg-job-transaction-collision
branch (based on wip-otel-rework). Each performs a job with a rake
transaction already active, mimicking the Rake-launched worker:
Both currently fail:
Resque, agent mode: expected namespace "background_job", got "rake"
Resque, collector mode: expected span kind :consumer, got :server (link dropped)
Delayed Job, agent: expected namespace "background_job", got "rake"
Delayed Job, collector: expected span kind :consumer, got :server
Possible directions
- Don't treat a long-running worker task as a Rake performance transaction.
These tasks have stable, gem-defined names, so they could be skipped by name.
Resque defines resque:work and resque:workers. Delayed Job defines
jobs:work and jobs:workoff.
- Let the background-job integrations always start their own transaction, even
when one is already active.
At minimum it's worth documenting that enable_rake_performance_instrumentation
(off by default) should not be enabled on Rake-launched worker processes.
When
enable_rake_performance_instrumentationis on and a background-jobworker is started through a Rake task, the jobs it processes are recorded as
Rake tasks instead of background jobs. They are emitted under the
rakenamespace rather than
background_job, with the wrong action grouping, and incollector mode they lose their distributed-tracing link back to the code that
enqueued them. Worse: because the worker reuses the Rake task's long-lived
transaction, every job shares that same transaction — same id and
rakenamespace — instead of getting its own
background_jobtransaction.Why this matters
Two of the most common background-job libraries are started through Rake:
rake resque:workrake jobs:workThis is the normal, documented way to run these workers, so any app that runs
Resque or Delayed Job this way and has
enable_rake_performance_instrumentationenabled is affected. It's an easy combination to end up with — you turn on Rake
instrumentation to see your
db:migrateand other tasks, and it quietly changeshow every job on those workers is reported.
The newer libraries (Sidekiq, Que, Shoryuken, Good Job, Solid Queue) ship their
own executables and are not affected.
What you see
Observed on a real app by running 20 Resque jobs through
rake resque:work:action is the job class (
PerformanceJob#perform), grouped under Rake.rake— so the jobsaren't lost, they're just filed under the wrong namespace (nothing under
Background jobs).
task's one transaction, they share its id, so the backend keeps one trace for
all of them.
single job (the kept sample showed a ~97s "job", ~57s mean) rather than the
jobs' real ~1s.
SERVERspan rather than aCONSUMERspan, and for integrations that propagate trace context (e.g. Resque) the
link back to the enqueueing request is dropped — so distributed traces don't
connect across the enqueue → perform boundary.
What's happening (briefly)
Running a worker through Rake means the Rake task stays running for the whole
life of the worker. With Rake instrumentation on, AppSignal opens one
transaction for that task. Each job then tries to open its own background-job
transaction, but a transaction is already active, so it reuses the Rake one
instead — inheriting the
rakenamespace and discarding the job's own tracecontext. The integration closes that transaction after each job (so it is sent
to the agent), but every job reuses the same underlying transaction, so the
agent receives one
raketransaction id completed over and over rather than adistinct
background_jobtransaction per job. (Resque forks a process per job,and each fork inherits the still-open Rake transaction, so this repeats for
every job it processes.)
Reproduction
Two failing specs are on the
rake-bg-job-transaction-collisionbranch (based on
wip-otel-rework). Each performs a job with araketransaction already active, mimicking the Rake-launched worker:
spec/lib/appsignal/integrations/resque_spec.rb— "with a rake transaction already active (rake-launched worker)"spec/lib/appsignal/integrations/delayed_job_plugin_spec.rb— sameBoth currently fail:
Possible directions
These tasks have stable, gem-defined names, so they could be skipped by name.
Resque defines
resque:workandresque:workers. Delayed Job definesjobs:workandjobs:workoff.when one is already active.
At minimum it's worth documenting that
enable_rake_performance_instrumentation(off by default) should not be enabled on Rake-launched worker processes.