Skip to content

[BUG]: Datadog APM Not Propagating Traces for Sidekiq Pro Batch Callbacks #4485

Open
@sundars1995

Description

@sundars1995

Tracer Version(s)

2.12.1

Ruby Version(s)

ruby 3.1.6p260 (2024-05-29 revision a777087be6) [arm64-darwin24]

Relevent Library and Version(s)

sidekiq 6.0, sidekiq pro

Bug Report

We are integrating Datadog APM tracing into our Rails application, which makes heavy use of Sidekiq Pro batch jobs. The default Sidekiq tracing instrumentation works well for most jobs, but we are encountering issues with tracing Sidekiq Pro batch callbacks (on_successon_complete).

Observed Issue

  • When a Sidekiq batch completes, its callbacks (such as on_success and on_complete) are treated as separate traces rather than being linked to the parent trace.

  • This is especially problematic because our batch jobs have deep nesting, where callbacks can create new batches and trigger further callbacks.

  • The result is that the entire process is fragmented into multiple traces, making it difficult to correlate jobs under a single trace.

Expected Behavior

  • The callback jobs should inherit and propagate the parent batch trace, ensuring all related jobs are under a single distributed trace.

Reproduction Code

Reproduction Workflow

  1. parent job creates a Sidekiq Pro batch job.

  2. The batch job enqueues multiple nested jobs.

  3. Upon batch completion, an **on_complete** callback is triggered.

  4. This callback schedules another batch job, which follows the same pattern.

  5. Each batch's callback job is treated as a new trace, breaking the trace propagation.

Parent Job
   ├── Batch 1
   │      ├── Nested Job A
   │      ├── Nested Job B
   │      └── (on_complete Callback → Creates Batch 2)
   │
   ├── Batch 2
   │      ├── Nested Job C
   │      ├── Nested Job D
   │      └── (on_complete Callback → Creates Batch 3)
   │
   ├── Batch 3
   │      ├── Nested Job E
   │      ├── Nested Job F
   │      └── (on_complete Callback → ...)
   └── (on_complete Callback -> )

Configuration Block

Datadog.configure do |c|
  c.tracing.instrument :sidekiq, distributed_tracing: true
end

Error Logs

No response

Operating System

No response

How does Datadog help you?

Our company currently uses datadog extensively for logging and we are instrumenting APM traces along with RUM for distributed tracing for all our applications.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugInvolves a bugcommunityWas opened by a community member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions