Fix flaky specs

[solnic]

Issue Description

This issue gathers random failures we get from some flaky tests.

Failure 01

• Ruby 3.3

Failures:

  1) Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error captures client report for transaction event with span counts
     Failure/Error: expect(client.transport).to have_recorded_lost_event(:network_error, 'transaction')
       expected #<Sentry::HTTPTransport:0x00007fad0197b5e8 @logger=#<Logger:0x00007fad0197b868 @level=0, @progname=ni...nt_reports=true, @discarded_events={}, @last_client_report_sent=2025-01-24 10:01:40.480909663 +0000> to have recorded lost event :network_error and "transaction"
     # ./spec/sentry/client/event_sending_spec.rb:492:in `block (5 levels) in <top (required)>'
     # ./spec/sentry/client/event_sending_spec.rb:359:in `block (4 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'

  2) Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error captures client report for error event
     Failure/Error: expect(client.transport).to have_recorded_lost_event(:network_error, 'error')
       expected #<Sentry::HTTPTransport:0x00007fad00bd46b0 @logger=#<Logger:0x00007fad00bd48e0 @level=0, @progname=ni...nt_reports=true, @discarded_events={}, @last_client_report_sent=2025-01-24 10:01:41.297596915 +0000> to have recorded lost event :network_error and "error"
     # ./spec/sentry/client/event_sending_spec.rb:486:in `block (5 levels) in <top (required)>'
     # ./spec/sentry/client/event_sending_spec.rb:359:in `block (4 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'

  3) Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error swallows and logs Sentry::ExternalError (caused by transport's networking error)
     Failure/Error: expect(string_io.string).to match(/Event sending failed: Failed to open TCP connection/)

       expected "D, [2025-01-24T10:01:41.915649 #2632] DEBUG -- sentry: Initializing the Sentry background worker wit...entry: [Transport] Sending envelope with items [event] 14d88737df1a4ab7b8581c7189f963e2 to Sentry\n" to match /Event sending failed: Failed to open TCP connection/
       Diff:
       @@ -1,3 +1,5 @@
       -/Event sending failed: Failed to open TCP connection/
       +D, [2025-01-24T10:01:41.915649 #2632] DEBUG -- sentry: Initializing the Sentry background worker with 2 threads
       +D, [2025-01-24T10:01:41.915735 #2632] DEBUG -- sentry: Sentry HTTP Transport will connect to http://sentry.localdomain/
       +D, [2025-01-24T10:01:41.919898 #2632] DEBUG -- sentry: [Transport] Sending envelope with items [event] 14d88737df1a4ab7b8581c7189f963e2 to Sentry
     # ./spec/sentry/client/event_sending_spec.rb:471:in `block (5 levels) in <top (required)>'
     # ./spec/sentry/client/event_sending_spec.rb:359:in `block (4 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.3.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'

Finished in 12.55 seconds (files took 0.73758 seconds to load)
1076 examples, 3 failures, 1 pending

Failed examples:

rspec ./spec/sentry/client/event_sending_spec.rb:489 # Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error captures client report for transaction event with span counts
rspec ./spec/sentry/client/event_sending_spec.rb:483 # Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error captures client report for error event
rspec ./spec/sentry/client/event_sending_spec.rb:467 # Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events in background causes error swallows and logs Sentry::ExternalError (caused by transport's networking error)

Failure 02

• jruby / rack3

  1) Sentry::Client#event_from_exception sets threads interface without stacktrace
     Failure/Error: expect(thread[:name]).to eq("Thread 1")

       expected: "Thread 1"
            got: nil

       (compared using ==)
     # ./spec/sentry/client_spec.rb:370:in `block in <main>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/jruby/3.1.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/jruby/3.1.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/jruby/3.1.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/jruby/3.1.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block in setup'

Finished in 31.92 seconds (files took 4.12 seconds to load)

Failures 03

Ruby 2.6

Randomized with seed 4066

/home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport/http_transport.rb:69:in `rescue in send_data': end of file reached (Sentry::ExternalError)
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport/http_transport.rb:32:in `send_data'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport.rb:65:in `send_envelope'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport.rb:173:in `flush'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/client.rb:101:in `flush'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry-ruby.rb:271:in `close'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry-ruby.rb:245:in `block in init'
/opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/protocol.rb:225:in `rbuf_fill': end of file reached (EOFError)
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/protocol.rb:191:in `readuntil'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/protocol.rb:201:in `readline'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http/response.rb:40:in `read_status_line'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http/response.rb:29:in `read_new'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http.rb:1509:in `block in transport_request'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http.rb:1506:in `catch'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http.rb:1506:in `transport_request'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http.rb:1479:in `request'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/net/http.rb:35:in `request'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport/http_transport.rb:52:in `block in send_data'
	from /opt/hostedtoolcache/Ruby/2.6.10/x64/lib/ruby/2.6.0/net/http.rb:920:in `start'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport/http_transport.rb:49:in `send_data'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport.rb:65:in `send_envelope'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/transport.rb:173:in `flush'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry/client.rb:101:in `flush'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry-ruby.rb:271:in `close'
	from /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/lib/sentry-ruby.rb:245:in `block in init'
Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
Coverage report generated for RSpec to /home/runner/work/sentry-ruby/sentry-ruby/sentry-ruby/coverage/coverage.xml. 10539 / 10898 LOC (96.71%) covered.
/opt/hostedtoolcache/Ruby/2.6.10/x64/bin/ruby -I/home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.6.0/gems/rspec-core-3.13.2/lib:/home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.6.0/gems/rspec-support-3.13.2/lib /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.6.0/gems/rspec-core-3.13.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb  --exclude-pattern spec/isolated/\*\*/\*_spec.rb --order rand failed

Failure 03

Ruby 2.5

Failures:

  1) Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events inline causes error swallows and logs Sentry::ExternalError (caused by transport's networking error)
     Failure/Error: expect(string_io.string).to match(/Event sending failed: Failed to open TCP connection/)

       expected "D, [2025-02-04T10:36:10.837790 #2503] DEBUG -- sentry: config.background_worker_threads is set to 0,...nE, [2025-02-04T10:36:15.957837 #2503] ERROR -- sentry: Event capturing failed: execution expired\n" to match /Event sending failed: Failed to open TCP connection/
       Diff:
       @@ -1,5 +1,9 @@
       -/Event sending failed: Failed to open TCP connection/
       +D, [[2025](https://github.com/getsentry/sentry-ruby/actions/runs/13133532880/job/36643752347?pr=2537#step:6:2026)-02-04T10:36:10.837790 #2503] DEBUG -- sentry: config.background_worker_threads is set to 0, all events will be sent synchronously
       +D, [2025-02-04T10:36:10.837828 #2503] DEBUG -- sentry: Sentry HTTP Transport will connect to http://sentry.localdomain
       +D, [2025-02-04T10:36:10.840989 #2503] DEBUG -- sentry: [Transport] Sending envelope with items [event] 25d5c36b24e243d0a4ec19359f739bcd to Sentry
       +E, [2025-02-04T10:36:15.957715 #2503] ERROR -- sentry: Event sending failed: execution expired
       +E, [2025-02-04T10:36:15.957837 #2503] ERROR -- sentry: Event capturing failed: execution expired
     # ./spec/sentry/client/event_sending_spec.rb:466:in `block (5 levels) in <top (required)>'
     # ./spec/sentry/client/event_sending_spec.rb:387:in `block (4 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.5.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.5.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `loop'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.5.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.5.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/2.5.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'

Finished in 12.47 seconds (files took 0.51572 seconds to load)
1079 examples, 1 failure, 25 pending

Failed examples:

rspec ./spec/sentry/client/event_sending_spec.rb:463 # Sentry::Client integrated error handling testing with HTTPTransport #capture_event when sending events inline causes error swallows and logs Sentry::ExternalError (caused by transport's networking error)

Randomized with seed 59399

Failure 04

• jruby
• rack3

Failures:

  1) Sentry::Client#event_from_exception sets threads interface without stacktrace
     Failure/Error: expect(thread[:name]).to eq("Thread 1")

       expected: "Thread 1"
            got: nil

       (compared using ==)
     # ./spec/sentry/client_spec.rb:370:in `block in <main>'

Finished in 29.07 seconds (files took 4.18 seconds to load)
1099 examples, 1 failure, 51 pending

Failed examples:

rspec ./spec/sentry/client_spec.rb:355 # Sentry::Client#event_from_exception sets threads interface without stacktrace

Failure 05

• jruby

Failures:

  1) Sentry::Sidekiq::SentryContextClientMiddleware with user sets user of the current scope to the job
     Failure/Error: expect(queue.size).to be(1)

       expected #<Integer:3> => 1
            got #<Integer:7> => 3

       Compared using equal?, which compares object identity,
       but expected and actual are not the same object. Use
       `expect(actual).to eq(expected)` if you don't care about
       object identity in this example.
     # ./spec/sentry/sidekiq/sentry_context_middleware_spec.rb:173:in `block in <main>'

Finished in 5.96 seconds (files took 3.66 seconds to load)
70 examples, 1 failure, 1 pending

Failed examples:

rspec ./spec/sentry/sidekiq/sentry_context_middleware_spec.rb:170 # Sentry::Sidekiq::SentryContextClientMiddleware with user sets user of the current scope to the job

Randomized with seed 59324

Failure 06

• ruby 3.4

  1) Sentry::Configuration#profiler_class= sets the profiler class to StackProf when Vernier is not available
     # Skipping because one or more guards `[[:ruby_version?, :<, "3.2"]]` returned false
     # ./spec/sentry/configuration_spec.rb:718

Failures:

  1) Sentry::Client#event_from_transaction correct dynamic_sampling_context when head SDK
     Failure/Error:
       expect(event.dynamic_sampling_context).to eq({
         "environment" => "development",
         "public_key" => "12345",
         "sample_rate" => "1.0",
         "sampled" => "true",
         "transaction" => "test transaction",
         "trace_id" => transaction.trace_id
       })

       expected: {"environment" => "development", "public_key" => "12345", "sample_rate" => "1.0", "sampled" => "true", "trace_id" => "2173a1acc82d48478d2b4d33675c9e24", "transaction" => "test transaction"}
            got: {"environment" => "development", "public_key" => "12345", "sample_rate" => "0.5", "sampled" => "false", "trace_id" => "2173a1acc82d48478d2b4d33675c9e24", "transaction" => "test transaction"}

       (compared using ==)

       Diff:
       @@ -1,6 +1,6 @@
        "environment" => "development",
        "public_key" => "12345",
       -"sample_rate" => "1.0",
       -"sampled" => "true",
       +"sample_rate" => "0.5",
       +"sampled" => "false",
        "trace_id" => "2173a1acc82d48478d2b4d33675c9e24",
        "transaction" => "test transaction",
     # ./spec/sentry/client_spec.rb:264:in 'block (3 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.4.0/gems/webmock-3.25.0/lib/webmock/rspec.rb:39:in 'block (2 levels) in <top (required)>'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.4.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in 'block in RSpec::Retry#run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.4.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in 'RSpec::Retry#run'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.4.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in 'RSpec::Core::Example::Procsy#run_with_retry'
     # /home/runner/work/sentry-ruby/sentry-ruby/vendor/bundle/ruby/3.4.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in 'block (2 levels) in RSpec::Retry.setup'

Finished in 11.46 seconds (files took 0.76603 seconds to load)
1100 examples, 1 failure, 1 pending

Failed examples:

rspec ./spec/sentry/client_spec.rb:261 # Sentry::Client#event_from_transaction correct dynamic_sampling_context when head SDK

Randomized with seed 60504

Failure 07

• jruby

1) Sentry::Sidekiq::SentryContextClientMiddleware does not add user to the job if they're absent in the current scope
     Failure/Error: expect(queue.size).to be(1)

       expected #<Integer:3> => 1
            got #<Integer:9> => 4

       Compared using equal?, which compares object identity,
       but expected and actual are not the same object. Use
       `expect(actual).to eq(expected)` if you don't care about
       object identity in this example.
     # ./spec/sentry/sidekiq/sentry_context_middleware_spec.rb:161:in `block in <main>'

Finished in 6.1 seconds (files took 3.72 seconds to load)
70 examples, 1 failure

Failed examples:

rspec ./spec/sentry/sidekiq/sentry_context_middleware_spec.rb:[158](https://github.com/getsentry/sentry-ruby/actions/runs/13653741004/job/38168055672#step:7:159) # Sentry::Sidekiq::SentryContextClientMiddleware does not add user to the job if they're absent in the current scope

Randomized with seed 21411

Failure 08

• Ruby 3.1
• Rails 7.1

Failures:

  1) Sentry::Rails::Tracing with traces_sample_rate set records transaction alone
     Failure/Error: expect(second_span[:timestamp] - second_span[:start_timestamp]).to be_between(10.0 / 1_000_000, 10.0 / 1000)
       expected 0.018867015838623047 to be between 1.0e-05 and 0.01 (inclusive)
     # ./spec/sentry/rails/tracing_spec.rb:92:in `block (3 levels) in <top (required)>'

Finished in 30.82 seconds (files took 1.39 seconds to load)
152 examples, 1 failure, 7 pending

Failed examples:

rspec ./spec/sentry/rails/tracing_spec.rb:59 # Sentry::Rails::Tracing with traces_sample_rate set records transaction alone

Randomized with seed 41705

[sl0thentr0py]

@solnic can we close this now?

[solnic]

@sl0thentr0py not yet :( I fixed a bunch of those (and possibly reduced the risk of introducing new ones in the future) but under jruby there are at least two more flaky tests to fix.