Skip to content

Commit ac2c2c2

Browse files
hannahramadanhannahramadan
authored
Merge pull request #3405 from newrelic/parallel_instrumentation_processes
Parallel instrumentation
2 parents 0efdb72 + 5f1d521 commit ac2c2c2

File tree

11 files changed

+243
-0
lines changed

11 files changed

+243
-0
lines changed

‎.github/workflows/scripts/slack_notifications/supported_gems.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,4 @@ typhoeus
4848
unicorn
4949
view_component
5050
yajl-ruby
51+
parallel

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22

33
## dev
44

5+
- **Feature: Add support for forking processes in Parallel gem instrumentation**
6+
7+
Parallel gem instrumentation has been added to allow more consistent monitoring in processes forked using the Parallel gem. [PR#3405](https://github.com/newrelic/newrelic-ruby-agent/pull/3405)
8+
59
- **Feature: Add support for Grape v3.1.0**
610

711
Grape's release of v3.1.0 introduced changes that were incompatible with the agent's instrumentation, causing issues when collecting transaction names. The agent has been updated to properly extract class names for transaction naming in the updated Grape API structure. [PR#3413](https://github.com/newrelic/newrelic-ruby-agent/pull/3413)

‎lib/new_relic/agent/configuration/default_source.rb‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1510,6 +1510,15 @@ def self.convert_to_constant_list(string_array)
15101510
:allowed_from_server => false,
15111511
:description => 'Controls auto-instrumentation of bunny at start-up. May be one of: `auto`, `prepend`, `chain`, `disabled`.'
15121512
},
1513+
:'instrumentation.parallel' => {
1514+
:default => 'auto',
1515+
:documentation_default => 'auto',
1516+
:public => true,
1517+
:type => String,
1518+
:dynamic_name => true,
1519+
:allowed_from_server => false,
1520+
:description => 'Controls auto-instrumentation of the parallel library at start-up. May be one of `auto`, `prepend`, `chain`, `disabled`.'
1521+
},
15131522
:'instrumentation.aws_sdk_firehose' => {
15141523
:default => 'auto',
15151524
:documentation_default => 'auto',

‎lib/new_relic/agent/instrumentation/parallel.rb‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# This file is distributed under New Relic's license terms.
2+
# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
3+
# frozen_string_literal: true
4+
5+
require_relative 'parallel/instrumentation'
6+
require_relative 'parallel/chain'
7+
require_relative 'parallel/prepend'
8+
9+
DependencyDetection.defer do
10+
@name = :parallel
11+
12+
depends_on do
13+
defined?(Parallel) &&
14+
NewRelic::LanguageSupport.can_fork?
15+
end
16+
17+
executes do
18+
if use_prepend?
19+
prepend_instrument Parallel.singleton_class, NewRelic::Agent::Instrumentation::Parallel::Prepend
20+
else
21+
chain_instrument NewRelic::Agent::Instrumentation::Parallel::Chain
22+
end
23+
end
24+
end

‎lib/new_relic/agent/instrumentation/parallel/chain.rb‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# This file is distributed under New Relic's license terms.
2+
# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
3+
# frozen_string_literal: true
4+
5+
module NewRelic::Agent::Instrumentation
6+
module Parallel
7+
module Chain
8+
def self.instrument!
9+
::Parallel.class_eval do
10+
class << self
11+
include NewRelic::Agent::Instrumentation::Parallel::Instrumentation
12+
13+
alias_method :worker_without_newrelic, :worker
14+
15+
def worker(job_factory, options, &block)
16+
return worker_without_newrelic(job_factory, options, &block) unless NewRelic::Agent.agent
17+
18+
# Make sure the pipe channel listener is listening
19+
NewRelic::Agent::PipeChannelManager.listener.start unless NewRelic::Agent::PipeChannelManager.listener.started?
20+
21+
# Create a unique id for the channel and register it
22+
channel_id = Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)
23+
NewRelic::Agent.register_report_channel(channel_id)
24+
25+
worker_without_newrelic(job_factory, options) do |*args|
26+
worker_with_tracing(channel_id) { yield(*args) }
27+
end
28+
end
29+
end
30+
end
31+
end
32+
end
33+
end
34+
end

‎lib/new_relic/agent/instrumentation/parallel/instrumentation.rb‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# This file is distributed under New Relic's license terms.
2+
# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
3+
# frozen_string_literal: true
4+
5+
module NewRelic::Agent::Instrumentation
6+
module Parallel
7+
module Instrumentation
8+
# This runs inside of the new process that was forked by Parallel
9+
def worker_with_tracing(channel_id, &block)
10+
NewRelic::Agent.after_fork(
11+
:report_to_channel => channel_id,
12+
:report_instance_busy => false
13+
)
14+
15+
setup_for_txn_metric_merge_at_exit
16+
17+
yield
18+
end
19+
20+
def setup_for_txn_metric_merge_at_exit
21+
# Clear out any existing transaction metrics to prevent duplicates
22+
# when merging metrics back in at the end of the forked process
23+
if (txn = NewRelic::Agent::Tracer.current_transaction)
24+
txn.instance_variable_set(:@metrics, NewRelic::Agent::TransactionMetrics.new)
25+
end
26+
27+
# Install at_exit hook only once per process
28+
unless @parallel_at_exit_installed
29+
@parallel_at_exit_installed = true
30+
at_exit do
31+
# Merge all newly recorded metrics back into the parent process
32+
# It's a little weird, but needed because the transaction does not
33+
# finish in the child processes, so without this the metrics would be lost.
34+
if (txn = NewRelic::Agent::Tracer.current_transaction)
35+
NewRelic::Agent.agent.stats_engine.merge_transaction_metrics!(
36+
txn.metrics,
37+
txn.best_name
38+
)
39+
end
40+
41+
# force data to be sent back to the parent process
42+
NewRelic::Agent.agent&.stop_event_loop
43+
NewRelic::Agent.agent&.flush_pipe_data
44+
end
45+
end
46+
end
47+
end
48+
end
49+
end

‎lib/new_relic/agent/instrumentation/parallel/prepend.rb‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# This file is distributed under New Relic's license terms.
2+
# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
3+
# frozen_string_literal: true
4+
5+
module NewRelic::Agent::Instrumentation
6+
module Parallel
7+
module Prepend
8+
include NewRelic::Agent::Instrumentation::Parallel::Instrumentation
9+
10+
def worker(job_factory, options, &block)
11+
return super unless NewRelic::Agent.agent
12+
13+
# Make sure the pipe channel listener is actually listening
14+
NewRelic::Agent::PipeChannelManager.listener.start unless NewRelic::Agent::PipeChannelManager.listener.started?
15+
16+
# Create a unique id for the channel and register it
17+
channel_id = Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)
18+
NewRelic::Agent.register_report_channel(channel_id)
19+
20+
super do |*args|
21+
worker_with_tracing(channel_id) { yield(*args) }
22+
end
23+
end
24+
end
25+
end
26+
end

‎lib/new_relic/agent/pipe_service.rb‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,12 @@ def shutdown
6969
@pipe.close if @pipe # rubocop:disable Style/SafeNavigation
7070
end
7171

72+
# Validates that data can be marshalled. For PipeService, we always
73+
# use Marshal, so this always returns true.
74+
def valid_to_marshal?(data)
75+
true
76+
end
77+
7278
# Invokes the block it is passed. This is used to implement HTTP
7379
# keep-alive in the NewRelicService, and is a required interface for any
7480
# Service class.

‎test/multiverse/suites/parallel/Envfile‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# This file is distributed under New Relic's license terms.
2+
# See https://github.com/newrelic/newrelic-ruby-agent/blob/main/LICENSE for complete details.
3+
# frozen_string_literal: true
4+
5+
instrumentation_methods :chain, :prepend
6+
7+
PARALLEL_VERSIONS = [
8+
[nil, 2.7],
9+
]
10+
11+
def gem_list(parallel_version = nil)
12+
<<~RB
13+
gem 'parallel'#{parallel_version}
14+
gem 'rack'
15+
RB
16+
end
17+
18+
create_gemfiles(PARALLEL_VERSIONS)

‎test/multiverse/suites/parallel/config/newrelic.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
development:
3+
error_collector:
4+
enabled: true
5+
apdex_t: 0.5
6+
monitor_mode: true
7+
license_key: bootstrap_newrelic_admin_license_key_000
8+
app_name: test
9+
ca_bundle_path: ../../../config/test.cert.crt
10+
instrumentation:
11+
parallel: <%= $instrumentation_method %>
12+
host: localhost
13+
port: <%= $collector && $collector.port %>
14+
transaction_tracer:
15+
record_sql: obfuscated
16+
enabled: true
17+
stack_trace_threshold: 0.5
18+
transaction_threshold: 1.0
19+
capture_params: false

0 commit comments

Comments
 (0)