Skip to content

Commit 8ebe9a3

Browse files
umaannamalaimergify[bot]TimPansinolrafeeihmstepanek
authored
Add google firestore instrumentation (#893)
* Add instrumentation for Google Firestore documents and collections (#876) * Initial GCP firestore instrumentation commit. * Add testing for documents and collections + test generators Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Add co-authors. Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Add co-authors. Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Trim whitespace --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: Timothy Pansino <[email protected]> * Firestore CI (#877) * Add firestore CI runner * Correct hook file name * Setup emulator credentials * Swap dependency to firestore alone * Hacky setup for firestore * Fix firestore hostname * Ensure firestore connection * Fix CI issues * Refactor Firestore Hooks (#879) * Remove unnecessary instrumentation * Simplify existing instrumentation * Remove unnecessary settings lookups * Firestore Sync Client Instrumentation (#880) * Remove unnecessary instrumentation * Simplify existing instrumentation * Remove unnecessary settings lookups * Client instrumentation * Add query and aggregation query instrumentation * Fix deprecation warning * Simplify collection lookup * Combine query test files * Rename methods for clarity * Instrument Firestore batching * Add transaction instrumentation * Consumer iterators on <=Py38 * Allow better parallelization in firestore tests * Clean out unnecessary code * [Mega-Linter] Apply linters fixes * Better parallelization safeguards * Add collection group instrumentation * [Mega-Linter] Apply linters fixes * Change imports to native APIs * Swap target functions to lambdas * Convert exercise functions to fixtures --------- Co-authored-by: TimPansino <[email protected]> * Update datastore_trace wrapper to take instance info (#883) * Update datastore trace wrapper to take instance info. * [Mega-Linter] Apply linters fixes * Make instance info args optional. * [Mega-Linter] Apply linters fixes * Add datastore trace testing. * Add background task decorator. * [Mega-Linter] Apply linters fixes * Fix typo in validator. --------- Co-authored-by: umaannamalai <[email protected]> * Async Generator Wrapper (#884) * Add async generator wrapper * Add no harm test * Remove anext calls * Add graphql traces to decorator testing * Remove pypy generator gc logic --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Trace Async Wrapper Argument (#886) * Add async_wrapper to datastore_trace api * Add async wrapper argument to all trace APIs * Add testing for automatic and manual asyncwrappers * Firstore Async Instrumentation (#882) * Remove unnecessary instrumentation * Simplify existing instrumentation * Remove unnecessary settings lookups * Client instrumentation * Add query and aggregation query instrumentation * Fix deprecation warning * Simplify collection lookup * Combine query test files * Rename methods for clarity * Instrument Firestore batching * Add transaction instrumentation * Consumer iterators on <=Py38 * Add async generator wrapper * Allow better parallelization in firestore tests * Fix issue in async generator wrapper * Add async client instrumentation * Squashed commit of the following: commit 9d411e0 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:57:39 2023 -0700 Clean out unnecessary code commit cb550ba Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:27:01 2023 -0700 Allow better parallelization in firestore tests * Add async collection instrumentation * Add async document instrumentation * Async Query instrumentation * Add async batch instrumentation * Add instrumentation for AsyncTransaction * Squashed commit of the following: commit c836f8f Author: TimPansino <[email protected]> Date: Thu Jul 27 19:54:35 2023 +0000 [Mega-Linter] Apply linters fixes commit 02a55a1 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 12:46:46 2023 -0700 Add collection group instrumentation commit ab1f4ff Author: Tim Pansino <[email protected]> Date: Thu Jul 27 12:00:33 2023 -0700 Better parallelization safeguards commit fa5f39a Author: TimPansino <[email protected]> Date: Wed Jul 26 22:59:11 2023 +0000 [Mega-Linter] Apply linters fixes commit 9d411e0 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:57:39 2023 -0700 Clean out unnecessary code commit cb550ba Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:27:01 2023 -0700 Allow better parallelization in firestore tests * Remove reset_firestore * Re-merge of test_query * Use public API imports * Add async collection group instrumentation * Refactor exercise functions to fixtures * Squashed commit of the following: commit 09c5e11 Author: Tim Pansino <[email protected]> Date: Wed Aug 2 14:33:24 2023 -0700 Add testing for automatic and manual asyncwrappers commit fc3ef6b Author: Tim Pansino <[email protected]> Date: Wed Aug 2 14:33:05 2023 -0700 Add async wrapper argument to all trace APIs commit 479f9e2 Merge: faf3ccc edd1f94 Author: Tim Pansino <[email protected]> Date: Wed Aug 2 13:44:24 2023 -0700 Merge remote-tracking branch 'origin/develop-google-firestore-instrumentation' into feature-async-wrapper-argument commit edd1f94 Author: Timothy Pansino <[email protected]> Date: Wed Aug 2 13:40:51 2023 -0700 Async Generator Wrapper (#884) * Add async generator wrapper * Add no harm test * Remove anext calls * Add graphql traces to decorator testing * Remove pypy generator gc logic --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit faf3ccc Author: Tim Pansino <[email protected]> Date: Mon Jul 31 15:10:56 2023 -0700 Add async_wrapper to datastore_trace api * Remove custom wrapper code from firestore * Undo wrapper edits --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Firestore Instance Info (#887) * Add instance info testing to query * Instance info for query.stream * Squashed commit of the following: commit 1c426c8 Author: umaannamalai <[email protected]> Date: Mon Jul 31 23:01:49 2023 +0000 [Mega-Linter] Apply linters fixes commit 7687c06 Author: Uma Annamalai <[email protected]> Date: Mon Jul 31 15:47:09 2023 -0700 Make instance info args optional. commit 53f8400 Author: umaannamalai <[email protected]> Date: Mon Jul 31 22:23:20 2023 +0000 [Mega-Linter] Apply linters fixes commit d95d477 Author: Uma Annamalai <[email protected]> Date: Mon Jul 31 15:20:41 2023 -0700 Update datastore trace wrapper to take instance info. * Add instance info testing to all apis * Separate transaction instance info tests * Implement all instance info getters * Squashed commit of the following: commit db3561e Merge: 844e556 edd1f94 Author: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Date: Wed Aug 2 22:10:32 2023 +0000 Merge branch 'develop-google-firestore-instrumentation' into feature-firstore-async-instrumentation commit 844e556 Author: Tim Pansino <[email protected]> Date: Wed Aug 2 15:09:49 2023 -0700 Remove custom wrapper code from firestore commit ad2999f Author: Tim Pansino <[email protected]> Date: Wed Aug 2 14:58:38 2023 -0700 Squashed commit of the following: commit 09c5e11 Author: Tim Pansino <[email protected]> Date: Wed Aug 2 14:33:24 2023 -0700 Add testing for automatic and manual asyncwrappers commit fc3ef6b Author: Tim Pansino <[email protected]> Date: Wed Aug 2 14:33:05 2023 -0700 Add async wrapper argument to all trace APIs commit 479f9e2 Merge: faf3ccc edd1f94 Author: Tim Pansino <[email protected]> Date: Wed Aug 2 13:44:24 2023 -0700 Merge remote-tracking branch 'origin/develop-google-firestore-instrumentation' into feature-async-wrapper-argument commit edd1f94 Author: Timothy Pansino <[email protected]> Date: Wed Aug 2 13:40:51 2023 -0700 Async Generator Wrapper (#884) * Add async generator wrapper * Add no harm test * Remove anext calls * Add graphql traces to decorator testing * Remove pypy generator gc logic --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit faf3ccc Author: Tim Pansino <[email protected]> Date: Mon Jul 31 15:10:56 2023 -0700 Add async_wrapper to datastore_trace api commit edd1f94 Author: Timothy Pansino <[email protected]> Date: Wed Aug 2 13:40:51 2023 -0700 Async Generator Wrapper (#884) * Add async generator wrapper * Add no harm test * Remove anext calls * Add graphql traces to decorator testing * Remove pypy generator gc logic --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 29579fc Merge: 4a8a3fe 7596fb4 Author: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Date: Wed Aug 2 19:54:09 2023 +0000 Merge branch 'develop-google-firestore-instrumentation' into feature-firstore-async-instrumentation commit 7596fb4 Author: Uma Annamalai <[email protected]> Date: Wed Aug 2 12:53:29 2023 -0700 Update datastore_trace wrapper to take instance info (#883) * Update datastore trace wrapper to take instance info. * [Mega-Linter] Apply linters fixes * Make instance info args optional. * [Mega-Linter] Apply linters fixes * Add datastore trace testing. * Add background task decorator. * [Mega-Linter] Apply linters fixes * Fix typo in validator. --------- Co-authored-by: umaannamalai <[email protected]> commit 4a8a3fe Merge: 7bf6f49 dcc92a9 Author: Tim Pansino <[email protected]> Date: Mon Jul 31 14:51:20 2023 -0700 Merge remote-tracking branch 'origin/develop-google-firestore-instrumentation' into feature-firstore-async-instrumentation commit 7bf6f49 Author: Tim Pansino <[email protected]> Date: Mon Jul 31 14:34:26 2023 -0700 Refactor exercise functions to fixtures commit d3e4732 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 13:20:37 2023 -0700 Add async collection group instrumentation commit 5902515 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 13:09:13 2023 -0700 Use public API imports commit 9266924 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 13:04:19 2023 -0700 Re-merge of test_query commit b6bc9a4 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 13:01:27 2023 -0700 Remove reset_firestore commit 87fbe62 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 13:00:37 2023 -0700 Squashed commit of the following: commit c836f8f Author: TimPansino <[email protected]> Date: Thu Jul 27 19:54:35 2023 +0000 [Mega-Linter] Apply linters fixes commit 02a55a1 Author: Tim Pansino <[email protected]> Date: Thu Jul 27 12:46:46 2023 -0700 Add collection group instrumentation commit ab1f4ff Author: Tim Pansino <[email protected]> Date: Thu Jul 27 12:00:33 2023 -0700 Better parallelization safeguards commit fa5f39a Author: TimPansino <[email protected]> Date: Wed Jul 26 22:59:11 2023 +0000 [Mega-Linter] Apply linters fixes commit 9d411e0 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:57:39 2023 -0700 Clean out unnecessary code commit cb550ba Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:27:01 2023 -0700 Allow better parallelization in firestore tests commit e04ec6f Author: Tim Pansino <[email protected]> Date: Thu Jul 27 11:55:44 2023 -0700 Add instrumentation for AsyncTransaction commit 6b7fc79 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 16:56:04 2023 -0700 Add async batch instrumentation commit c392e78 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 16:36:03 2023 -0700 Async Query instrumentation commit aab244b Author: Tim Pansino <[email protected]> Date: Wed Jul 26 16:20:58 2023 -0700 Add async document instrumentation commit 3fb6a6c Author: Tim Pansino <[email protected]> Date: Wed Jul 26 16:11:17 2023 -0700 Add async collection instrumentation commit 7851baf Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:58:12 2023 -0700 Squashed commit of the following: commit 9d411e0 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:57:39 2023 -0700 Clean out unnecessary code commit cb550ba Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:27:01 2023 -0700 Allow better parallelization in firestore tests commit c49a1cf Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:54:13 2023 -0700 Add async client instrumentation commit c857358 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 15:53:21 2023 -0700 Fix issue in async generator wrapper commit 5693dd2 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:27:01 2023 -0700 Allow better parallelization in firestore tests commit fbe40ea Author: Tim Pansino <[email protected]> Date: Wed Jul 26 14:22:53 2023 -0700 Add async generator wrapper commit b9a91e5 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 12:21:25 2023 -0700 Consumer iterators on <=Py38 commit ef06df5 Author: Tim Pansino <[email protected]> Date: Wed Jul 26 12:01:25 2023 -0700 Add transaction instrumentation commit 2ce45c8 Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:55:50 2023 -0700 Instrument Firestore batching commit d17b62f Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:31:48 2023 -0700 Rename methods for clarity commit 6214f0b Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:30:23 2023 -0700 Combine query test files commit b4e8700 Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:23:03 2023 -0700 Simplify collection lookup commit a0c78a2 Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:18:51 2023 -0700 Fix deprecation warning commit 44598cc Author: Tim Pansino <[email protected]> Date: Tue Jul 25 15:15:13 2023 -0700 Add query and aggregation query instrumentation commit b9eaa5b Author: Tim Pansino <[email protected]> Date: Tue Jul 25 13:33:42 2023 -0700 Client instrumentation commit 19f5a48 Author: Tim Pansino <[email protected]> Date: Mon Jul 24 15:55:52 2023 -0700 Remove unnecessary settings lookups commit ba7850a Author: Tim Pansino <[email protected]> Date: Mon Jul 24 15:44:54 2023 -0700 Simplify existing instrumentation commit e07ffc3 Author: Tim Pansino <[email protected]> Date: Mon Jul 24 15:44:10 2023 -0700 Remove unnecessary instrumentation * Add instance info to async client * Simplify lookup logic for instance info * Precompute closures for memory usage * Undo wrapper edits * Fix typo * Change port from int ot str * Fix Generator Wrappers (#890) * Fix async wrapper implementations * Add regression testing --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: Timothy Pansino <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: umaannamalai <[email protected]>
1 parent 17f8937 commit 8ebe9a3

34 files changed

+3281
-76
lines changed

.github/workflows/tests.yml

+65
Original file line numberDiff line numberDiff line change
@@ -1024,3 +1024,68 @@ jobs:
10241024
name: coverage-${{ github.job }}-${{ strategy.job-index }}
10251025
path: ./**/.coverage.*
10261026
retention-days: 1
1027+
1028+
firestore:
1029+
env:
1030+
TOTAL_GROUPS: 1
1031+
1032+
strategy:
1033+
fail-fast: false
1034+
matrix:
1035+
group-number: [1]
1036+
1037+
runs-on: ubuntu-20.04
1038+
container:
1039+
image: ghcr.io/newrelic/newrelic-python-agent-ci:latest
1040+
options: >-
1041+
--add-host=host.docker.internal:host-gateway
1042+
timeout-minutes: 30
1043+
1044+
services:
1045+
firestore:
1046+
# Image set here MUST be repeated down below in options. See comment below.
1047+
image: gcr.io/google.com/cloudsdktool/google-cloud-cli:437.0.1-emulators
1048+
ports:
1049+
- 8080:8080
1050+
# Set health checks to wait 5 seconds in lieu of an actual healthcheck
1051+
options: >-
1052+
--health-cmd "echo success"
1053+
--health-interval 10s
1054+
--health-timeout 5s
1055+
--health-retries 5
1056+
--health-start-period 5s
1057+
gcr.io/google.com/cloudsdktool/google-cloud-cli:437.0.1-emulators /bin/bash -c "gcloud emulators firestore start --host-port=0.0.0.0:8080" ||
1058+
# This is a very hacky solution. GitHub Actions doesn't provide APIs for setting commands on services, but allows adding arbitrary options.
1059+
# --entrypoint won't work as it only accepts an executable and not the [] syntax.
1060+
# Instead, we specify the image again the command afterwards like a call to docker create. The result is a few environment variables
1061+
# and the original command being appended to our hijacked docker create command. We can avoid any issues by adding || to prevent that
1062+
# from every being executed as bash commands.
1063+
1064+
steps:
1065+
- uses: actions/checkout@v3
1066+
1067+
- name: Fetch git tags
1068+
run: |
1069+
git config --global --add safe.directory "$GITHUB_WORKSPACE"
1070+
git fetch --tags origin
1071+
1072+
- name: Get Environments
1073+
id: get-envs
1074+
run: |
1075+
echo "envs=$(tox -l | grep '^${{ github.job }}\-' | ./.github/workflows/get-envs.py)" >> $GITHUB_OUTPUT
1076+
env:
1077+
GROUP_NUMBER: ${{ matrix.group-number }}
1078+
1079+
- name: Test
1080+
run: |
1081+
tox -vv -e ${{ steps.get-envs.outputs.envs }} -p auto
1082+
env:
1083+
TOX_PARALLEL_NO_SPINNER: 1
1084+
PY_COLORS: 0
1085+
1086+
- name: Upload Coverage Artifacts
1087+
uses: actions/upload-artifact@v3
1088+
with:
1089+
name: coverage-${{ github.job }}-${{ strategy.job-index }}
1090+
path: ./**/.coverage.*
1091+
retention-days: 1

newrelic/api/database_trace.py

+7-7
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
import logging
1717

1818
from newrelic.api.time_trace import TimeTrace, current_trace
19-
from newrelic.common.async_wrapper import async_wrapper
19+
from newrelic.common.async_wrapper import async_wrapper as get_async_wrapper
2020
from newrelic.common.object_wrapper import FunctionWrapper, wrap_object
2121
from newrelic.core.database_node import DatabaseNode
2222
from newrelic.core.stack_trace import current_stack
@@ -244,9 +244,9 @@ def create_node(self):
244244
)
245245

246246

247-
def DatabaseTraceWrapper(wrapped, sql, dbapi2_module=None):
247+
def DatabaseTraceWrapper(wrapped, sql, dbapi2_module=None, async_wrapper=None):
248248
def _nr_database_trace_wrapper_(wrapped, instance, args, kwargs):
249-
wrapper = async_wrapper(wrapped)
249+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
250250
if not wrapper:
251251
parent = current_trace()
252252
if not parent:
@@ -273,9 +273,9 @@ def _nr_database_trace_wrapper_(wrapped, instance, args, kwargs):
273273
return FunctionWrapper(wrapped, _nr_database_trace_wrapper_)
274274

275275

276-
def database_trace(sql, dbapi2_module=None):
277-
return functools.partial(DatabaseTraceWrapper, sql=sql, dbapi2_module=dbapi2_module)
276+
def database_trace(sql, dbapi2_module=None, async_wrapper=None):
277+
return functools.partial(DatabaseTraceWrapper, sql=sql, dbapi2_module=dbapi2_module, async_wrapper=async_wrapper)
278278

279279

280-
def wrap_database_trace(module, object_path, sql, dbapi2_module=None):
281-
wrap_object(module, object_path, DatabaseTraceWrapper, (sql, dbapi2_module))
280+
def wrap_database_trace(module, object_path, sql, dbapi2_module=None, async_wrapper=None):
281+
wrap_object(module, object_path, DatabaseTraceWrapper, (sql, dbapi2_module, async_wrapper))

newrelic/api/datastore_trace.py

+90-11
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
import functools
1616

1717
from newrelic.api.time_trace import TimeTrace, current_trace
18-
from newrelic.common.async_wrapper import async_wrapper
18+
from newrelic.common.async_wrapper import async_wrapper as get_async_wrapper
1919
from newrelic.common.object_wrapper import FunctionWrapper, wrap_object
2020
from newrelic.core.datastore_node import DatastoreNode
2121

@@ -82,6 +82,9 @@ def __enter__(self):
8282
self.product = transaction._intern_string(self.product)
8383
self.target = transaction._intern_string(self.target)
8484
self.operation = transaction._intern_string(self.operation)
85+
self.host = transaction._intern_string(self.host)
86+
self.port_path_or_id = transaction._intern_string(self.port_path_or_id)
87+
self.database_name = transaction._intern_string(self.database_name)
8588

8689
datastore_tracer_settings = transaction.settings.datastore_tracer
8790
self.instance_reporting_enabled = datastore_tracer_settings.instance_reporting.enabled
@@ -92,7 +95,14 @@ def __repr__(self):
9295
return "<%s object at 0x%x %s>" % (
9396
self.__class__.__name__,
9497
id(self),
95-
dict(product=self.product, target=self.target, operation=self.operation),
98+
dict(
99+
product=self.product,
100+
target=self.target,
101+
operation=self.operation,
102+
host=self.host,
103+
port_path_or_id=self.port_path_or_id,
104+
database_name=self.database_name,
105+
),
96106
)
97107

98108
def finalize_data(self, transaction, exc=None, value=None, tb=None):
@@ -125,7 +135,7 @@ def create_node(self):
125135
)
126136

127137

128-
def DatastoreTraceWrapper(wrapped, product, target, operation):
138+
def DatastoreTraceWrapper(wrapped, product, target, operation, host=None, port_path_or_id=None, database_name=None, async_wrapper=None):
129139
"""Wraps a method to time datastore queries.
130140
131141
:param wrapped: The function to apply the trace to.
@@ -140,6 +150,16 @@ def DatastoreTraceWrapper(wrapped, product, target, operation):
140150
or the name of any API function/method in the client
141151
library.
142152
:type operation: str or callable
153+
:param host: The name of the server hosting the actual datastore.
154+
:type host: str
155+
:param port_path_or_id: The value passed in can represent either the port,
156+
path, or id of the datastore being connected to.
157+
:type port_path_or_id: str
158+
:param database_name: The name of database where the current query is being
159+
executed.
160+
:type database_name: str
161+
:param async_wrapper: An async trace wrapper from newrelic.common.async_wrapper.
162+
:type async_wrapper: callable or None
143163
:rtype: :class:`newrelic.common.object_wrapper.FunctionWrapper`
144164
145165
This is typically used to wrap datastore queries such as calls to Redis or
@@ -155,7 +175,7 @@ def DatastoreTraceWrapper(wrapped, product, target, operation):
155175
"""
156176

157177
def _nr_datastore_trace_wrapper_(wrapped, instance, args, kwargs):
158-
wrapper = async_wrapper(wrapped)
178+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
159179
if not wrapper:
160180
parent = current_trace()
161181
if not parent:
@@ -187,7 +207,33 @@ def _nr_datastore_trace_wrapper_(wrapped, instance, args, kwargs):
187207
else:
188208
_operation = operation
189209

190-
trace = DatastoreTrace(_product, _target, _operation, parent=parent, source=wrapped)
210+
if callable(host):
211+
if instance is not None:
212+
_host = host(instance, *args, **kwargs)
213+
else:
214+
_host = host(*args, **kwargs)
215+
else:
216+
_host = host
217+
218+
if callable(port_path_or_id):
219+
if instance is not None:
220+
_port_path_or_id = port_path_or_id(instance, *args, **kwargs)
221+
else:
222+
_port_path_or_id = port_path_or_id(*args, **kwargs)
223+
else:
224+
_port_path_or_id = port_path_or_id
225+
226+
if callable(database_name):
227+
if instance is not None:
228+
_database_name = database_name(instance, *args, **kwargs)
229+
else:
230+
_database_name = database_name(*args, **kwargs)
231+
else:
232+
_database_name = database_name
233+
234+
trace = DatastoreTrace(
235+
_product, _target, _operation, _host, _port_path_or_id, _database_name, parent=parent, source=wrapped
236+
)
191237

192238
if wrapper: # pylint: disable=W0125,W0126
193239
return wrapper(wrapped, trace)(*args, **kwargs)
@@ -198,7 +244,7 @@ def _nr_datastore_trace_wrapper_(wrapped, instance, args, kwargs):
198244
return FunctionWrapper(wrapped, _nr_datastore_trace_wrapper_)
199245

200246

201-
def datastore_trace(product, target, operation):
247+
def datastore_trace(product, target, operation, host=None, port_path_or_id=None, database_name=None, async_wrapper=None):
202248
"""Decorator allows datastore query to be timed.
203249
204250
:param product: The name of the vendor.
@@ -211,6 +257,16 @@ def datastore_trace(product, target, operation):
211257
or the name of any API function/method in the client
212258
library.
213259
:type operation: str
260+
:param host: The name of the server hosting the actual datastore.
261+
:type host: str
262+
:param port_path_or_id: The value passed in can represent either the port,
263+
path, or id of the datastore being connected to.
264+
:type port_path_or_id: str
265+
:param database_name: The name of database where the current query is being
266+
executed.
267+
:type database_name: str
268+
:param async_wrapper: An async trace wrapper from newrelic.common.async_wrapper.
269+
:type async_wrapper: callable or None
214270
215271
This is typically used to decorate datastore queries such as calls to Redis
216272
or ElasticSearch.
@@ -224,10 +280,21 @@ def datastore_trace(product, target, operation):
224280
... time.sleep(*args, **kwargs)
225281
226282
"""
227-
return functools.partial(DatastoreTraceWrapper, product=product, target=target, operation=operation)
228-
229-
230-
def wrap_datastore_trace(module, object_path, product, target, operation):
283+
return functools.partial(
284+
DatastoreTraceWrapper,
285+
product=product,
286+
target=target,
287+
operation=operation,
288+
host=host,
289+
port_path_or_id=port_path_or_id,
290+
database_name=database_name,
291+
async_wrapper=async_wrapper,
292+
)
293+
294+
295+
def wrap_datastore_trace(
296+
module, object_path, product, target, operation, host=None, port_path_or_id=None, database_name=None, async_wrapper=None
297+
):
231298
"""Method applies custom timing to datastore query.
232299
233300
:param module: Module containing the method to be instrumented.
@@ -244,6 +311,16 @@ def wrap_datastore_trace(module, object_path, product, target, operation):
244311
or the name of any API function/method in the client
245312
library.
246313
:type operation: str
314+
:param host: The name of the server hosting the actual datastore.
315+
:type host: str
316+
:param port_path_or_id: The value passed in can represent either the port,
317+
path, or id of the datastore being connected to.
318+
:type port_path_or_id: str
319+
:param database_name: The name of database where the current query is being
320+
executed.
321+
:type database_name: str
322+
:param async_wrapper: An async trace wrapper from newrelic.common.async_wrapper.
323+
:type async_wrapper: callable or None
247324
248325
This is typically used to time database query method calls such as Redis
249326
GET.
@@ -256,4 +333,6 @@ def wrap_datastore_trace(module, object_path, product, target, operation):
256333
... 'sleep')
257334
258335
"""
259-
wrap_object(module, object_path, DatastoreTraceWrapper, (product, target, operation))
336+
wrap_object(
337+
module, object_path, DatastoreTraceWrapper, (product, target, operation, host, port_path_or_id, database_name, async_wrapper)
338+
)

newrelic/api/external_trace.py

+8-8
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
from newrelic.api.cat_header_mixin import CatHeaderMixin
1818
from newrelic.api.time_trace import TimeTrace, current_trace
19-
from newrelic.common.async_wrapper import async_wrapper
19+
from newrelic.common.async_wrapper import async_wrapper as get_async_wrapper
2020
from newrelic.common.object_wrapper import FunctionWrapper, wrap_object
2121
from newrelic.core.external_node import ExternalNode
2222

@@ -66,9 +66,9 @@ def create_node(self):
6666
)
6767

6868

69-
def ExternalTraceWrapper(wrapped, library, url, method=None):
69+
def ExternalTraceWrapper(wrapped, library, url, method=None, async_wrapper=None):
7070
def dynamic_wrapper(wrapped, instance, args, kwargs):
71-
wrapper = async_wrapper(wrapped)
71+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
7272
if not wrapper:
7373
parent = current_trace()
7474
if not parent:
@@ -103,7 +103,7 @@ def dynamic_wrapper(wrapped, instance, args, kwargs):
103103
return wrapped(*args, **kwargs)
104104

105105
def literal_wrapper(wrapped, instance, args, kwargs):
106-
wrapper = async_wrapper(wrapped)
106+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
107107
if not wrapper:
108108
parent = current_trace()
109109
if not parent:
@@ -125,9 +125,9 @@ def literal_wrapper(wrapped, instance, args, kwargs):
125125
return FunctionWrapper(wrapped, literal_wrapper)
126126

127127

128-
def external_trace(library, url, method=None):
129-
return functools.partial(ExternalTraceWrapper, library=library, url=url, method=method)
128+
def external_trace(library, url, method=None, async_wrapper=None):
129+
return functools.partial(ExternalTraceWrapper, library=library, url=url, method=method, async_wrapper=async_wrapper)
130130

131131

132-
def wrap_external_trace(module, object_path, library, url, method=None):
133-
wrap_object(module, object_path, ExternalTraceWrapper, (library, url, method))
132+
def wrap_external_trace(module, object_path, library, url, method=None, async_wrapper=None):
133+
wrap_object(module, object_path, ExternalTraceWrapper, (library, url, method, async_wrapper))

newrelic/api/function_trace.py

+8-8
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
import functools
1616

1717
from newrelic.api.time_trace import TimeTrace, current_trace
18-
from newrelic.common.async_wrapper import async_wrapper
18+
from newrelic.common.async_wrapper import async_wrapper as get_async_wrapper
1919
from newrelic.common.object_names import callable_name
2020
from newrelic.common.object_wrapper import FunctionWrapper, wrap_object
2121
from newrelic.core.function_node import FunctionNode
@@ -89,9 +89,9 @@ def create_node(self):
8989
)
9090

9191

92-
def FunctionTraceWrapper(wrapped, name=None, group=None, label=None, params=None, terminal=False, rollup=None):
92+
def FunctionTraceWrapper(wrapped, name=None, group=None, label=None, params=None, terminal=False, rollup=None, async_wrapper=None):
9393
def dynamic_wrapper(wrapped, instance, args, kwargs):
94-
wrapper = async_wrapper(wrapped)
94+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
9595
if not wrapper:
9696
parent = current_trace()
9797
if not parent:
@@ -147,7 +147,7 @@ def dynamic_wrapper(wrapped, instance, args, kwargs):
147147
return wrapped(*args, **kwargs)
148148

149149
def literal_wrapper(wrapped, instance, args, kwargs):
150-
wrapper = async_wrapper(wrapped)
150+
wrapper = async_wrapper if async_wrapper is not None else get_async_wrapper(wrapped)
151151
if not wrapper:
152152
parent = current_trace()
153153
if not parent:
@@ -171,13 +171,13 @@ def literal_wrapper(wrapped, instance, args, kwargs):
171171
return FunctionWrapper(wrapped, literal_wrapper)
172172

173173

174-
def function_trace(name=None, group=None, label=None, params=None, terminal=False, rollup=None):
174+
def function_trace(name=None, group=None, label=None, params=None, terminal=False, rollup=None, async_wrapper=None):
175175
return functools.partial(
176-
FunctionTraceWrapper, name=name, group=group, label=label, params=params, terminal=terminal, rollup=rollup
176+
FunctionTraceWrapper, name=name, group=group, label=label, params=params, terminal=terminal, rollup=rollup, async_wrapper=async_wrapper
177177
)
178178

179179

180180
def wrap_function_trace(
181-
module, object_path, name=None, group=None, label=None, params=None, terminal=False, rollup=None
181+
module, object_path, name=None, group=None, label=None, params=None, terminal=False, rollup=None, async_wrapper=None
182182
):
183-
return wrap_object(module, object_path, FunctionTraceWrapper, (name, group, label, params, terminal, rollup))
183+
return wrap_object(module, object_path, FunctionTraceWrapper, (name, group, label, params, terminal, rollup, async_wrapper))

0 commit comments

Comments
 (0)