Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds async Elasticsearch support #1309

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

fagnerzulin
Copy link

Overview

This Pull Request instruments the Elasticsearch python library with asynchronous implementation. Basically, I reused what was already implemented for the synchronous version, however, I ran into some issues with the unit tests in the following files:

  • tests/datastore_elasticsearch/test_async_connection.py
  • tests/datastore_elasticsearch/test_async_mget.py
  • tests/datastore_elasticsearch/test_async_multiple_dbs.py

If anyone can help me understand what’s going on, I would be grateful.

Related Github Issue

#1204

Testing

The agent includes a suite of tests which should be used to
verify your changes don't break existing functionality. These tests will run with
Github Actions when a pull request is made. More details on running the tests locally can be found
here,
For most contributions it is strongly recommended to add additional tests which
exercise your changes.

@fagnerzulin fagnerzulin requested a review from a team as a code owner February 20, 2025 17:09
@CLAassistant
Copy link

CLAassistant commented Feb 20, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ fagnerzulin
❌ mergify[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Contributor

@TimPansino TimPansino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed just the instrumentation so far but these are the issues I immediately see. I'll take a pass at fixing them and redesigning the instance_info section for async, and we'll see if that fixes the rest of the tests.


transaction._nr_datastore_instance_info = (None, None, None)

dt = DatastoreTrace(product="Elasticsearch", target=index, operation=operation, source=async_wrapper)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source argument here should be source=wrapped, that's for reporting the code's file name and line number for the Code Level Metrics feature. The async_wrapper code isn't needed here. The actual wrapper needs to be a coroutine that awaits wrapped() (probably, will check that's always true).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I fixed it.

Comment on lines 769 to 781
def _nr__async_Connection__init__wrapper(wrapped, instance, args, kwargs):
"""Cache datastore instance info on Connection object"""

def _bind_params(host="localhost", port=9200, *args, **kwargs):
return host, port

host, port = _bind_params(*args, **kwargs)
port = str(port)
instance._nr_host_port = (host, port)

return wrapped(*args, **kwargs)


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this wrapper is identical to _nr_Connection__init__wrapper, we can reuse the original instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved!

Comment on lines 796 to 801
def async_BaseNode__init__wrapper(wrapped, instance, args, kwargs):
result = wrapped(*args, **kwargs)
instance._nr_host_port = (instance.host, str(instance.port))
return result


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also identical to the original wrapper, we can reuse it instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved!

except Exception:
instance_info = ("unknown", "unknown", None)

transaction._nr_datastore_instance_info = instance_info
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wrapper is mostly right, but unfortunately it doesn't work in async contexts. When connecting to multiple databases at once there's a race condition that gets introduced if we store instance_info on the transaction object directly. We'll have to instead work out a different design where we stash it elsewhere, or call current_trace() and carefully insert it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any suggestions on how to do this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants