Add end-user logging and tracing for drivers

[birschick-bq]

What feature or improvement would you like to see?

I'd like to suggest the following plan based on the use of OpenTelemetry

Create a proof-of-concept

1. Target a Go driver like Snowflake
2. Enable the "zero code" automatic instrumentation option

Enable output to

1. Local file logging
2. Console logging
3. Possibly to a remote tracing system

Determine if

1. automatic instrumentation is a viable option
2. if automatic instrumentation is too verbose, exposes private, or personally identifying information.
3. or if we need to use complete manual instrumentation or some combination
4. determine if cross-runtime contexts can be bridged and if so, how best to accomplish that.

[lidavidm]

I'd also be curious if you have ideas about how to bridge OpenTelemetry across runtimes; presumably C#/Python would like to be able to start a span that will properly be the parent span of anything the Go driver does. Possibly we can even do this mostly automatically inside the Go bridge code

[zeroshade]

The easiest way to bridge OpenTelemetry across the runtimes would be to pass metadata containing the headers to identify the context/span. Since they would be key/value pairs. I've done it with C++/Go before, perhaps through options on the connection/statement

[lidavidm]

Yeah - we would need some manual code (possibly, we can do this automatically in the Python driver manager too when OpenTelemetry is installed) and would just extract/inject the context across the FFI boundary.

[lidavidm]

If there's a better way though...

[birschick-bq]

I'd also be curious if you have ideas about how to bridge OpenTelemetry across runtimes; presumably C#/Python would like to be able to start a span that will properly be the parent span of anything the Go driver does. Possibly we can even do this mostly automatically inside the Go bridge code

I've added this to the list of goals for the POC.

[birschick-bq]

Observations:

• ADBC drivers should adopt OpenTelemetry instrumentation for tracing as it provides for a good variety of export targets (OTLP, debug, , etc.).
• Using tracing, as opposed to logging, adds the ability to correlate telemetry from different sources. However, tracing is more expensive and implies that telemetry messages should be less frequent than traditional logging.
• Automatic instrumentation is not realistic or desirable (inadvertent exposure of PII).
• Normally, exporters should be added/enabled in the end-user application. This is because the end-user application can define the lifetime of the exporter and what option and parameters to set.
• However, this is not possible with a multiple-runtime scenario. The wrapping language runtime does not have access to the telemetry data in the consumed runtime.
• It is recommended that each driver implementation should allow the addition, configuration, and enablement of the OTLP exporter.
• End user applications can deploy and configure an OpenTelemetry Collector to receive OTLP messages and export them to a target of their choice.
• As the overhead and complexity of deploying and running a Collector is considerable, it still makes sense to provide the addition and enablement of console and/or file exporters.
• It is recommended that a new property, TraceParent, is added to the API for the Connection and Statement objects to allow telemetry messages to have a correlation id with an existing trace parent.
• We need a recommendation on how to enable and configure the behavior of each exporter. Should it be by database/connection options, configuration file, or environment variables?
• Because drivers are contained in libraries and not applications, care must be taken to ensure the correct lifetime of enabled exporters.