feat(csharp/src/Drivers/Databricks): Add minimal Activity-based logging to RetryHttpHandler

[msrathore-db]

Summary

Adds Activity-based logging to RetryHttpHandler.cs for improved observability of HTTP retry behavior.

Changes

Logging Strategy

• ✅ No logging for successful first-attempt requests (most common case ~99%)
• ✅ Per-attempt logging when retries occur
• ✅ Exception logging with context for failures

What Gets Logged

Tags (When Retries Occur):

1. http.retry.attempt - Current attempt number (logged per retry)
2. http.retry.total_attempts - Total number of retry attempts
3. http.response.status_code - HTTP status code
4. http.retry.outcome - Only for failures: cancelled or timeout_exceeded

Exception Context (On Failures):

• error.context - Failure reason (http.retry.cancelled or http.retry.timeout_exceeded)
• attempts - Number of attempts made
• total_retry_seconds / timeout_seconds - Timing information (timeout cases only)

Retryable Status Codes: 408, 502, 503, 504

Example Log Output

Success (No Retries):

// NO LOGGING - Most efficient path

Success After Retries:

{
  "Tags": {
    "http.retry.attempt": 1,
    "http.response.status_code": 503,
    "http.retry.attempt": 2,
    "http.response.status_code": 503,
    "http.retry.total_attempts": 2,
    "http.response.status_code": 200
  }
}

Timeout Exceeded:

{
  "Tags": {
    "http.retry.total_attempts": 3,
    "http.response.status_code": 503,
    "http.retry.outcome": "timeout_exceeded"
  },
  "Exception": {
    "error.context": "http.retry.timeout_exceeded",
    "attempts": 3,
    "total_retry_seconds": 7,
    "timeout_seconds": 30
  }
}

Testing

Tested locally with log output.

Breaking Changes

None. Additive logging only.

[eric-wang-1990]

Have you tested this E2E with PowerBI?
I am actually not sure if this can be logged correctly, I wonder if you need to do similar things of CloudFetchDownlader which is to propagate the top level Trace function:

arrow-adbc/csharp/src/Drivers/Databricks/Reader/CloudFetch/CloudFetchDownloader.cs

Line 603 in 0b01c6e

ActivityTrace IActivityTracer.Trace => _statement.Trace;

.

Can you test it with latest PBI desktop and see if you can see your logs.