Skip to content

Commit 8be5101

Browse files
authored
Add OpenTelemetry support for tracing and metrics (#77)
* feat: add OpenTelemetry support for tracing and metrics in Ratatosk framework * feat: add ConnectorMeter utility for consistent meter naming in telemetry * feat: enhance telemetry metrics with message count and improve activity scope management * feat: enhance telemetry with additional messaging attributes and improve activity tracking * feat: enhance telemetry metrics for message sending, receiving, and status queries * feat: introduce request classes for messaging operations and enhance telemetry context handling
1 parent 605c785 commit 8be5101

36 files changed

Lines changed: 3089 additions & 681 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -419,3 +419,6 @@ FodyWeavers.xsd
419419

420420
.idea
421421
samples/**/src/**/appsettings.local.json
422+
423+
# macOS
424+
.DS_Store

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ Those are application-level concerns, so you can choose your own architecture.
5757
| `Ratatosk.Senders` | Sender identity infrastructure: `ISenderRepository<TSender>`, `ISenderResolver`, cache, per-connector configuration, `MessageBuilder.FromSender()` extension. | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders.svg?label=NuGet)](https://www.nuget.org/packages/Ratatosk.Senders/) |
5858
| `Ratatosk.Senders.InMemory` | In-memory `IRepository<Sender>` for development and testing. | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders.InMemory.svg?label=NuGet)](https://www.nuget.org/packages/Ratatosk.Senders.InMemory/) |
5959
| `Ratatosk.Senders.EntityFramework` | EF Core persistence for the sender identity registry (`SenderDbContext`, `EntitySenderRepository`, `DbSender`). | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders.EntityFramework.svg?label=NuGet)](https://www.nuget.org/packages/Ratatosk.Senders.EntityFramework/) |
60+
| `Ratatosk.Extensions.OpenTelemetry` | Convenience extensions for wiring Ratatosk telemetry sources into OpenTelemetry SDK. | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Extensions.OpenTelemetry.svg?label=NuGet)](https://www.nuget.org/packages/Ratatosk.Extensions.OpenTelemetry/) |
6061

6162
## Quick example
6263

@@ -144,7 +145,7 @@ Locks the public API and ships stable package releases with production-ready gua
144145
Adds retry/circuit-breaker policies, OpenTelemetry signals, health checks, and timeout controls.
145146

146147
- [x] **Retry policies** — Polly-based retry and circuit-breaker integrated at the connector base level.
147-
- [ ] **OpenTelemetry tracing & metrics** — ActivitySources, Meters, and histograms per connector.
148+
- [x] **OpenTelemetry tracing & metrics** — ActivitySources, Meters, and histograms per connector.
148149
- [ ] **Health checks** — ASP.NET Core `IHealthCheck` implementations auto-registered per connector.
149150
- [ ] **Connector-level timeout configuration** — Per-operation timeout via the DI registration API.
150151

ROADMAP.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22

33
This document describes the planned evolution of the **Ratatosk Framework** — a .NET library that provides a unified, provider-agnostic model for sending and receiving messages across multiple channels and connectors. Each milestone below includes a summary of its intent, detailed descriptions of every planned feature, and the rationale behind each decision.
44

5-
> **Current stable release:** `v0.4.0`
5+
> **Current stable release:** `v1.0.1`
66
> Connectors available: Facebook Messenger, Firebase Push, SendGrid Email, Telegram Bot, Twilio SMS/WhatsApp
7-
> Framework includes: `IMessagingClient` facade, `MessageBuilder`, auto-initialization, `ChannelSchema` derivation, expanded validation
7+
> Framework includes: `IMessagingClient` facade, `MessageBuilder`, auto-initialization, `ChannelSchema` derivation, expanded validation, retry policies, OpenTelemetry tracing & metrics
88
99
---
1010

@@ -15,7 +15,7 @@ This document describes the planned evolution of the **Ratatosk Framework** —
1515
| [Framework Foundations](#v040--framework-foundations) | v0.4.0 | [Test Coverage Target](#test-coverage-target--80) · [CI/CD Pipeline Hardening](#cicd-pipeline-hardening) · [Structured Logging Improvements](#structured-logging-improvements) · [Documentation](#documentation) |
1616
| [Inbound Messaging](#v050--inbound-messaging) | v0.5.0 | [Twilio Inbound Messages](#twilio-inbound-messages-sms--whatsapp) · [SendGrid Inbound Messages](#sendgrid-inbound-messages-inbound-parse) · [Firebase Inbound Messages](#firebase-inbound-messages-data--notification-messages) |
1717
| [First Stable Release](#v100--first-stable-release) | v1.0.0 | [API Freeze](#api-freeze) · [NuGet GA Release](#nuget-ga-release) · [Interactive Content](#interactive-content) · [Sender Identity Model](#sender-identity-model) |
18-
| [Resilience & Observability](#v110--resilience--observability) | v1.1.0 | [Retry Policies](#retry-policies) · [OpenTelemetry Tracing & Metrics](#opentelemetry-tracing--metrics) · [Health Checks](#health-checks) · [Connector-Level Timeout Configuration](#connector-level-timeout-configuration) |
18+
| [Resilience & Observability](#v110--resilience--observability) | v1.1.0 | [Retry Policies](#retry-policies) · [OpenTelemetry Tracing & Metrics](#opentelemetry-tracing--metrics) · [Health Checks](#health-checks) · [Connector-Level Timeout Configuration](#connector-level-timeout-configuration) |
1919
| [New SaaS Connectors](#v120--new-saas-connectors) | v1.2.0 | [Slack](#slack-connector) · [Microsoft Teams](#microsoft-teams-connector) · [WhatsApp Business API](#whatsapp-business-api-connector-direct-cloud-api) · [Viber Business](#viber-business-connector) · [LINE](#line-connector) |
2020
| [Protocol Connectors](#v130--protocol-connectors) | v1.3.0 | [Base Classes](#protocol-connector-base-classes) · [SMPP](#smpp-connector) · [SMTP](#smtp-connector) · [RCS](#rcs-connector) · [APNs](#apns-connector-direct) |
2121
| [Content Adaptation & Transcoding](#v140--content-adaptation--transcoding) | v1.4.0 | [IContentTranscoder Abstraction](#icontenttrancoder-abstraction) · [Built-In Transcoders](#built-in-transcoders) · [Channel-Aware Fallback](#channel-aware-content-fallback) · [SMS Segmentation](#sms-segmentation) · [Character Encoding Detection](#character-encoding-detection) |
@@ -272,6 +272,8 @@ A generic sender identity system that decouples message composition from sender
272272

273273
A messaging connector that fails silently, cannot be monitored, or brings down dependent services on provider outages is not production-ready. This milestone adds the operational layer that connectors need to be trusted in production: structured retry and circuit-breaker policies, distributed tracing and metrics via OpenTelemetry, health check endpoints, and consistent structured logging across all connectors.
274274

275+
Retry policies and OpenTelemetry tracing & metrics are complete in this release. Health checks and timeout configuration remain in progress.
276+
275277
---
276278

277279
### Retry Policies
@@ -294,23 +296,24 @@ Polly-based retry and circuit-breaker policies integrated at the connector base
294296

295297
---
296298

297-
### OpenTelemetry Tracing & Metrics
299+
### OpenTelemetry Tracing & Metrics
298300

299301
> *"You cannot improve what you cannot observe."*
300302
301-
#### The Problem Today
303+
#### The Problem (Before)
302304

303-
There is no built-in telemetry in the connector layer. Developers who want to trace message send latency, track delivery rates, or count failures must instrument their own wrapper code on top of the connectors.
305+
There was no built-in telemetry in the connector layer. Developers who wanted to trace message send latency, track delivery rates, or count failures had to instrument their own wrapper code on top of the connectors.
304306

305-
#### What We Are Building
307+
#### What We Built
306308

307-
Activity sources and metric instruments built into the connector base: an `ActivitySource` per connector for distributed tracing (spans for send, receive, status query), and `Meter` instruments for counters (messages sent, received, failed) and histograms (send latency, payload size). All telemetry follows OpenTelemetry semantic conventions and is automatically exported via the application's configured OTLP pipeline.
309+
Activity sources and metric instruments built into the connector base: an `ActivitySource` per connector for distributed tracing (spans for send, receive, status query), and `Meter` instruments for counters (messages sent, received, failed) and histograms (send latency, payload size). All telemetry follows OpenTelemetry semantic conventions and is automatically exported via the application's configured OTLP pipeline. A dedicated `Ratatosk.Extensions.OpenTelemetry` package provides convenience extensions (`WithOpenTelemetry()` on `MessagingBuilder`, `AddRatatoskInstrumentation()` on `TracerProviderBuilder`/`MeterProviderBuilder`). Telemetry options (`EnableTracing`, `EnableMetrics`, `EnablePayloadSizeMetrics`) are configurable per connector through connection settings or the builder API, following the same pattern as retry policies.
308310

309311
#### Benefits
310312

311313
- End-to-end distributed traces include connector-level spans out of the box
312314
- Dashboards and alerts can be built on connector metrics without custom instrumentation
313315
- Fully compatible with any OpenTelemetry-compliant backend (Jaeger, Prometheus, Azure Monitor, etc.)
316+
- Payload size metrics are opt-in (disabled by default) to avoid serialisation overhead in high-throughput scenarios
314317

315318
---
316319

Ratatosk.sln

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,10 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Ratatosk.Senders.InMemory.X
6363
EndProject
6464
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Ratatosk.Senders.EntityFramework.XUnit", "test\Ratatosk.Senders.EntityFramework.XUnit\Ratatosk.Senders.EntityFramework.XUnit.csproj", "{15A47650-BE84-4ECC-86DD-245F7C0BA865}"
6565
EndProject
66+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Ratatosk.Extensions.OpenTelemetry", "src\Ratatosk.Extensions.OpenTelemetry\Ratatosk.Extensions.OpenTelemetry.csproj", "{A5930328-ADBF-46E1-80D5-C14501FC93DD}"
67+
EndProject
68+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Ratatosk.Extensions.OpenTelemetry.XUnit", "test\Ratatosk.Extensions.OpenTelemetry.XUnit\Ratatosk.Extensions.OpenTelemetry.XUnit.csproj", "{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}"
69+
EndProject
6670
Global
6771
GlobalSection(SolutionConfigurationPlatforms) = preSolution
6872
Debug|Any CPU = Debug|Any CPU
@@ -373,6 +377,30 @@ Global
373377
{15A47650-BE84-4ECC-86DD-245F7C0BA865}.Release|x64.Build.0 = Release|Any CPU
374378
{15A47650-BE84-4ECC-86DD-245F7C0BA865}.Release|x86.ActiveCfg = Release|Any CPU
375379
{15A47650-BE84-4ECC-86DD-245F7C0BA865}.Release|x86.Build.0 = Release|Any CPU
380+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
381+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|Any CPU.Build.0 = Debug|Any CPU
382+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|x64.ActiveCfg = Debug|Any CPU
383+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|x64.Build.0 = Debug|Any CPU
384+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|x86.ActiveCfg = Debug|Any CPU
385+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Debug|x86.Build.0 = Debug|Any CPU
386+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|Any CPU.ActiveCfg = Release|Any CPU
387+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|Any CPU.Build.0 = Release|Any CPU
388+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|x64.ActiveCfg = Release|Any CPU
389+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|x64.Build.0 = Release|Any CPU
390+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|x86.ActiveCfg = Release|Any CPU
391+
{A5930328-ADBF-46E1-80D5-C14501FC93DD}.Release|x86.Build.0 = Release|Any CPU
392+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
393+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|Any CPU.Build.0 = Debug|Any CPU
394+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|x64.ActiveCfg = Debug|Any CPU
395+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|x64.Build.0 = Debug|Any CPU
396+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|x86.ActiveCfg = Debug|Any CPU
397+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Debug|x86.Build.0 = Debug|Any CPU
398+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|Any CPU.ActiveCfg = Release|Any CPU
399+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|Any CPU.Build.0 = Release|Any CPU
400+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|x64.ActiveCfg = Release|Any CPU
401+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|x64.Build.0 = Release|Any CPU
402+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|x86.ActiveCfg = Release|Any CPU
403+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F}.Release|x86.Build.0 = Release|Any CPU
376404
EndGlobalSection
377405
GlobalSection(SolutionProperties) = preSolution
378406
HideSolutionNode = FALSE
@@ -403,6 +431,8 @@ Global
403431
{BB959F65-44CD-4444-8607-108AE75AE7EA} = {0F91077D-AC47-4319-96FF-09CA6EE50CC6}
404432
{D1A54750-C8F4-4B48-A7E2-0CDAF0F6DD29} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
405433
{15A47650-BE84-4ECC-86DD-245F7C0BA865} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
434+
{A5930328-ADBF-46E1-80D5-C14501FC93DD} = {0F91077D-AC47-4319-96FF-09CA6EE50CC6}
435+
{8AAD9B57-AE90-4916-B0A2-182C0DC1202F} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8}
406436
EndGlobalSection
407437
GlobalSection(ExtensibilityGlobals) = postSolution
408438
SolutionGuid = {64BF9420-C7A8-4D2B-804F-41EB2E83437F}

docs/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ The framework is deliberately focused on the messaging contract and connector co
7070
| `Ratatosk.Senders` | Sender identity infrastructure: `ISenderRepository<TSender>`, `ISenderResolver`, cache, per-connector configuration, and `MessageBuilder.FromSender()` extension. | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders)](https://nuget.org/packages/Ratatosk.Senders) |
7171
| `Ratatosk.Senders.InMemory` | In-memory sender repository for development and testing, with optional seed data. | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders.InMemory)](https://nuget.org/packages/Ratatosk.Senders.InMemory) |
7272
| `Ratatosk.Senders.EntityFramework` | EF Core persistence for the sender identity registry (`SenderDbContext`, `EntitySenderRepository`, `DbSender`). | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Senders.EntityFramework)](https://nuget.org/packages/Ratatosk.Senders.EntityFramework) |
73+
| `Ratatosk.Extensions.OpenTelemetry` | Convenience extensions for wiring Ratatosk telemetry sources into OpenTelemetry SDK (`WithOpenTelemetry()` on `MessagingBuilder`, `AddRatatoskInstrumentation()` on `TracerProviderBuilder`/`MeterProviderBuilder`). | [![NuGet](https://img.shields.io/nuget/v/Ratatosk.Extensions.OpenTelemetry)](https://nuget.org/packages/Ratatosk.Extensions.OpenTelemetry) |
7374

7475
## Reading path
7576

@@ -96,6 +97,7 @@ The framework is deliberately focused on the messaging contract and connector co
9697
- [Authentication](authentication.md) — auth providers, credential management, OAuth flows
9798
- [Result types](result-types.md)`OperationResult<T>`, send results, health data
9899
- [Retry policies](retry-policies.md) — automatic retry with backoff, jitter, and circuit breaker
100+
- [Telemetry](telemetry.md) — OpenTelemetry tracing and metrics, wiring, configuration
99101
- [Advanced configuration](advanced.md) — security, named connector isolation, health checks, testing
100102

101103
**Connector guides:**

docs/advanced.md

Lines changed: 3 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -145,41 +145,11 @@ Logger.LogInformation("Sending to {Receiver}", message.Receiver?.Address);
145145
// [Connector: Twilio/SMS] [Message: sms-123] Sending to +15550002222
146146
```
147147

148-
### Metrics to track
148+
### OpenTelemetry tracing & metrics
149149

150-
For production monitoring, track per-connector metrics:
150+
Ratatosk emits `ActivitySource` and `Meter` instruments from every connector operation — spans for send, receive, status query, and initialization; counters for messages sent, received, and failed; histograms for latency and payload size. The `IMessagingClient` facade emits its own telemetry under `Ratatosk.Client`.
151151

152-
| Metric | Source | What it detects |
153-
|---|---|---|
154-
| Send attempts | Count before `SendMessageAsync` call | Volume trends |
155-
| Send success rate | `OperationResult.IsSuccess()` | Provider degradation |
156-
| Send latency | Stopwatch around `SendMessageAsync` | Provider performance |
157-
| Error codes | `OperationResult.Error.Code` | Failure type distribution |
158-
| Connection state | `Connector.State` | Connectivity issues |
159-
160-
```csharp
161-
public class MetricsDecorator : IChannelConnector
162-
{
163-
private readonly IChannelConnector _inner;
164-
private readonly IMeterFactory _meters;
165-
166-
public async Task<OperationResult<SendResult>> SendMessageAsync(
167-
IMessage message, CancellationToken ct)
168-
{
169-
var sw = Stopwatch.StartNew();
170-
var result = await _inner.SendMessageAsync(message, ct);
171-
sw.Stop();
172-
173-
_meters.CreateCounter("messaging.sends").Add(1);
174-
_meters.CreateHistogram("messaging.latency").Record(sw.ElapsedMilliseconds);
175-
176-
if (result.IsFailure())
177-
_meters.CreateCounter($"messaging.errors.{result.Error?.Code}").Add(1);
178-
179-
return result;
180-
}
181-
}
182-
```
152+
See [Telemetry](telemetry.md) for the full signal reference, wiring instructions, configuration options, and performance considerations.
183153

184154
## Retry policies
185155

docs/installation.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ dotnet add package Ratatosk.Abstractions
2929
# Custom connector authoring — needed only if you build your own connector
3030
dotnet add package Ratatosk.Connector.Abstractions
3131
dotnet add package Ratatosk.Connectors
32+
33+
# OpenTelemetry — convenience extensions for wiring Ratatosk telemetry sources
34+
dotnet add package Ratatosk.Extensions.OpenTelemetry
3235
```
3336

3437
`Ratatosk` depends on `Ratatosk.Abstractions` and `Ratatosk.Connectors` (which bring in `Microsoft.Extensions.DependencyInjection.Abstractions` and `Microsoft.Extensions.Logging.Abstractions`). The `Abstractions` and `Connector.Abstractions` packages have no external dependencies beyond the .NET BCL.

0 commit comments

Comments
 (0)