feat: integrate OpenTelemetry for enhanced telemetry and monitoring in Copilot CLI

Co-authored-by: Copilot <copilot@github.com>

Author: alfarahn

.github/copilot-instructions.md

@@ -14,25 +14,18 @@ Write-Host "All consent grants revoked — user will see fresh consent screens"
 
 This ensures the incremental consent flow works as expected and the user sees the real Microsoft Entra ID consent UI for each tier.
 
-always check if you are logged into:
-Basic information
-Name
-Contoso
-Tenant ID
-51650aad-d085-4ecb-8b07-d7ed4f5355e0
-Primary domain
-MngEnvMCAP237604.onmicrosoft.com
-License
-Microsoft Entra ID Free
-Users
-2
-Groups
-3
-Applications
-3
-Devices
-2
-admin@MngEnvMCAP237604.onmicrosoft.com
+always check if you are logged into the right tenant + subscription before running Azure CLI commands:
+
+- **Tenant**: Contoso — `51650aad-d085-4ecb-8b07-d7ed4f5355e0` (`MngEnvMCAP237604.onmicrosoft.com`)
+- **Subscription**: `ME-MngEnvMCAP237604-alfarahn-1` — `3f359915-1adb-4464-a4c0-8b0bc65c7959`
+- **Account**: `admin@MngEnvMCAP237604.onmicrosoft.com`
+
+If `az account show` returns a different tenant, run:
+
+```powershell
+az login --tenant 51650aad-d085-4ecb-8b07-d7ed4f5355e0
+az account set --subscription 3f359915-1adb-4464-a4c0-8b0bc65c7959
+```
 
 ## Project Purpose
 
@@ -58,7 +51,7 @@ The agent acts as a frontend on top of Azure Cost Management, Billing, ARM REST
 - **AI**: GitHub Copilot SDK (`GitHub.Copilot.SDK`) with BYOK (Bring Your Own Key) using Azure OpenAI via Entra ID bearer tokens. Sessions managed via `CopilotClient` / `CopilotSession`. Reasoning effort set to `xhigh`. The Copilot CLI provides built-in tools (file operations, bash, grep, glob, web fetch, memory) — custom tools handle Azure-specific APIs.
 - **Auth**: Auto-assigned anonymous sessions (no login required for chat); Microsoft Entra ID OAuth (multi-tenant) for Azure ARM, Microsoft Graph, and Log Analytics APIs
 - **Data Sources**: Azure Retail Prices API (no auth), Azure Service Health (no auth), Azure Cost Management APIs, Microsoft Graph APIs, Azure Monitor / Log Analytics APIs, ECharts visualization
-- **Observability**: OpenTelemetry + Azure Monitor (Application Insights) — structured traces via `ActivitySource("AzureFinOps.AI")` and custom metrics via `Meter("AzureFinOps.AI")` (chat requests, tool calls, errors, token refreshes, session lifecycle, duration histograms). Frontend telemetry in `client/src/main.js` captures page views, failed browser dependencies, uncaught JS errors, unhandled promise rejections, Vue component errors, and CSP violations. Third-party correlation headers are excluded for `cdn.jsdelivr.net` and `js.monitor.azure.com` so browser telemetry does not break public fetches.
+- **Observability**: OpenTelemetry end-to-end. The .NET app uses `UseAzureMonitor()` (auto-instruments HttpClient, ASP.NET Core, custom `ActivitySource("AzureFinOps.AI")` + `Meter("AzureFinOps.AI")`). The Copilot CLI subprocess emits OTLP via the SDK's built-in `TelemetryConfig` (GenAI + MCP semantic conventions — every tool call, LLM round-trip, prompt, tool args, result, token usage). Both feeds reach Application Insights via an in-container **OpenTelemetry Collector** (`otel/opentelemetry-collector-contrib`) using the `azuremonitor` exporter — config at `src/Dashboard/otel-collector-config.yaml`, launched by `entrypoint.sh` before the .NET app. Trace context (W3C `traceparent`) is auto-propagated SDK→CLI so Application Map shows one continuous transaction. Custom metrics (`finops.chat.requests`, `finops.tool.calls`, `finops.sessions.active`, etc.) keep flowing through the .NET exporter. Frontend telemetry in `client/src/main.js` captures page views, failed browser dependencies, uncaught JS errors, unhandled promise rejections, Vue component errors, and CSP violations. Third-party correlation headers are excluded for `cdn.jsdelivr.net` and `js.monitor.azure.com`.
 - **Deployment**: Azure App Service (Linux, P0v3 Premium) via Docker container image from Azure Container Registry (ACR). Multi-stage Dockerfile bakes Python 3, pip packages (python-pptx, matplotlib, pandas, numpy, lxml), and CLI tools into the image — no runtime install needed. Legacy zip deployment via `deploy.ps1` still supported for the original `finops-agent` app.
 - **Container Registry**: Azure Container Registry (`crfinopsagent.azurecr.io`) — Basic SKU, admin credentials, images built via `az acr build`
 - **Container App (staging)**: `finops-agent-container.azurewebsites.net` — Docker container on same P0v3 plan, used for testing before swapping to production
@@ -91,7 +84,9 @@ src/Dashboard/
 │   ├── ScriptTools.cs      # GenerateScript — generates downloadable Azure CLI/PowerShell scripts from FinOps recommendations
 │   └── UploadedFileTools.cs # QueryUploadedFile — inspect/query files (CSV/TSV/JSON/TXT/XLSX/PDF/Parquet) the user dropped into chat (no Azure consent needed). Backed by AI/Tools/Resources/file_inspect.py (pandas/openpyxl/pyarrow/pdfminer).
 │   └── TokenContext.cs     # UserTokens — per-user mutable token holder with volatile fields for concurrent access
-├── Dockerfile              # Multi-stage Docker build (node:22 + dotnet/sdk:10.0 + dotnet/aspnet:10.0 + Python 3)
+├── Dockerfile              # Multi-stage Docker build (node:22 + dotnet/sdk:10.0 + dotnet/aspnet:10.0 + Python 3 + OTel collector)
+├── entrypoint.sh           # Container entrypoint — starts OTel collector in background, then exec dotnet
+├── otel-collector-config.yaml # OpenTelemetry Collector config — bridges OTLP from Copilot CLI → Application Insights via azuremonitor exporter
 ├── .dockerignore           # Excludes bin/, obj/, node_modules/, wwwroot/ from Docker context
 ├── client/
 │   ├── index.html          # SPA entry point

src/Dashboard/AI/CopilotSessionFactory.cs

@@ -115,7 +115,22 @@ public static async Task<CopilotSessionFactory> CreateAsync(
         string azureOpenAIDeployment,
         ILoggerFactory loggerFactory)
     {
-        var copilotClient = new CopilotClient();
+        // Forward CLI telemetry (GenAI + MCP semantic conventions) to the local
+        // OTel collector when one is configured. The collector translates OTLP into
+        // Azure Monitor format and ships it to Application Insights so we get full
+        // tool-call and LLM-roundtrip visibility without any custom span wiring.
+        var otlpEndpoint = Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT");
+        var clientOptions = new CopilotClientOptions();
+        if (!string.IsNullOrWhiteSpace(otlpEndpoint))
+        {
+            clientOptions.Telemetry = new TelemetryConfig
+            {
+                OtlpEndpoint = otlpEndpoint,
+                CaptureContent = true, // include prompts, tool args, results
+                SourceName = "AzureFinOps.AI.CLI",
+            };
+        }
+        var copilotClient = new CopilotClient(clientOptions);
         await copilotClient.StartAsync();
 
         var credential = new ClientSecretCredential(

src/Dashboard/Dockerfile

@@ -31,8 +31,15 @@ RUN apt-get update -qq && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* /tmp/* /root/.cache
 
+# OpenTelemetry Collector (contrib build — needed for the azuremonitor exporter).
+# Bridges OTLP from the Copilot CLI subprocess into Azure Application Insights.
+COPY --from=otel/opentelemetry-collector-contrib:0.108.0 /otelcol-contrib /usr/local/bin/otelcol
+COPY otel-collector-config.yaml /etc/otelcol/config.yaml
+
 WORKDIR /app
 COPY --from=build /app/publish .
+COPY entrypoint.sh /usr/local/bin/entrypoint.sh
+RUN chmod +x /usr/local/bin/entrypoint.sh
 
 # Build metadata baked into the image
 ARG BUILD_SHA=dev
@@ -42,6 +49,10 @@ ENV BUILD_NUMBER=${BUILD_NUMBER}
 
 # App Service expects port 8080 by default for containers
 ENV ASPNETCORE_URLS=http://+:8080
+# Tell the Copilot CLI (and any other OTLP-capable child process) to push
+# telemetry to the local collector.
+ENV OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+ENV OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
 EXPOSE 8080
 
-ENTRYPOINT ["dotnet", "Dashboard.dll"]
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

src/Dashboard/Program.cs

@@ -53,8 +53,14 @@
 {
     builder.Services.AddOpenTelemetry()
         .UseAzureMonitor(o => o.ConnectionString = appInsightsCs)
-        .WithTracing(t => t.AddSource("AzureFinOps.AI"))
-        .WithMetrics(m => m.AddMeter("AzureFinOps.AI"));
+        .WithTracing(t => t
+            .AddSource("AzureFinOps.AI")
+            // Copilot SDK W3C-propagated tool/LLM spans surface here when the
+            // SDK's TelemetryConfig.SourceName is set to "AzureFinOps.AI.CLI".
+            .AddSource("AzureFinOps.AI.CLI"))
+        .WithMetrics(m => m
+            .AddMeter("AzureFinOps.AI")
+            .AddMeter("AzureFinOps.AI.CLI"));
 }
 
 var telemetry = new AiTelemetry();

src/Dashboard/entrypoint.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+# Container entrypoint — starts the OTel collector in the background, then
+# launches the .NET app in the foreground so its stdout/stderr (and exit
+# status) drive container lifecycle.
+set -e
+
+if [ -n "$APPLICATIONINSIGHTS_CONNECTION_STRING" ] || [ -n "$ApplicationInsights__ConnectionString" ]; then
+  # Normalise both env-var spellings so the collector config picks one up.
+  export APPLICATIONINSIGHTS_CONNECTION_STRING="${APPLICATIONINSIGHTS_CONNECTION_STRING:-$ApplicationInsights__ConnectionString}"
+  echo "[entrypoint] starting OTel collector → Azure Monitor"
+  /usr/local/bin/otelcol --config /etc/otelcol/config.yaml &
+  COLLECTOR_PID=$!
+  # Forward signals so SIGTERM from App Service shuts both down cleanly.
+  trap "kill -TERM $COLLECTOR_PID 2>/dev/null || true" TERM INT
+else
+  echo "[entrypoint] APPLICATIONINSIGHTS_CONNECTION_STRING not set — skipping collector"
+fi
+
+exec dotnet Dashboard.dll

src/Dashboard/otel-collector-config.yaml

@@ -0,0 +1,60 @@
+# OpenTelemetry Collector — bridges OTLP traces/metrics/logs from the Copilot
+# CLI subprocess (and any other OTLP source running locally) into Azure
+# Application Insights.
+#
+# Why a sidecar collector?
+#   - The Copilot CLI subprocess emits OTLP, not Azure Monitor's proprietary
+#     wire format. App Insights does not accept raw OTLP, so we need a
+#     translator. The "azuremonitor" exporter in opentelemetry-collector-contrib
+#     is the supported bridge.
+#   - It also acts as an out-of-process batch buffer so spans survive .NET
+#     SSE-stream cancellations and graceful container restarts.
+#
+# Connection string is picked up from the same env var the .NET app uses, so
+# there is exactly one place to configure it (App Service app setting
+# APPLICATIONINSIGHTS_CONNECTION_STRING or ApplicationInsights__ConnectionString).
+
+receivers:
+  otlp:
+    protocols:
+      http:
+        endpoint: 0.0.0.0:4318
+      grpc:
+        endpoint: 0.0.0.0:4317
+
+processors:
+  batch:
+    send_batch_size: 512
+    timeout: 5s
+  # Drop noisy /robots.txt and bot-crawler spans that bloat the bill.
+  filter/drop_noise:
+    error_mode: ignore
+    traces:
+      span:
+        - 'attributes["url.path"] == "/robots.txt"'
+        - 'attributes["url.path"] == "/wp-admin/install.php"'
+
+exporters:
+  azuremonitor:
+    connection_string: ${env:APPLICATIONINSIGHTS_CONNECTION_STRING}
+  # Tiny stdout exporter for local debugging (only enabled when OTEL_DEBUG=1).
+  debug:
+    verbosity: basic
+
+service:
+  telemetry:
+    logs:
+      level: warn
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: [filter/drop_noise, batch]
+      exporters: [azuremonitor]
+    metrics:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [azuremonitor]
+    logs:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [azuremonitor]