feat(analytics): break usage and cost down by calling extension

The caller identity from ADR-177 reached telemetry only, where no cost
is recorded — so it could not answer the question it exists for. It now
travels the cost path as well (ADR-178): source_extension is a column on
tx_nrllm_service_usage and part of its daily aggregation key, fed from
the same request metadata TelemetryMiddleware reads, so the cost row and
the telemetry row cannot disagree about who called.

The Analytics module gains a By-extension chart and a per-extension
table (cost, requests, tokens). Calls that name no caller are listed as
Unattributed rather than hidden.

trackUsage() grows one optional trailing parameter; api-surface.txt is
regenerated additively.

Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>

Author: CybotTM

CHANGELOG.md

@@ -7,6 +7,15 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ## [Unreleased]
 
 ### Added
+- **Analytics answers "which extension spent what"** (ADR-178). The usage
+  table carries the caller's extension key next to the money, so the
+  Analytics module gains a *By extension* chart and a per-extension table
+  with cost, requests and tokens. Calls that name no caller — wizard
+  tasks, scheduler runs, anything unannotated — are listed as
+  *Unattributed*. `UsageTrackerServiceInterface::trackUsage()` grows one
+  optional trailing `$sourceExtension` parameter (additive); rows written
+  before this change stay unattributed.
+
 - **Agent-harness verification.** `docs/ARCHITECTURE.md` (component map + phpat dependency-rule summary), `docs/exec-plans/` scaffold, `Build/Scripts/verify-harness.sh`, and a `harness-verify.yml` workflow — a thin caller of the shared `script-check` reusable — that fails CI on an AGENTS.md line-budget or dead-reference regression.
 
 - A caller can choose the correlation id its call is traced under, through

Classes/Controller/Backend/AnalyticsController.php

@@ -70,6 +70,7 @@ public function indexAction(): ResponseInterface
         $byProvider = $this->analytics->getBreakdownByProvider($period->from, $period->to);
         $byModel = $this->analytics->getBreakdownByModel($period->from, $period->to);
         $byService = $this->analytics->getBreakdownByService($period->from, $period->to);
+        $bySource = $this->analytics->getBreakdownBySourceExtension($period->from, $period->to);
 
         $moduleTemplate->assignMultiple([
             'preset'     => $period->preset,
@@ -82,6 +83,11 @@ public function indexAction(): ResponseInterface
             'byProvider' => $byProvider,
             'byModel'    => $byModel,
             'byService'  => $byService,
+            // Who called: the extension key a consumer named via
+            // withCallerSource() (ADR-178). The compatibility layer tags every
+            // bridged third-party call, so this is the AI inventory of the
+            // installation, priced.
+            'bySource'   => $bySource,
             'perUser'    => $this->analytics->getPerUserUsage($period->from, $period->to),
             // Reads tx_nrllm_telemetry, not the usage table: the runs a sibling
             // configuration answered for after the requested one failed.
@@ -99,6 +105,7 @@ public function indexAction(): ResponseInterface
                 'byProvider' => $byProvider,
                 'byModel'    => $byModel,
                 'byService'  => $byService,
+                'bySource'   => $bySource,
             ], JSON_THROW_ON_ERROR | JSON_HEX_TAG | JSON_HEX_AMP | JSON_HEX_APOS | JSON_HEX_QUOT),
         ]);
 

Classes/Provider/Middleware/UsageMiddleware.php

@@ -251,9 +251,22 @@ private function track(
             taskUid: $taskUid,
             beUserUid: $beUserUid,
             countsAsRequest: $countsAsRequest,
+            sourceExtension: $this->sourceExtension($context),
         );
     }
 
+    /**
+     * The caller identity as {@see TelemetryMiddleware} reads it — same
+     * metadata key, so the cost row and the telemetry row cannot disagree
+     * about who called (ADR-178).
+     */
+    private function sourceExtension(ProviderCallContext $context): string
+    {
+        $value = $context->metadata[TelemetryMiddleware::METADATA_SOURCE_EXTENSION] ?? null;
+
+        return is_string($value) ? $value : '';
+    }
+
     /**
      * One token figure as the PROVIDER reported it, or null where it reported
      * nothing (ADR-174).
@@ -301,6 +314,7 @@ private function trackSpecialized(ProviderCallContext $context, mixed $result):
                     taskUid: $record->taskUid,
                     beUserUid: $record->beUserUid,
                     countsAsRequest: $record->countsAsRequest,
+                    sourceExtension: $this->sourceExtension($context),
                 );
             }
 

Classes/Service/UsageAnalyticsService.php

@@ -40,6 +40,7 @@
         'service_provider',
         'model_id',
         'service_type',
+        'source_extension',
         'model_uid',
         'configuration_uid',
         'task_uid',
@@ -117,6 +118,15 @@ public function getBreakdownByService(DateTimeInterface $from, DateTimeInterface
         return $this->breakdown('service_type', $from, $to);
     }
 
+    public function getBreakdownBySourceExtension(DateTimeInterface $from, DateTimeInterface $to): array
+    {
+        // Unattributed rows are the normal case for anything that does not
+        // annotate (a wizard task, a scheduler run), so they get a label of
+        // their own instead of the generic "unknown" the other breakdowns use
+        // for a missing provider or model (ADR-178).
+        return $this->breakdown('source_extension', $from, $to, 'unattributed');
+    }
+
     public function getTotalsGroupedBy(string $column, DateTimeInterface $from, DateTimeInterface $to): array
     {
         $this->assertGroupableColumn($column);
@@ -317,7 +327,7 @@ private function assertGroupableColumn(string $column): void
     /**
      * @return list<array{label: string, cost: float, requests: int, tokens: int}>
      */
-    private function breakdown(string $column, DateTimeInterface $from, DateTimeInterface $to): array
+    private function breakdown(string $column, DateTimeInterface $from, DateTimeInterface $to, string $emptyLabel = 'unknown'): array
     {
         $this->assertGroupableColumn($column);
         $qb = $this->connectionPool->getQueryBuilderForTable(self::TABLE);
@@ -333,7 +343,7 @@ private function breakdown(string $column, DateTimeInterface $from, DateTimeInte
         foreach ($rows as $row) {
             $label = is_string($row[$column] ?? null) ? $row[$column] : '';
             if ($label === '') {
-                $label = 'unknown';
+                $label = $emptyLabel;
             }
 
             $out[] = [

Classes/Service/UsageAnalyticsServiceInterface.php

@@ -44,6 +44,15 @@ public function getBreakdownByModel(DateTimeInterface $from, DateTimeInterface $
      */
     public function getBreakdownByService(DateTimeInterface $from, DateTimeInterface $to): array;
 
+    /**
+     * Cost, requests and tokens per calling extension — the `source_extension`
+     * a consumer named via `AbstractOptions::withCallerSource()` (ADR-178).
+     * Calls that named nobody are grouped under `unattributed`.
+     *
+     * @return list<array{label: string, cost: float, requests: int, tokens: int}>
+     */
+    public function getBreakdownBySourceExtension(DateTimeInterface $from, DateTimeInterface $to): array;
+
     /**
      * Sum cost/requests/tokens grouped by an internal column, keyed by that
      * column's value. $column MUST be a hardcoded internal column name

Classes/Service/UsageTrackerService.php

@@ -70,8 +70,12 @@ public function trackUsage(
         int $taskUid = 0,
         ?int $beUserUid = null,
         bool $countsAsRequest = true,
+        string $sourceExtension = '',
     ): void {
         $beUser = $beUserUid ?? $this->getCurrentBackendUserId();
+        // Mirrors the telemetry column width; a longer claim is truncated
+        // rather than rejected (ADR-178: attribution is a label, not a key).
+        $sourceExtension = substr($sourceExtension, 0, 64);
         $today = strtotime('today');
         $now = time();
         // Sub-calls of a larger operation (e.g. a translation's language-detection
@@ -90,6 +94,10 @@ public function trackUsage(
         // indexed (config_lookup) and grouped on by the analytics module, so two
         // configurations on the same model must not merge into one row (which
         // would keep only the first configuration_uid and misattribute usage).
+        // source_extension joins the key for the same reason (ADR-178): two
+        // extensions calling one model on one day must stay two rows, or the
+        // per-extension cost breakdown attributes everything to whoever wrote
+        // the row first.
         $existingUid = $queryBuilder
             ->select('uid')
             ->from(self::TABLE)
@@ -101,6 +109,7 @@ public function trackUsage(
                 $queryBuilder->expr()->eq('model_uid', $modelUid),
                 $queryBuilder->expr()->eq('model_id', $queryBuilder->createNamedParameter($modelId)),
                 $queryBuilder->expr()->eq('task_uid', $taskUid),
+                $queryBuilder->expr()->eq('source_extension', $queryBuilder->createNamedParameter($sourceExtension)),
                 $queryBuilder->expr()->eq('request_date', $today),
             )
             ->executeQuery()
@@ -143,6 +152,7 @@ public function trackUsage(
                 'model_uid' => $modelUid,
                 'model_id' => $modelId,
                 'task_uid' => $taskUid,
+                'source_extension' => $sourceExtension,
                 'be_user' => $beUser,
                 'request_count' => $requestIncrement,
                 'tokens_used' => $metrics['tokens'] ?? 0,

Classes/Service/UsageTrackerServiceInterface.php

@@ -47,6 +47,10 @@ interface UsageTrackerServiceInterface
      *                                   request row (e.g. the language-detection step of a
      *                                   translation), so the metrics (tokens/cost) are still
      *                                   aggregated but the request is counted only once.
+     * @param string   $sourceExtension  Extension key the caller named via
+     *                                   `AbstractOptions::withCallerSource()`; '' when the call
+     *                                   is unattributed. Part of the daily aggregation key, so
+     *                                   per-extension cost stays separable (ADR-178).
      */
     public function trackUsage(
         string $serviceType,
@@ -58,6 +62,7 @@ public function trackUsage(
         int $taskUid = 0,
         ?int $beUserUid = null,
         bool $countsAsRequest = true,
+        string $sourceExtension = '',
     ): void;
 
     /**

Documentation/Administration/Analytics.rst

@@ -86,7 +86,7 @@ continuous rather than skipping gaps.
 Breakdown charts
 ================
 
-Three bar charts split the window's usage along different axes:
+Four bar charts split the window's usage along different axes:
 
 - **By provider** — cost and requests per ``service_provider``
   (OpenAI, Anthropic, Ollama, …).
@@ -95,6 +95,36 @@ Three bar charts split the window's usage along different axes:
   usage table, so it only reflects usage recorded after that change.
 - **By service** — cost and requests per service type (chat, vision,
   translation, speech, image).
+- **By extension** — cost and requests per calling extension, see
+  :ref:`administration-analytics-per-extension`.
+
+..  _administration-analytics-per-extension:
+
+Per-extension table
+===================
+
+A table lists cost, requests and tokens per **calling extension**, ordered
+by cost. It answers which piece of software spent what: a consumer names
+itself with ``AbstractOptions::withCallerSource()`` (:ref:`ADR-177
+<adr-177>`), and that name is stored on the usage row alongside the money
+(:ref:`ADR-178 <adr-178>`).
+
+Where the entries come from:
+
+- The compatibility layer `nr-llm-compat
+  <https://github.com/netresearch/t3x-nr-llm-compat>`__ tags every call it
+  reroutes, so an intercepted third-party extension (``ai_filemetadata``,
+  ``texter``, ``ns_t3ai``, …) shows up under its own extension key.
+- Any other consumer that annotates its calls appears the same way.
+- Everything else — wizard tasks, scheduler runs, playground calls and any
+  consumer that does not annotate — is listed as **Unattributed**. That is
+  the normal state, not an error.
+
+..  note::
+    The name is what the caller claims. Attribution is an inventory of the
+    installation, not an access control: a caller can name itself anything,
+    and nothing verifies it. Rows written before the column existed are
+    unattributed — no migration invents an origin for them.
 
 ..  _administration-analytics-per-user:
 

Resources/Private/Language/de.locallang.xlf

@@ -1212,6 +1212,30 @@
                 <source>By service</source>
                 <target>Nach Dienst</target>
             </trans-unit>
+            <trans-unit id="analytics.breakdown.source">
+                <source>By extension</source>
+                <target>Nach Extension</target>
+            </trans-unit>
+            <trans-unit id="analytics.source.title">
+                <source>Per calling extension</source>
+                <target>Pro aufrufender Extension</target>
+            </trans-unit>
+            <trans-unit id="analytics.source.description">
+                <source>What each extension spent through this installation. An extension appears here once it names itself on its calls; usage from wizards, scheduler runs and other unnamed callers is listed as unattributed.</source>
+                <target>Was jede Extension über diese Installation ausgegeben hat. Eine Extension erscheint hier, sobald sie sich bei ihren Aufrufen benennt; Verbrauch aus Assistenten, Scheduler-Läufen und anderen unbenannten Aufrufern steht unter "Nicht zugeordnet".</target>
+            </trans-unit>
+            <trans-unit id="analytics.source.unattributed">
+                <source>Unattributed</source>
+                <target>Nicht zugeordnet</target>
+            </trans-unit>
+            <trans-unit id="analytics.source.empty">
+                <source>No usage recorded in this period.</source>
+                <target>In diesem Zeitraum wurde kein Verbrauch erfasst.</target>
+            </trans-unit>
+            <trans-unit id="analytics.table.extension">
+                <source>Extension</source>
+                <target>Extension</target>
+            </trans-unit>
             <trans-unit id="analytics.peruser.title">
                 <source>Per backend user</source>
                 <target>Pro Backend-Benutzer</target>

Resources/Private/Language/locallang.xlf

@@ -916,6 +916,24 @@
             <trans-unit id="analytics.breakdown.service">
                 <source>By service</source>
             </trans-unit>
+            <trans-unit id="analytics.breakdown.source">
+                <source>By extension</source>
+            </trans-unit>
+            <trans-unit id="analytics.source.title">
+                <source>Per calling extension</source>
+            </trans-unit>
+            <trans-unit id="analytics.source.description">
+                <source>What each extension spent through this installation. An extension appears here once it names itself on its calls; usage from wizards, scheduler runs and other unnamed callers is listed as unattributed.</source>
+            </trans-unit>
+            <trans-unit id="analytics.source.unattributed">
+                <source>Unattributed</source>
+            </trans-unit>
+            <trans-unit id="analytics.source.empty">
+                <source>No usage recorded in this period.</source>
+            </trans-unit>
+            <trans-unit id="analytics.table.extension">
+                <source>Extension</source>
+            </trans-unit>
             <trans-unit id="analytics.peruser.title">
                 <source>Per backend user</source>
             </trans-unit>