feat(analytics): break usage and cost down by calling extension - #829
Conversation
github-actions
Bot
added
documentation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
d657569 to
00040a6
Compare
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #829 +/- ##
============================================
- Coverage 87.02% 87.01% -0.02%
- Complexity 9806 9809 +3
============================================
Files 572 572
Lines 31720 31733 +13
============================================
+ Hits 27604 27611 +7
- Misses 4116 4122 +6
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
00040a6 to
92e810f
Compare
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
The merge-base changed after approval.
The caller identity from ADR-177 reached telemetry only, where no cost is recorded — so it could not answer the question it exists for. It now travels the cost path as well (ADR-178): source_extension is a column on tx_nrllm_service_usage and part of its daily aggregation key, fed from the same request metadata TelemetryMiddleware reads, so the cost row and the telemetry row cannot disagree about who called. The Analytics module gains a By-extension chart and a per-extension table (cost, requests, tokens). Calls that name no caller are listed as Unattributed rather than hidden. trackUsage() grows one optional trailing parameter; api-surface.txt is regenerated additively. Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>
92e810f to
b82dc0c
Compare
There was a problem hiding this comment.
Automated approval for maintainer PR
All automated quality gates passed. See SECURITY_CONTROLS.md for compensating controls.
|
|
Self-review: b82dc0c The review this pull request demands is unsatisfiable (Copilot quota wall or repeated bot failures on this head). Per the documented fallback, the diff on this head was reviewed by the PR author; this comment is the on-the-record attestation the merge gate reads back. It stops matching on the next push. |
Makes the caller attribution from ADR-177 answer the question it exists for: which extension spends what. Stacked on #827 (the ADR); the diff against that branch is the implementation.
Why the telemetry column was not enough. Cost is aggregated daily in
tx_nrllm_service_usage, and every analytics figure reads that table; telemetry carries no cost, is purged on its own retention schedule, and shares no key a per-day aggregate could join on. So the identity travels the cost path too:source_extensionbecomes a column on the usage table and part of its daily aggregation key — two extensions on one model on one day stay two rows.UsageMiddlewarereads it from the same request metadataTelemetryMiddlewarereads, so the cost row and the telemetry row cannot disagree about who called.In the backend: the Analytics module gains a By extension chart next to provider/model/service and a per-extension table with cost, requests and tokens. Unannotated usage — wizard tasks, scheduler runs, anything that names no caller — is labelled Unattributed instead of being hidden or folded into another bucket. EN and DE labels included; the admin docs page gained a section stating what the numbers mean and that attribution is an unverified claim, not an access control.
Surface:
UsageTrackerServiceInterface::trackUsage()grows one optional trailing$sourceExtension;api-surface.txtis regenerated and the diff is that one additive line. Rows written before this change carry''and stay unattributed — no migration invents an origin.Tests (both guards were watched failing before they were kept):
source_extensionfrom the tracker's lookup makes the two new functional tests fail with1 is identical to 2— the rows collapse into one.''fails the new unit test.unattributedlabel.Gates run locally: cgl, PHPStan level 10, full unit suite (7217 tests), functional
-d sqlitefor the touched classes (103 tests). Rector was not run locally (its-p 8.2pin needs a second.Build); CI is the verdict there.