wire dashboard to per-track metrics (#533) - #539
Open
gmarzot wants to merge 7 commits into
Open
Conversation
gmarzot
marked this pull request as ready for review
August 10, 2026 15:31
gmarzot
force-pushed
the
feature/track-metrics-533
branch
from
August 10, 2026 15:31
12e0746 to
284a044
Compare
gmarzot
changed the title
Contributor
Author
|
waiting got PR#533 to merge |
gmarzot
force-pushed
the
feature/track-metrics-533
branch
from
August 11, 2026 15:09
284a044 to
96ef6ef
Compare
Namespace is a required parameter and there is no enumeration endpoint, so namespaces are targets relabelled into the query. The static list should be generated from /state.namespace_tree into a file_sd file; noted in the README. Needs #533.
Rework of #533 made every query parameter optional, so one target covers all namespaces instead of a per-namespace list. Namespace splitting is only needed past the endpoint's limit ceiling; noted in the README along with the label encoding and the live-tracks-only lifetime.
/metrics/track takes one namespace per request and rejects a match wider than its limit, so an unscoped scrape fails once the relay's total track count passes the limit and takes every namespace with it. Scoping per namespace keeps working until a single namespace passes it. Namespaces change as events start and end, so ns-targets walks the relay's namespace tree and writes the file_sd target list; Prometheus rereads it without a restart. Encoding matches the endpoint's safe form.
/metrics/track matches a namespace prefix, so a parent and its child return the same tracks and every aggregate double-counts. Targets are top-level namespaces only: they do not overlap and their prefixes still cover everything beneath them. The endpoint documents itself as enabled by default but the resolved value was false, so it shipped dead; the deployed config states it explicitly.
gmarzot
force-pushed
the
feature/track-metrics-533
branch
from
August 11, 2026 18:09
96ef6ef to
3ba4740
Compare
Both sources emitted moqx_track_subscribers, so leaving the /state module in place would blend two label schemes into one metric name. The relay's own counters are durable where /state's entries are transient (#501), and they carry bytes and object counts /state never had. Tracks table gains real Bitrate (ingest rate) and Data Tx (delivered payload).
The deployed config sets track_metrics_enabled explicitly, so the header default does not matter here. Whether it should be true — the descriptions say so while the value is false — is #558 to settle.
mkstemp creates 0600 and the generator runs as root, so prometheus (nobody) could not read the target file: it was written correctly and every target was silently absent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blocked on #533. Prepared so the dashboard can move onto real per-track data as soon as it lands.
In this branch: a
moqx-trackscrape job. Every query parameter on/metrics/trackis optional, so a single target covers all namespaces;limitis set to the 1000 ceiling.To follow once #533 merges, replacing placeholders in the tracks table:
moqx_track_subscribersmax_over_time(moqx_track_subscribers[$__range])rate(moqx_track_bytes_received_total[1m]) * 8moqx_track_bytes_sent_totaltime() - moqx_track_last_object_timestamp_secondsThis also retires the
/statejson-exporter path for per-track data, whose entries are transient and undercount under load (#501).Type/Codec/Resolution/FPS stay
—: they are catalog attributes, not metrics.Notes carried from #533's shape
rate()/increase()rather than deltas across a lifetime.moq-test/interop->moq.2dtest-interop), so tables show the encoded form unless decoded for display./state.namespace_treecan generate that target list into afile_sdfile. Noted indocker/prometheus/README.md.This change is