Perf improvements for exporter, instance & job support for Grafana

[dcelasun]

Performance improvements

After the first run, only fetch recently active torrents. This improves RPC response times for Transmission instances with thousands of torrents.

This is a port of @leijurv's work from his fork. Including the new metrics, peersSendingToUs and downloadedEver. Unlike that commit, I did not make any backwards incompatible changes.

Instance and job support for Grafana

This adds instance and job filters to the dashboard so if you have multiple exporters running you can choose which one(s) to view.

Here's how it looks:

I also adjusted a few panel widths for consistency.

Peers UL (uploading to us)

I also updated the stats table to make use of the new peersSendingToUs metric:

This closes #32.

[leijurv]

It's been a long time so I legitimately don't remember: what backwards incompatible changes did I make?

also for what it's worth I slightly regret having added the per torrent metrics because now my prometheus is 300gb lol

[dcelasun]

It's been a long time so I legitimately don't remember: what backwards incompatible changes did I make?

You dropped some metrics like leechers & seeders.

also for what it's worth I slightly regret having added the per torrent metrics because now my prometheus is 300gb lol

Yeah, it's a disk hog for sure :)

[bagusnl]

Heyya, good work on the improvements!

I've tested the build locally and on my case, the "Torrent" field is a bit of a problem as it will error the grafana out
422: too long query; got 49098 bytes; mustn't exceed -search.maxQueryLen=16384 bytes

I have a modest 649 entries, though some with a long names so its kind of a problem. Removing the filter works for me.
Everything else is fine so far, will update if I find any issue on mine, thanks for the work!

[dcelasun]

Heyya, good work on the improvements!

Glad you like it! I tested against a server with 347 entries, so I must have avoided the query length issue. I'll see if I can do anything about that. Maybe we can restrict the set somehow.

[bagusnl]

I store my metrics under VictoriaMetrics and so far it adds about 200-300MB per-day just from the transmission exporter itself. Which is fine by me. But one of my suggestion is to use stats is it possible to use total uploaded instead of average uploaded in the stats table? or maybe change one of them like total uploaded 30d?
My transmission activity is not that much so the average upload speed doesn't really reflect much on the individual entry activities.

[dcelasun]

But one of my suggestion is to use stats is it possible to use total uploaded instead of average uploaded in the stats table? or maybe change one of them like total uploaded 30d?

I can add a second table, Torrent stats minimal that only uses transmission_torrent_uploaded_ever. You can then delete the full Torrent stats dashboard from Grafana and update your Prometheus config to drop the transmission_torrent_upload_bytes metric.

Does that sound OK?

[bagusnl]

I can add a second table, Torrent stats minimal that only uses transmission_torrent_uploaded_ever. You can then delete the full Torrent stats dashboard from Grafana and update your Prometheus config to drop the transmission_torrent_upload_bytes metric.

Sounds good by me, but for the transmission_torrent_upload_bytes I can keep as the per-torrent upload graph looks neat

[dcelasun]

OK, just pushed an updated Grafana config. I split the dashboard into 2 rows. The first row (the one you're all used to) now has this torrent table:

As you can see, current and average upload columns are gone. These depend on the transmission_torrent_upload_bytes metric, which is fairly expensive.

The old table is in a new "Extended statistics" row, hidden by default:

If you don't want this at all / would like to reduce Prometheus storage costs, just delete that row and add this to the transmission-exporter job in your Prometheus config:

metric_relabel_configs:
  - source_labels: [ __name__ ]
    regex: 'transmission_torrent_upload_bytes'
    action: drop