Add opt in metrics

[jacobheun]

It would be helpful for debugging and improvements if we were able to collect metrics from Boost, such as deal success rates, errors, system information, etc.

This should be opt in only, and we could add this to the settings page with clear information about what we're collecting and where it's being collected.

---

In order to increase visibility into Boost and understand how Boost is used and what its performance is on SPs systems, we'd like to add functionality to Boost so that SPs can opt-in and periodically send statistics and metrics to a centralized system maintained by the development team.

We would then be able to track a measure a lot of data that is not available on-chain, such as:

• deal failure rates (after acceptance) prior to PublishStorageDeals message
• CommP benchmarks and performance
• various retrieval errors
• etc.

Given that Boost is exposed on the internet, the opt-in metrics could be implemented as either push- or pull- based.

Opt-in push metrics

As an example of push- based system we could implement something similar to go-ethereum: https://github.com/ethereum/go-ethereum/tree/master/metrics

Each SP would then periodically emit metrics to our centralized time-series db backend.

Opt-in pull metrics

As an example of pull- based system we could implement a metrics package similar to Lotus: https://github.com/filecoin-project/lotus/blob/master/metrics/metrics.go

Additionally to the metrics package, we'd need a way to announce and discover every endpoint for an SP who has enabled opt-in metrics, so that we know to pull them periodically.

Issue Tracking

• Add random sample tracing to the Boost RPC api #721
• Add parent trace context passing to the RPC API #722
• Monitor retrieval reliability

Refs

• feat: add basic tracing ipfs/go-bitswap#562

[dirkmc]

@jacobheun to follow up with data systems team about the metrics tooling they have implemented and if we can reuse it