Add metrics module

[weeco]

For a better monitoring of my Nest microservices I'd like to have a metrics endpoint which can be scraped by monitoring systems like Prometheus.

I believe a module written for NestJS could provide quite a couple metrics by default:

• Metrics about the Node process resources (offered by default in the prom-client library)
• Requests per second
  • All endpoints alltogether
  • Each endpoint
• Request duration
  • All endpoints alltogether
  • Each endpoint
• Number of responses grouped by response status codes for
  • All endpoints alltogether
  • Each endpoint
• Response times in a bucket ([0-20ms, 20-50ms, ...]) for
  • All endpoints alltogether
  • Each endpoint

Beside these default metrics which could be offered out of the box, it should be easily possible to inject the metrics module into services, so that users can create custom metrics for their business code (e. g. number of registrations).

Using these metrics we can easily attach grafana to it and monitor our services:

[kamilmysliwiec]

Sounds great! Do you have any further ideas regarding this feature request? if so, feel free to share them, I'd love to get familiar with the community expectations. 🔥

[weeco]

Well the list of possible integrations with NestJS is kinda endless. Beside the given integrations which could come out of the box in NestJS, you could also offer metrics about:

• Number of total routes bound in NestJS
• Integration of other NestJS modules (e. g. auth - so that you can see Logins (and registrations) grouped by strategy), or websockets (expose number of actibe websocket connections, ...)
• Offer the possibility to manage these out of the box metrics. Basically let a user disable, rename or manipulate specific metrics
• Default Grafana board for NestJS (I am happy to create this)

You may want to keep in mind that there are other metrics formats beside Prometheus (such as DataDog, Graphite, etc.) which you may want to add at some point.

Spring boot already offers such a metrics module, you can probably get further inspiration there: https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html

[cyril-lakech]

Maybe Trace from risingstack?

[dawiss1337]

By the way, swagger-stats (http://swaggerstats.io/) is very easy to plug into nest.js app, and you get an API endpoint (+ nice UI), that can be used to scrape metrics. However, you can't add your own metrics that easily.

[weeco]

@DavisJaunzems That looks interesting. Maybe we can use this as inspiration for our own metrics module. Two questions I wasn't able to figure out at a first glance:

1. How does swaggerstats persist the information it shows in the panel? Does it use prometheus and puts a UI on top of that (similiar to Grafana)?
2. How do you configure dynamic routes (e. g. https://example.com/users/{id}) . How does it know it should aggregate stats on example.com/users instead of collecting stats for each queried userId ?

[dawiss1337]

@weeco swagger stats stores the data in memory attached to the main node process (when restarted it starts from scratch), and for some metrics it only keeps last 100 entries. Then these values are exposed through prometheus friendly format that prometheus can pull.
It uses swagger.json information to know which routes to aggregate.

I have a simple monitoring solution on top of nest - you create injectable X and register values/functions to it. Then there is a global injectable that scans the nest.js context for these X injectables, and reads their values. Then there is a route that takes the global injectable values and displays them when route is accessed. Pretty simple solution, but not sure if it would make sense to be part of the main nest.js.

[spike008t]

I've just write a nest module for prometheus (see: https://github.com/digikare/nestjs-prom).
At the moment the module is quite simple... that is my first nestjs module.

[cojack]

@weeco but exactly what do you need swaggerstats.io provide, and this what @spike008t create solve a own metrics. So basically thread to close, or provide an PR for documentation HOW TO.

We're using swaggerstats.io for few our projects, and this is really great.

[weeco]

No @cojack you are wrong. swagger-stats (as the name already says) relies on swagger definitions. It's written a generic JavaScript library and therefore not written for a specific framework.

1. First of all the NestJS way to create swagger docs is using Type annotations using @nestjs/swagger , which is really nice because this way the swagger definition is very close to the corresponding code.

2. Also in NestJS we've got a lot more potential to do way more things automatically, instead of relying on self written swagger definition, which in practice or probably often out of sync with the actual code. NestJS does know or routes and our params. That's why NestJS is capable of creating a swagger documentation without you needing to write any documentation. We should use this potential and automatically create our prometheus stats for the given endpoints too. We can nicely integrate it into the framework which leads to a very nice user experience for developers

3. Using Swagger stats you couldn't easily add your own custom business metrics as proposed here in the issue.

[kamilmysliwiec]

I'm pretty sure that you started to work on such integration already @weeco, correct? :)