Cloud Cost exporter is a tool designed to collect cost data from cloud providers and export the data in Prometheus format. The cost data can then be combined with usage data from tools such as stackdriver, yace, and promitor to measure the spend of resources at a granular level.
The goal of this project is to provide a consistent interface for collecting the rate of cost data from multiple cloud providers and exporting the data in Prometheus format. There was a need to track the costs of both kubernetes and non-kubernetes resources across multiple cloud providers at a per minute interval. Billing data for each cloud provider takes hours to days for it to be fully accurate, and we needed a way of having a more real-time view of costs.
Primary goals:
- Track the rate(IE, $/cpu/hr) for resources across
- Export the rate in Prometheus format
- Support the major cloud providers(AWS, GCP, Azure)
Non Goals:
- Billing level accuracy
- Measure the spend of resources
- Take into account CUDs/Discounts/Reservations pricing information
- AWS
- GCP
- Azure
Each tagged version of the Cloud Cost Exporter will publish a Docker image to https://hub.docker.com/r/grafana/cloudcost-exporter and a Helm chart.
The image can be used to deploy Cloud Cost Exporter to a Kubernetes cluster or to run it locally.
Cloud Cost Exporter has an opinionated way of authenticating against each cloud provider:
| Provider | Notes |
|---|---|
| GCP | Depends on default credentials |
| AWS | Uses profile names from your credentials file or AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION env variables |
| Azure | Uses the default azure credential chain, e.g. enviornment variables: AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_CLIENT_SECRET |
When running in a Kubernetes cluster, it is recommended to create an IAM role for a Service Account (IRSA) with the necessary permissions for the cloud provider.
Documentation about the necessary permission for AWS can be found here. Documentation for GCP and Azure are under development.
When deploying to Kubernetes, it is recommended to use the Helm chart, which can be found here: https://github.com/grafana/helm-charts/tree/main/charts/cloudcost-exporter
Additional Helm chart configuration for AWS can be found here
Check out the follow docs for metrics:
This project is in the early stages of development and is subject to change. Grafana Labs builds and maintains this project as part of our commitment to the open-source community, but we do not provide support for it. In its current state, the exporter exports rates for resources and not the total spend.
For a better understanding of how we view measuring costs, view a talk given at KubeCon NA 2023
In the future, we intend to opensource recording rules we use internally to measure the spend of resources.
Grafana Labs is always looking to support new contributors! Please take a look at our contributing guide for more information on how to get started.