Parameterise the Prometheus Data Source in Grafana Dashboards

[martincostello]

Each panel in the Grafana dashboards are tied directly to a specific datasource ID at the point it is imported, rather than being a variable in of itself.

https://github.com/dotnet/aspire/blob/fc0d681c0921984da03e61dcbf69ce5528a658d3/src/Grafana/dashboards/aspnetcore.json#L77-L80

In the case that a Grafana instance has multiple data sources, this makes it difficult to switch between different deployments of an application in the same dashboard.

It would be useful if the dashboards instead defined a Data Source variable and referenced that instead for easy switching.

For example:

{
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${prometheus}"
      }
      // The actual panel...
    }
  ],
  "templating": {
    "list": [
      {
        "current": {
          "selected": true,
          "text": "Prometheus Production-EU-WEST-1",
          "value": "Prometheus Production-EU-WEST-1"
        },
        "hide": 0,
        "includeAll": false,
        "multi": false,
        "name": "prometheus",
        "options": [],
        "query": "prometheus",
        "queryValue": "",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      }
    ]
  }
}

[josephaw1022]

Ties in directly to dotnet/aspire#1037 and discussions in the pr for the issue dotnet/aspire#1173 (comment)

And more specifically
dotnet/aspire#1173 (comment)

Probably going to have a solution for this problem this week

[martincostello]

Just to be clear, I'm not referring to usage of Aspire itself and it producing any dashboards. I took the dashboards from here and then imported them into our instance of Grafana directly.

[josephaw1022]

@martincostello Sorry, I was skimming through this during my lunch yesterday and just went through this too quickly. But I read this again yesterday and again today and I think I know what you're getting at now.

So, currently the Prometheus data source in the json file here seems to be only configured to point to what is supposed to be an "aspire orchestrated" prometheus instance.

For clarification

So the current functionality seems to be

graph TD
    subgraph prod ["Production Environment"]
        grafanaProd[(prod Grafana)] --> prometheusProd[(prod Prometheus)]
        prometheusProd --> dataSourceProd[(prod scrape targets)]
    end
    subgraph dev ["Development Environment"]
        grafanaDev[(dev Grafana)] --> prometheusDev[(dev Prometheus)]
        prometheusDev --> dataSourceDev[(dev scrape targets)]
    end
    subgraph qa ["QA Environment"]
        grafanaQA[(qa Grafana)] --> prometheusQA[(qa Prometheus)]
        prometheusQA --> dataSourceQA[(qa scrape targets)]
    end

Loading

And what I think you are suggesting is configuring it to point to potentially 1-many datasources (that you can toggle between in the ui)

graph TD
    grafana[(Grafana)]
    grafana --> prometheusDev[(dev Prometheus)]
    grafana --> prometheusProd[(prod Prometheus)]
    grafana --> prometheusQA[(qa Prometheus)]

Loading

NOTE 🚧
I use Prometheus (as the DataSource) in these mermaid charts, but it can be whatever datasource you have in mind. And I just used dev, prod, and qa as a way to get the point across, it can be whatever you want it to be.

But from the wording of this, it also seems like you want the the list of data sources to initially include what is currently the "singular DataSource" that's in the json. keeping it simple initially but also "appendable" for easy toggling

Is this what you have in mind?

[martincostello]

Is this what you have in mind?

Yep, that's it exactly. I'd like to be able to use a single dashboard to pivot between environments/deployments from a single dashboard.

For context, we have two instances of Grafana: one for non-production and one for production. All non-production environments use one, and all production environments use the other.

We have many environments of each type, but the preference would be to import a dashboard once for each Grafana instance, and then easily pivot between an environment to see the health of an application in two different AWS accounts (which are broadly equivalent to an "environment") in the same dashboard without changing the URL/losing context/etc.