Deprecation of Jsonnet dashboard type

[theSuess]

The operator currently supports jsonnet as an option to specify dashboards. This allows users to use the jsonnet data templating language in their resources directly, which will then be applied to Grafana as a rendered JSON object.

Jsonnet doesn't add much functionality without libraries. This is why we bundled the (then still maintained) grafonnet-lib library with the operator.

Over time, this library got replaced by grafonnet (without -lib) and this change never made it into the operator.

The clean solution to this would be a system, in which users can add their own libraries to the operator (as requested in #2161).

We discussed this in today's weekly meeting and the prevailing opinion is that we should rather spend our effort elsewhere, as a clean solution would be a significant undertaking that doesn't add much value for most users.

As such, we propose to officially deprecate this feature. It'll still be available, but won't receive any new functionality until its eventual deletion when we decide to release an updated CRD version.

A possible workaround for heavy users of this feature would be to evaluate jsonnet one level higher and use it to generate the GrafanaDashboard itself. Tools like tanka or kubecfg are purpose built for this and allow for dependency management using jsonnet-bundler.

This community post should also serve as an opinion poll to validate our assumption that this feature is not heavy used. So if you have a strong opinion, please don't hesitate to comment here.

[siegenthalerroger]

I use the jsonnet feature purely to allow the templating of a value in the dashboard's json from a configmap provided env-variable. This allows us to share a dashboard across many tenants and provide a value for the dashboard's selector from a single place. As such, we never really used any of the jsonnet libraries and I agree with the argumentation that that kind of usage is better handled by a separate build pipeline. But some simple replace logic would still be a great feature to keep.

While we could build our dashboards in a jsonnet ci pipeline, but that would be a lot more convoluted for our usecase, with the sources not being as "close" to the rest of the tenant deployments.

kind: GrafanaDashboard
metadata:
  ...
spec:
  ...
  envFrom:
    - configMapKeyRef:
        name: observability-dashboard-selected-cluster-options
        key: SELECTED_CLUSTER_OPTIONS
  jsonnet: |
    local selectedClusterOptions = std.extVar('SELECTED_CLUSTER_OPTIONS');

    {
      "annotations": {
        "list": [
    ...
      "templating": {
        "list": [
          {
            ...
            "options": selectedClusterOptions,
          },
  ...

[theSuess]

This is a good point. A generic variable replacement feature has been requested a lot, so we can try and time this to be available before we deprecate jsonnet support

[sebhoss]

Behold!

I've created https://github.com/metio/jaas (Jsonnet-as-a-Service) which works together with https://github.com/metio/jsonnet-oci-images to allow writing dashboards in jsonnet. The idea here is to use OCI volumes to package both jsonnet libraries as well as individual Grafana dashboards. Jsonnet-as-a-Service then allows downloading the dashboards through a URL which can be used by the grafana-operator to install these dashboards in a running Grafana instance.

There is a helm chart at https://github.com/metio/jaas/tree/main/helm which can be configured to load additional libraries as well as any kind of jsonnet snippet (e.g., a Grafana dashboard).

[sebhoss]

That's an awesome idea! I was thinking about how to use JaaS to produce k8s resources, but did not know about ExternalArtifact - thanks a lot! Opened metio/jaas#6 to track that ❤️

[theSuess]

This is a cool effort! If you want to, we can document a full example around this in the operator documentation as the recommended way to do jsonnet dashboards in combination with the operator

[sebhoss]

yeah that sounds awesome! I already created some documentation at https://github.com/metio/jaas/tree/main/helm, but would love to extend that and make it official 🥳 Should I just replace the example in https://github.com/grafana/grafana-operator/tree/master/examples/dashboard/jsonnet or rather create a new one next to it?

[weisdd]

@sebhoss I think it would be better to have an extra folder with a self-contained example, and then we can cross-reference the docs (= nudge users to switch to jaas). - As long as we support the embedded jsonnet, we should still have an example of its usage.

[sebhoss]

OK - created #2370 for that