Skip to content

grafana_oncall_integration: dynamic_labels causes perpetual drift due to TypeList ordering #2559

@derrix060

Description

@derrix060

Terraform Version

OpenTofu 1.11.5 (also reproducible with Terraform)

Terraform Grafana Provider Version

Tested with the latest available provider version.

Affected Resource(s)

  • grafana_oncall_integration

Description

The dynamic_labels attribute on grafana_oncall_integration is defined as schema.TypeList in internal/resources/oncall/resource_integration.go. Because TypeList is ordered, Terraform tracks elements by index position. However, the Grafana OnCall API returns labels in a non-deterministic order (likely by internal ID or creation time), which differs from the order Terraform sends them in.

This causes perpetual drift on every terraform plan — even immediately after a successful terraform apply. The plan shows labels being removed from one position and re-added at another, despite the actual label set being identical.

Example plan output (immediately after apply)

~ dynamic_labels = [
    - {
        - "id"    = "service_name"
        - "key"   = "service_name"
        - "value" = "{{ payload.commonLabels.service_name }}"
      },
      {
          "id"    = "namespace"
          "key"   = "namespace"
          "value" = "{{ payload.commonLabels.namespace }}"
      },
      # (1 unchanged element hidden)
      {
          "id"    = "project"
          "key"   = "project"
          "value" = "{{ payload.commonLabels.project }}"
      },
    + {
        + "id"    = "service_name"
        + "key"   = "service_name"
        + "value" = "{{ payload.commonLabels.service_name }}"
      },
      {
          "id"    = "severity"
          "key"   = "severity"
          "value" = "{{ payload.commonLabels.severity }}"
      },
      # (1 unchanged element hidden)
  ]

The label service_name is removed from position 0 and re-added at position 3. The actual content is identical — only the order differs.

Root Cause

In internal/resources/oncall/resource_integration.go:

"dynamic_labels": {
    Type: schema.TypeList,
    // ...
},

The flattenLabels function preserves the API's return order:

func flattenLabels(labels []*onCallAPI.Label) []map[string]string {
    flattenedLabels := make([]map[string]string, 0, 1)
    for _, l := range labels {
        flattenedLabels = append(flattenedLabels, map[string]string{
            "id":    l.Key.Name,
            "key":   l.Key.Name,
            "value": l.Value.Name,
        })
    }
    return flattenedLabels
}

Since the API returns labels in a different order than what was sent, and TypeList is order-sensitive, every plan detects drift.

Expected Behavior

terraform plan should show no changes after a successful terraform apply when the set of dynamic labels has not changed.

Proposed Fix

Change dynamic_labels (and likely labels too, since they share the same pattern) from schema.TypeList to schema.TypeSet with an appropriate hash function. This makes the attribute order-insensitive, which matches the actual semantics — label order is not meaningful in Grafana OnCall.

Alternatively, sort the labels by key in flattenLabels before setting the state, so the order is deterministic regardless of API response ordering.

Workaround

The only current workaround is using lifecycle { ignore_changes = [dynamic_labels] }, which prevents Terraform from managing label changes entirely.

Steps to Reproduce

  1. Create a grafana_oncall_integration with 2+ dynamic labels
  2. Run terraform apply
  3. Run terraform plan immediately after
  4. Observe that the plan shows changes (label reordering)
  5. Apply again, plan again — drift persists indefinitely

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions