Skip to content

newrelic_notification_channel: orphaned resources when API returns error alongside valid ID #3037

@dariocazas

Description

@dariocazas

Terraform Version, Provider Version and New Relic environment

  • Terraform: v1.6.3
  • Provider: v3.80.x (bug present since notification resources were introduced)
  • New Relic: US and Staging regions

Expected Behavior

When newrelic_notification_channel (or newrelic_notification_destination) is created and the NR API returns a valid resource ID, the resource should be recorded in Terraform state — even if the response also contains errors.

Actual Behavior

The resource is created in New Relic but not recorded in Terraform state, producing an orphaned resource and an infinite create→fail loop on subsequent applies.

The error shown is:

UNKNOWN_ERROR: 

(empty description and details — the API returned null for both fields)

Root Cause

AiNotificationsChannelResponse can contain both a valid Channel (with ID) and entries in the Errors array simultaneously. The Create function checks errors before calling d.SetId():

// resource_newrelic_notifications_channel.go:111-116
errors := buildAiNotificationsErrors(channelResponse.Errors)
if len(errors) > 0 {
    return errors                            // ← exits here
}
d.SetId(channelResponse.Channel.ID)          // ← never reached

The same pattern exists in resource_newrelic_notifications_destination.go.

Steps to Reproduce

This occurs intermittently (the NR backend returns UNKNOWN_ERROR with null fields on some channel creations despite the channel being created). We've observed it consistently during alert policy migrations involving PagerDuty notification channels (PAGERDUTY_ACCOUNT_INTEGRATION).

  1. Define a newrelic_notification_channel resource
  2. Run terraform apply
  3. If the NR API returns a response with both a valid channel ID and an UNKNOWN_ERROR:
    • Apply fails with UNKNOWN_ERROR:
    • Channel exists in NR (verifiable via NerdGraph aiNotifications.channels query)
    • Channel is not in Terraform state
  4. Subsequent terraform apply attempts to create the channel again → same error → loop

Impact

  • Orphaned resources accumulate in New Relic (channels exist but aren't managed by Terraform)
  • Blocked workflows: terraform apply fails indefinitely until manual terraform import
  • Affects both newrelic_notification_channel and newrelic_notification_destination

Suggested Fix

Move d.SetId() before the error check, guarded by a non-empty ID:

if channelResponse.Channel.ID != "" {
    d.SetId(channelResponse.Channel.ID)
}

errors := buildAiNotificationsErrors(channelResponse.Errors)
if len(errors) > 0 {
    return errors
}

Per the Terraform Plugin SDK documentation, SetId must be called with a non-empty value for the resource to be saved into state. Per the Plugin Framework Create Caveats, any response errors will cause Terraform to mark the resource as tainted — which is the correct behavior here: the resource exists but may need reconciliation.

Workaround

Query NerdGraph for orphaned channels and import them manually:

{
  actor {
    account(id: ACCOUNT_ID) {
      aiNotifications {
        channels(filters: {destinationId: "DESTINATION_ID"}) {
          entities { id name status }
        }
      }
    }
  }
}
terraform import 'newrelic_notification_channel.example' CHANNEL_ID

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions