[Bug]: Problem with the skip_read not working fix 113

[mytchln]

Github handle

mytchln

Please describe the issue you are facing?

With the 2.1.0 i am now getting this:

Stack trace from the terraform-provider-terracurl_v2.1.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x81c7c1]

goroutine 27 [running]:
github.com/devops-rob/terraform-provider-terracurl/internal/provider.(*CurlResource).UpgradeState.func1({0xe77b98, 0xc0003cc360}, {0xc00007f260?, 0x0?}, 0xc0003aaa00)
/Users/rbarnes/Developer/devops-rob/terraform-provider-terracurl/internal/provider/curl_resource.go:951 +0x81
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).UpgradeResourceState(0xc0002da008, {0xe77b98, 0xc0003cc360}, 0xc0003cc8a0, 0xc0000ed5f0)
/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.14.1/internal/fwserver/server_upgraderesourcestate.go:205 +0x94d
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).UpgradeResourceState(0xc0002da008, {0xe77b98?, 0xc0003cc270?}, 0xc00007f280)
/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.14.1/internal/proto6server/server_upgraderesourcestate.go:51 +0x2ce
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).UpgradeResourceState(0xc000262140, {0xe77b98?, 0xc0003a1ad0?}, 0xc0003aa820)
/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.26.0/tfprotov6/tf6server/server.go:752 +0x262
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_UpgradeResourceState_Handler({0xd1e420, 0xc000262140}, {0xe77b98, 0xc0003a1ad0}, 0xc0003a2c80, 0x0)
/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.26.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:539 +0x1a9
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000218200, {0xe77b98, 0xc0003a1a70}, 0xc0001db7a0, 0xc0002dc000, 0x143d938, 0x0)
/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1392 +0xfbe
google.golang.org/grpc.(*Server).handleStream(0xc000218200, {0xe786c8, 0xc000183860}, 0xc0001db7a0)
/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1802 +0xb88
google.golang.org/grpc.(*Server).serveStreams.func2.1()
/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1030 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 10
/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1041 +0x11d

Error: The terraform-provider-terracurl_v2.1.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Please detail the steps to reproduce the issue?

step 1: do this
step 2: do that
step 3: etc

TerraCurl provider version

1.2.2 (Default)

What version of Terraform are you using?

None

What Operating system architecture are you running?

None

Relevant log output

[devops-rob]

hmmmm checking...all the tests were passing so unsure about this

[devops-rob]

can you share the TF resource please? I need as much details as possible @mytchln

[mytchln]

resource "terracurl_request" "post-adsec-ro" {
count = length(var.adsec_ro_group) > 0 ? 1 : 0
name = "set_ro_permissions"
url = "${local.sag_active_base_url}/icacls_set_share_perms"
method = "POST"
skip_tls_verify = true
request_body = jsonencode({
share_path = "${local.sag_dfs_apath}",
permissions = "RO",
operation = "grant",
groups = "${var.adsec_ro_group}"
})

headers = {
Accept = "application/json"
X-API-Key = "${var.sag_api_key}"
Content-type = "application/json"
}

response_codes = [200, 201, 202, 204]

timeout = 45

skip_read = true

depends_on = [
resource.terracurl_request.post-adsec-rw
]
lifecycle {
ignore_changes = all
}

destroy_url = "${local.sag_active_base_url}/dumy"
destroy_method = "POST"
destroy_skip_tls_verify = true
destroy_request_body = jsonencode({
type = "data",
cluster = "b"
})

destroy_headers = {
Accept = "application/json"
X-API-Key = "${var.sag_api_key}"
Content-type = "application/json"
}
destroy_response_codes = [200, 201, 202, 204, 400, 401, 444, 500]
}

resource "terracurl_request" "email_notification" {
name = "email_share_notification"
url = "${local.sag_active_base_url}/send_email"
method = "POST"
skip_tls_verify = true
request_body = jsonencode({
subject = local.email_subject,
sender = local.email_from,
receiver = "${local.email_receivers}",
bcc = "${local.email_bcc}",
html_msg = "${local.email_message}"
})

headers = {
Accept = "application/json"
X-API-Key = "${var.sag_api_key}"
Content-type = "application/json"
}

response_codes = [200, 201, 202, 204]

skip_read = true

depends_on = [
resource.aws_storagegateway_smb_file_share.this,
resource.time_sleep.wait_90_seconds
]
lifecycle {
ignore_changes = all
}

destroy_url = "${local.sag_active_base_url}/dumy"
destroy_method = "POST"
destroy_skip_tls_verify = true
destroy_request_body = jsonencode({
type = "data",
cluster = "CHAPDFASCLSA01"
})

destroy_headers = {
Accept = "application/json"
X-API-Key = "${var.sag_api_key}"
Content-type = "application/json"
}

destroy_response_codes = [200, 201, 202, 204, 400, 401, 444, 500]
}

[mytchln]

• Using previously-installed hashicorp/local v2.5.3
• Using previously-installed devops-rob/terracurl v2.1.0
• Using previously-installed hashicorp/awscc v1.51.0
• Using previously-installed hashicorp/http v3.5.0
• Using previously-installed hashicorp/time v0.13.1
• Using previously-installed hashicorp/archive v2.7.1
• Using previously-installed hashicorp/aws v6.8.0
• Using previously-installed hashicorp/null v3.2.4

[devops-rob]

ok i think i have a fix. If i do a release candidate, are you able to test it @mytchln ?

[mytchln]

Warning: Failed to decode resource from state
│
│ Error decoding
│ "module.A1586346_sgw_nfs_tiaa-ifs-preprod-uw2-ips-at-dcioa-recordkeeper-esm894.terracurl_request.email_notification"
│ from prior state: unsupported attribute "destroy_parameters"
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding
│ "module.A1586346_sgw_nfs_tiaa-ifs-preprod-uw2-ips-at-dcioa-recordkeeper-est842.terracurl_request.email_notification"
│ from prior state: unsupported attribute "destroy_parameters"
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding
│ "module.A1586346_sgw_nfs_tiaa-ifs-preprod-uw2-ips-st4-dcioa-recordkeeper-rk4-jkl.terracurl_request.email_notification"
│ from prior state: unsupported attribute "destroy_parameters"
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding
│ "module.A1586346_sgw_nfs_tiaa-ifs-preprod-uw2-ips-st2-dcioa-recordkeeper-esm894.terracurl_request.email_notification"
│ from prior state: unsupported attribute "destroy_parameters"
╵
╷
│ Warning: Failed to decode resource from state
│
│ Error decoding
│ "module.A1586346_sgw_nfs_tiaa-ifs-preprod-uw2-ips-st4-dcioa-recordkeeper-esm894.terracurl_request.email_notification"
│ from prior state: unsupported attribute "destroy_parameters"

Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵
╷
│ Error: Request cancelled
│
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.

[mytchln]

I can test it

[devops-rob]

thanks @mytchln I've published v2.1.1-rc1.

A member of the terracurl community has put together a set of guidance instructions to test pre-release provider:

Terracurl Beta Testing

Overview

This guide will walk you through setting up and testing the Terracurl Terraform provider, a custom provider for making HTTP requests.

Setup

Step 1: Override Provider Installation

Create a dev.tfrc file with the following contents:

provider_installation {
  dev_overrides {
    "devcastops.com/devops-rob/terracurl" = "./plugins"
  }
  direct {}
}

This will override the provider installation to point to a local plugin directory (./plugins).

Step 2: Declare Provider

Create a terraform.tf file with the following contents:

terraform {
  required_providers {
    terracurl = {
      source  = "devcastops.com/devops-rob/terracurl"
    }
  }
}

provider "terracurl" {
  # Configuration options
}

This will declare the Terracurl provider and its source.

Step 3: Configure CLI

Set the TF_CLI_CONFIG_FILE environment variable to point to the dev.tfrc file:

export TF_CLI_CONFIG_FILE=$(pwd)/dev.tfrc

Step 4: Download Plugin Binary

Download the Terracurl plugin binary for your operating system and extract it to the plugins directory. For Linux AMD64, you can use the following command:

wget -q https://github.com/devops-rob/terraform-provider-terracurl/releases/download/v2.0.0-rc1/terraform-provider-terracurl_2.0.0-rc1_linux_amd64.zip && \
unzip -q terraform-provider-terracurl_2.0.0-rc1_linux_amd64.zip && \
mkdir -p plugins && \
mv terraform-provider-terracurl_v2.0.0-rc1 plugins/ && \
rm terraform-provider-terracurl_2.0.0-rc1_linux_amd64.zip

Step 5: Test the Setup

Create a main.tf file with a simple example configuration that makes a GET request to a fake API and outputs the response as JSON:

resource "terracurl_request" "get_products" {
  name   = "get-products"
  url    = "https://fakestoreapi.com/products?limit=5"
  method = "GET"

  response_codes = [
    200
  ]

}

output "get_products" {
  value = jsondecode(terracurl_request.get_products.response)
}

Run the following commands to test the setup:

terraform fmt
terraform apply -auto-approve

This should output the response from the fake API as JSON.

Let me know how you get on. Thanks

[mytchln]

Well, it appears I can't test. We use "remote" backend to SaaS scalr.io and when I enable the overrides i am unable to connect back to scalr.io

│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - devcastops.com/devops-rob/terracurl in plugins
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵
╷
│ Error: Required token could not be found
│
│ Run the following command to generate a token for tiaa.scalr.io:
│ terraform login company.scalr.io

[apankowski]

tl;dr

@devops-rob unfortunately the RCs don't work for me. Errors seem to be the same or very similar as for official published versions.

Longer version

Similarly to @mytchln I'm trying to upgrade from 1.2.2 to 2.0.0+ - both 2.0.0 and 2.1.0 fail for different reasons for me. 1.2.2 is the last working version.

My resource working on 1.2.2 and problematic on 2.0.0+:

resource "terracurl_request" "init" {
  name = "vault-init"

  method = "POST"
  url    = "${google_cloud_run_v2_service.vault.uri}/v1/sys/init"
  request_body = jsonencode({
    recovery_shares    = var.recovery.shares
    recovery_threshold = var.recovery.threshold
  })
  headers = {
    "Authorization" = "Bearer ${data.google_service_account_id_token.vault_sa.id_token}"
  }
  response_codes = [200]

  max_retry      = 18
  retry_interval = 10

  lifecycle {
    ignore_changes = [headers] # ID token will be different each time - don't re-attempt to initialize
  }
}

2.0.0

I'm getting the following error.

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Read Configuration Error
│ 
│   with module.xyz.terracurl_request.init,
│   on ../modules/xyz/main.tf line 218, in resource "terracurl_request" "init":
│  218: resource "terracurl_request" "init" {
│ 
│ `read_url`, `read_method`, and `read_response_codes` are required when `skip_read` is false.
╵

After adding skip_read = true on the resource, the error stays the same (it seems to be ineffective).

2.1.0

I'm getting the following error:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Warning: Failed to decode resource from state
│ 
│ Error decoding "module.xyz.terracurl_request.init" from prior state: unsupported attribute
│ "destroy_parameters"
╵
╷
│ Error: Request cancelled
│ 
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵

Stack trace from the terraform-provider-terracurl_v2.1.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x81c7c1]

goroutine 52 [running]:
github.com/devops-rob/terraform-provider-terracurl/internal/provider.(*CurlResource).UpgradeState.func1({0xe77b98, 0xc000537500}, {0xc0003a88a0?, 0x0?}, 0xc00039d680)
	/Users/rbarnes/Developer/devops-rob/terraform-provider-terracurl/internal/provider/curl_resource.go:951 +0x81
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).UpgradeResourceState(0xc0002f8488, {0xe77b98, 0xc000537500}, 0xc000537b90, 0xc0005595f0)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.14.1/internal/fwserver/server_upgraderesourcestate.go:205 +0x94d
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).UpgradeResourceState(0xc0002f8488, {0xe77b98?, 0xc000537410?}, 0xc0003a88c0)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.14.1/internal/proto6server/server_upgraderesourcestate.go:51 +0x2ce
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).UpgradeResourceState(0xc0002fa140, {0xe77b98?, 0xc000536540?}, 0xc00039d130)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.26.0/tfprotov6/tf6server/server.go:752 +0x262
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_UpgradeResourceState_Handler({0xd1e420, 0xc0002fa140}, {0xe77b98, 0xc000536540}, 0xc0003b1380, 0x0)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.26.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:539 +0x1a9
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000208200, {0xe77b98, 0xc0005364e0}, 0xc000381a40, 0xc000304a50, 0x143d938, 0x0)
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1392 +0xfbe
google.golang.org/grpc.(*Server).handleStream(0xc000208200, {0xe786c8, 0xc00047c000}, 0xc000381a40)
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1802 +0xb88
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1030 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 39
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.69.4/server.go:1041 +0x11d

Error: The terraform-provider-terracurl_v2.1.0 plugin crashed!

2.0.0-rc1

I see:

╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - devops-rob/terracurl in /home/xyz/plugins
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may
│ cause the state to become incompatible with published releases.
╵

I see in TF_LOG=debug that the plugin is loaded.

I get the following error:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Read Configuration Error
│ 
│   with module.xyz.terracurl_request.init,
│   on ../modules/xyz/main.tf line 218, in resource "terracurl_request" "init":
│  218: resource "terracurl_request" "init" {
│ 
│ `read_url`, `read_method`, and `read_response_codes` are required when `skip_read` is false.
╵

After adding skip_read = true on the resource, the error stays the same (it seems to be ineffective).

2.1.1-rc1

I see:

╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - devops-rob/terracurl in /home/xyz/plugins
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may
│ cause the state to become incompatible with published releases.
╵

I see in TF_LOG=debug that the plugin is loaded.

I get the following error:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Warning: Failed to decode resource from state
│ 
│ Error decoding "module.xyz.terracurl_request.init" from prior state: unsupported attribute
│ "destroy_parameters"
╵
╷
│ Error: Request cancelled
│ 
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵

Stack trace from the terraform-provider-terracurl_v2.1.1-rc1 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8204e6]

goroutine 36 [running]:
github.com/devops-rob/terraform-provider-terracurl/internal/provider.(*CurlResource).UpgradeState.func1({0xeb29a8, 0xc0004d16b0}, {0xc00049c540?, 0x0?}, 0xc00048c8c0)
	/Users/rbarnes/Developer/devops-rob/terraform-provider-terracurl/internal/provider/curl_resource.go:950 +0x66
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).UpgradeResourceState(0xc0000faa08, {0xeb29a8, 0xc0004d16b0}, 0xc0004d1d40, 0xc0004ab600)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.15.1/internal/fwserver/server_upgraderesourcestate.go:205 +0x94d
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).UpgradeResourceState(0xc0000faa08, {0xeb29a8?, 0xc0004d15c0?}, 0xc00049c560)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-framework@v1.15.1/internal/proto6server/server_upgraderesourcestate.go:51 +0x2ce
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).UpgradeResourceState(0xc000212500, {0xeb29a8?, 0xc0004d06c0?}, 0xc00048c370)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.28.0/tfprotov6/tf6server/server.go:797 +0x262
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_UpgradeResourceState_Handler({0xd4c280, 0xc000212500}, {0xeb29a8, 0xc0004d06c0}, 0xc0004c8100, 0x0)
	/Users/rbarnes/go/pkg/mod/github.com/hashicorp/terraform-plugin-go@v0.28.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:597 +0x1a9
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000208200, {0xeb29a8, 0xc0004d0630}, 0xc0004b56e0, 0xc000300f60, 0x148b490, 0x0)
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1405 +0x1036
google.golang.org/grpc.(*Server).handleStream(0xc000208200, {0xeb3490, 0xc0004f6000}, 0xc0004b56e0)
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1815 +0xb88
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1035 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 35
	/Users/rbarnes/go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1046 +0x11d

Error: The terraform-provider-terracurl_v2.1.1-rc1 plugin crashed!