Can't upgrade from v8 to v9

[sam-utila]

Describe what happened

After upgrading from v8 to v9 I am getting the following error (due to bad state migration)

Previewing update (default):
     Type                   Name                 Plan     Info
     pulumi:pulumi:Stack    pulumi-test-default
     └─ gcp:storage:Bucket  bucket                        2 errors

Diagnostics:
  gcp:storage:Bucket (bucket):
    error:   sdk-v2/provider2.go:572: sdk.helper_schema: retention_period in state has unexpected type string: provider=google-beta@9.3.0
    error: diffing urn:pulumi:default::pulumi-test::gcp:storage/bucket:Bucket::bucket: 1 error occurred:
    	* retention_period in state has unexpected type string

Sample program

See attachment for complete program with all deps

pulumi-gcp-v9-upgrade-error.zip

Log output

No response

Affected Resource(s)

No response

Output of pulumi about

CLI
Version      3.199.0
Go Version   go1.25.1
Go Compiler  gc

Plugins
KIND      NAME  VERSION
resource  gcp   9.3.0
language  go    3.199.0

Host
OS       darwin
Version  15.6.1
Arch     arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.25.1 darwin/arm64'

Current Stack: default

TYPE                       URN
pulumi:pulumi:Stack        urn:pulumi:default::pulumi-test::pulumi:pulumi:Stack::pulumi-test-default
pulumi:providers:gcp       urn:pulumi:default::pulumi-test::pulumi:providers:gcp::default_8_32_1
gcp:storage/bucket:Bucket  urn:pulumi:default::pulumi-test::gcp:storage/bucket:Bucket::bucket


Found no pending operations associated with default

Backend
Name           REDACTED
URL            file://~
User           REDACTED
Organizations
Token type     personal

Dependencies:
NAME                                 VERSION
github.com/pulumi/pulumi-gcp/sdk/v9  v9.3.0
github.com/pulumi/pulumi/sdk/v3      v3.190.0

Pulumi locates its logs in /var/folders/yb/6v8z95_x0h5gsqwc6sfrsh8m0000gn/T/ by default

Additional context

See additional context in slack channel:

https://pulumi-community.slack.com/archives/CRFUR2DGB/p1759225241158029

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[corymhall]

@sam-utila thanks for reporting this! It looks like this is due to how the upstream provider has coded the state migration for this field. In Terraform it is fine, but for Pulumi it is causing this error message. The current workaround is to upgrade in a two step process, first to a version that has the RawStateUpgrade enabled.

1. Upgrade to v8.41.1 and run pulumi up
2. Upgrade to v9.3.0 and run pulumi up --refresh

Alternatively you should also be able to do:

1. Upgrade to v8.41.1 and run pulumi up --refresh --run-program
2. Upgrade to v9.3.0 and run pulumi up

I've created a PR upstream to see if they will accept a fix, I'll update the status if they do.

[sam-utila]

Thanks @corymhall!

I was thinking of doing the same, but from what I understand, the issue is totally a pulumi only thing (some lost state information in previous versions) - so the Terraform fix is a bit of a workaround for Pulumi and not a real bug in magic-modules (aka Terraform).

Just from "politics" perspective, I find it hard to upstream Pulumi fixes into Terraform code - specifically here I am not sure who is in charge here, GCP or Terraform, but I would assume it should be nice to have GCP on board here with Pulumi.

I have had many problems with the GCP bridged provider, mainly because Terraform behaves differently than Pulumi but the provider development team is agnostic to Pulumi. (for example Terraform refreshing the state of every changed resource by default while Pulumi does not - caused weird behavior for us that terraform users do not experience)