Skip to content

[7.0.0-alpha]: Upgrade upstream to v6.0.0-beta1 #5511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 16, 2025

Conversation

corymhall
Copy link
Contributor

@corymhall corymhall commented May 9, 2025

Upgrade upstream to v6.0.0-beta1

This PR upgrades upstream to the new v6.0.0-beta1 tag

The major update to the SDKs in this update is the addition of the new region resource level property

How to review

  1. commit Update patches for
    Contains the changes to patches. Pretty minor changes
  2. commit this commit does a couple of things
    This is the bulk of the changes I made in this commit
  3. commit fix tests
    Contains all the updates I had to make to the tests. I also regenerated all of our upgrade tests so they are testing the upgrade from v6 => v7.

The SDK changes are all in a single commit. Pretty much every resource is updated, but almost all the changes are related to adding the region parameter.

re #5226

@corymhall
Copy link
Contributor Author

corymhall commented May 9, 2025

This change is part of the following stack:

Change managed by git-spice.

@corymhall corymhall force-pushed the corymhall/v6-beta1-upgrade branch 2 times, most recently from d1c99a6 to cc11ec3 Compare May 9, 2025 19:51
@corymhall corymhall force-pushed the corymhall/v6-beta1-upgrade branch from cc11ec3 to 923da17 Compare May 12, 2025 17:00
@corymhall corymhall requested review from t0yv0, guineveresaenger and a team May 12, 2025 17:57
Copy link

github-actions bot commented May 12, 2025

Does the PR have any schema changes?

Found 97 breaking changes:

Resources

  • 🟡 "aws:acm/certificate:Certificate": inputs: "tagsAll" missing
  • 🟡 "aws:appintegrations/dataIntegration:DataIntegration": inputs: "tagsAll" missing
  • 🟢 "aws:auditmanager/assessment:Assessment": required: "roles" property is no longer Required
  • 🟡 "aws:budgets/budget:Budget": inputs: "tagsAll" missing
  • 🟡 "aws:budgets/budgetAction:BudgetAction": inputs: "tagsAll" missing
  • 🟢 "aws:cfg/aggregateAuthorization:AggregateAuthorization": required: "region" property is no longer Required
  • 🟡 "aws:chime/sdkvoiceSipMediaApplication:SdkvoiceSipMediaApplication": inputs: "tagsAll" missing
  • 🟡 "aws:chime/sdkvoiceVoiceProfileDomain:SdkvoiceVoiceProfileDomain": inputs: "tagsAll" missing
  • 🟡 "aws:cleanrooms/collaboration:Collaboration": inputs: "tagsAll" missing
  • 🟡 "aws:cleanrooms/configuredTable:ConfiguredTable": inputs: "tagsAll" missing
  • 🟡 "aws:cloudfront/distribution:Distribution": inputs: "tagsAll" missing
  • 🟡 "aws:codebuild/fleet:Fleet": inputs: "tagsAll" missing
  • 🟡 "aws:controltower/landingZone:LandingZone": inputs: "tagsAll" missing
  • 🟡 "aws:customerprofiles/domain:Domain": inputs: "tagsAll" missing
  • 🟡 "aws:datasync/locationAzureBlob:LocationAzureBlob": inputs: "tagsAll" missing
  • 🟡 "aws:datasync/locationFsxOntapFileSystem:LocationFsxOntapFileSystem": inputs: "tagsAll" missing
  • "aws:datazone/environmentBlueprintConfiguration:EnvironmentBlueprintConfiguration":
    • 🟡 inputs: "regionalParameters": additional properties type changed from "object" to "string":
      • 🟡 additional properties had &{Type:string Ref: AdditionalProperties: Items: OneOf:[] Discriminator: Plain:false} but now has no type
    • 🟡 properties: "regionalParameters": additional properties type changed from "object" to "string":
      • 🟡 additional properties had &{Type:string Ref: AdditionalProperties: Items: OneOf:[] Discriminator: Plain:false} but now has no type
  • 🟡 "aws:dms/replicationConfig:ReplicationConfig": inputs: "tagsAll" missing
  • 🟡 "aws:eks/accessEntry:AccessEntry": inputs: "tagsAll" missing
  • 🟡 "aws:emrcontainers/jobTemplate:JobTemplate": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxCluster:KxCluster": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxDatabase:KxDatabase": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxDataview:KxDataview": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxEnvironment:KxEnvironment": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxScalingGroup:KxScalingGroup": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxUser:KxUser": inputs: "tagsAll" missing
  • 🟡 "aws:finspace/kxVolume:KxVolume": inputs: "tagsAll" missing
  • 🟡 "aws:globalaccelerator/customRoutingAccelerator:CustomRoutingAccelerator": inputs: "tagsAll" missing
  • 🟡 "aws:glue/dataQualityRuleset:DataQualityRuleset": inputs: "tagsAll" missing
  • 🟡 "aws:imagebuilder/workflow:Workflow": inputs: "tagsAll" missing
  • 🟡 "aws:iot/caCertificate:CaCertificate": inputs: "tagsAll" missing
  • 🟡 "aws:iot/domainConfiguration:DomainConfiguration": inputs: "tagsAll" missing
  • 🟡 "aws:lb/trustStore:TrustStore": inputs: "tagsAll" missing
  • 🟡 "aws:msk/replicator:Replicator": inputs: "tagsAll" missing
  • 🟡 "aws:msk/vpcConnection:VpcConnection": inputs: "tagsAll" missing
  • 🟡 "aws:organizations/resourcePolicy:ResourcePolicy": inputs: "tagsAll" missing
  • 🟡 "aws:quicksight/theme:Theme": inputs: "tagsAll" missing
  • 🟡 "aws:rds/customDbEngineVersion:CustomDbEngineVersion": inputs: "tagsAll" missing
  • 🟡 "aws:sagemaker/hub:Hub": inputs: "tagsAll" missing
  • 🟡 "aws:sagemaker/mlflowTrackingServer:MlflowTrackingServer": inputs: "tagsAll" missing
  • 🟡 "aws:sagemaker/pipeline:Pipeline": inputs: "tagsAll" missing
  • 🟡 "aws:transfer/agreement:Agreement": inputs: "tagsAll" missing
  • 🟡 "aws:transfer/connector:Connector": inputs: "tagsAll" missing
  • 🟡 "aws:transfer/profile:Profile": inputs: "tagsAll" missing
  • 🟡 "aws:verifiedaccess/endpoint:Endpoint": inputs: "tagsAll" missing
  • 🟡 "aws:verifiedaccess/group:Group": inputs: "tagsAll" missing
  • 🟡 "aws:verifiedaccess/instance:Instance": inputs: "tagsAll" missing
  • 🟡 "aws:verifiedaccess/trustProvider:TrustProvider": inputs: "tagsAll" missing
  • 🟡 "aws:vpclattice/accessLogSubscription:AccessLogSubscription": inputs: "tagsAll" missing

Functions

  • 🔴 "aws:amp/getDefaultScraperConfiguration:getDefaultScraperConfiguration" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:auditmanager/getControl:getControl": inputs: "controlMappingSources" missing input "controlMappingSources"
  • 🟡 "aws:auditmanager/getFramework:getFramework": inputs: "controlSets" missing input "controlSets"
  • 🔴 "aws:bedrock/getCustomModels:getCustomModels" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:bedrock/getInferenceProfiles:getInferenceProfiles" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:cloudfront/getLogDeliveryCanonicalUserId:getLogDeliveryCanonicalUserId" signature change (Args, pulumi.InvokeOptions)->T => (pulumi.InvokeOptions)->T:
    • 🟡 inputs: "region" missing input "region"
  • "aws:devopsguru/getResourceCollection:getResourceCollection": inputs:
    • 🟡 "cloudformations" missing input "cloudformations"
    • 🟡 "tags" missing input "tags"
  • 🔴 "aws:directconnect/getLocations:getLocations" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ebs/getDefaultKmsKey:getDefaultKmsKey" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ebs/getEncryptionByDefault:getEncryptionByDefault" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ec2/getSerialConsoleAccess:getSerialConsoleAccess" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ec2/getSpotDatafeedSubscription:getSpotDatafeedSubscription" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • "aws:ec2/getVpcPeeringConnection:getVpcPeeringConnection": inputs:
    • 🟡 "peerRegion" missing input "peerRegion"
    • 🟡 "region" missing input "region"
  • 🔴 "aws:ecr/getRepositories:getRepositories" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ecrpublic/getAuthorizationToken:getAuthorizationToken" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:ecs/getClusters:getClusters" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:eks/getClusters:getClusters" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:emr/getSupportedInstanceTypes:getSupportedInstanceTypes": inputs: "supportedInstanceTypes" missing input "supportedInstanceTypes"
  • 🔴 "aws:inspector/getRulesPackages:getRulesPackages" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:iot/getRegistrationCode:getRegistrationCode" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:lambda/getFunctions:getFunctions" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:oam/getLinks:getLinks" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:oam/getSinks:getSinks" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:outposts/getSites:getSites" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:redshift/getDataShares:getDataShares": inputs: "dataShares" missing input "dataShares"
  • 🟡 "aws:redshift/getProducerDataShares:getProducerDataShares": inputs: "dataShares" missing input "dataShares"
  • 🔴 "aws:route53/getProfilesProfiles:getProfilesProfiles" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:s3/getCanonicalUserId:getCanonicalUserId" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🔴 "aws:s3/getDirectoryBuckets:getDirectoryBuckets" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:servicequotas/getTemplates:getTemplates": inputs: "templates" missing input "templates"
  • 🔴 "aws:ses/getActiveReceiptRuleSet:getActiveReceiptRuleSet" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:ssoadmin/getApplication:getApplication": inputs: "portalOptions" missing input "portalOptions"
  • 🟡 "aws:ssoadmin/getApplicationAssignments:getApplicationAssignments": inputs: "applicationAssignments" missing input "applicationAssignments"
  • 🟡 "aws:ssoadmin/getApplicationProviders:getApplicationProviders": inputs: "applicationProviders" missing input "applicationProviders"
  • 🔴 "aws:ssoadmin/getInstances:getInstances" signature change (pulumi.InvokeOptions)->T => (Args, pulumi.InvokeOptions)->T
  • 🟡 "aws:synthetics/getRuntimeVersions:getRuntimeVersions": inputs: "runtimeVersions" missing input "runtimeVersions"

Types

  • "aws:auditmanager/getControlControlMappingSource:getControlControlMappingSource":
    • 🟡 properties: "sourceKeyword" missing
    • 🟢 required: "sourceKeywords" property has changed to Required
  • 🟢 "aws:auditmanager/getFrameworkControlSet:getFrameworkControlSet": required: "controls" property has changed to Required
  • 🟢 "aws:ssoadmin/getApplicationPortalOption:getApplicationPortalOption": required: "signInOptions" property has changed to Required
  • 🟢 "aws:ssoadmin/getApplicationProvidersApplicationProvider:getApplicationProvidersApplicationProvider": required: "displayDatas" property has changed to Required
  • 🟢 "aws:vpclattice/ListenerRuleMatch:ListenerRuleMatch": required: "httpMatch" property has changed to Required

New resources:

  • ec2/defaultCreditSpecification.DefaultCreditSpecification

Maintainer note: consult the runbook for dealing with any breaking changes.

Copy link

codecov bot commented May 12, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (7.0.0-alpha@ca9aaf3). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             7.0.0-alpha    #5511   +/-   ##
==============================================
  Coverage               ?   24.43%           
==============================================
  Files                  ?      360           
  Lines                  ?   143138           
  Branches               ?        0           
==============================================
  Hits                   ?    34969           
  Misses                 ?   108069           
  Partials               ?      100           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created follow up to re-enable.
#5518

@@ -1932,6 +1932,9 @@ compatibility shim in favor of the new "name" field.`)
"aws_internet_gateway_attachment": {Tok: awsResource(ec2Mod, "InternetGatewayAttachment")},
"aws_ec2_image_block_public_access": {Tok: awsResource(ec2Mod, "ImageBlockPublicAccess")},
"aws_ec2_instance_metadata_defaults": {Tok: awsResource(ec2Mod, "InstanceMetadataDefaults")},
"aws_ec2_default_credit_specification": {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from master. These updates were included in both v5 & v6-beta

Base automatically changed from corymhall/remove-tags-patches-v7 to corymhall/upstream-v6-beta May 13, 2025 12:48
@corymhall corymhall force-pushed the corymhall/v6-beta1-upgrade branch from 0acf7d5 to 2282852 Compare May 13, 2025 13:43
Base automatically changed from corymhall/upstream-v6-beta to 7.0.0-alpha May 14, 2025 15:12
@corymhall corymhall force-pushed the corymhall/v6-beta1-upgrade branch from 2282852 to dcb6976 Compare May 14, 2025 15:56
@@ -67,10 +67,6 @@ func TestAccPluginFramework(t *testing.T) {
integration.ProgramTest(t, &test)
}

func TestBucketUpgrade(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have much more robust bucket upgrade tests now. Removing this simple one.

@@ -394,85 +372,6 @@ func TestNonIdempotentSnsTopic(t *testing.T) {
require.ErrorContains(t, err, "already exists")
}

func TestOpenZfsFileSystemUpgrade(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test was specific to testing the removal of MaxItemsOne during a minor version update and testing we could go from a string to a list. This isn't relevant anymore now that we are going from v6 to v7.

//
// Updated in https://github.com/pulumi/pulumi-aws/pull/3881
// replacements.
func TestMigrateRdsInstance(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this since it tested an issue with going from v5 to v6. Not relevant anymore for v6 to v7.

if isRegionOverrideEnabled && getAttribute != nil {
if region, ok := getAttribute(names.AttrRegion); ok {
- overrideRegion = region.(string)
+ if val, ok := region.(string); ok {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: once merged create tracking issue. Maybe a bridge issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's overrideRegion prior to this line? The remaining code is in another patch, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of the code is just upstream code. The only patch we have in this code is what i've added here.

The line above which isn't included in the patch just initializes overrideRegion.

var overrideRegion string

getAttribute is a function looks up the attribute in the rawState, essentially something like this:

getAttribute := func(key: string): (any, bool) {
  v, ok := rawState[key]; return v, ok
}

In Terraform it looks like at the point this code is running rawState does not have a region attribute at all so getAttribute returns false. In Pulumi for some reason rawState does have a region attribute with a nil value. I think it's a bridge issue so I'll create a follow up issue.

corymhall added 6 commits May 15, 2025 14:43
Add new patch for region issue
- Adds a `PreCheckCallback` for the `region` property similar to how we
  handle `tagsAll`
  - Because of this I also refactored out the `tagsAll` handling into a
    separate function.
- Added missing upstream resources added in latest release
- Ran `make tfgen`
@corymhall corymhall force-pushed the corymhall/v6-beta1-upgrade branch from ad45f4e to e34bb4e Compare May 15, 2025 19:39
if isRegionOverrideEnabled && getAttribute != nil {
if region, ok := getAttribute(names.AttrRegion); ok {
- overrideRegion = region.(string)
+ if val, ok := region.(string); ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's overrideRegion prior to this line? The remaining code is in another patch, right?

// doesn't work with the way the bridge/pulumi works. The interceptor will add the region after
// Pulumi runs `Check` which causes issues. This workaround sets the `region` the same way upstream will
// but does it on the pulumi side before `Check`
// TODO[pulumi/pulumi-aws#5521]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we just consider these precheck callbacks a temporary workaround then? What would a permanent solution look like? Would the suggestion here be to align Pulumi with TF where refresh behavior is concerned?

I suppose my question boils down to: why would we still be seeing a Region diff when we're setting it here before Check?

Copy link
Contributor Author

@corymhall corymhall May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two separate issues here that are both at play.

  1. In Pulumi all input properties have to be set in Check. If input properties are added after Check then it causes weird diff issues. Terraform is setting the region input property in an interceptor that runs for us after Check. The PreCheckCallback is a way for us to make sure the region is set prior to Check so the diff shows correctly. Related to Consider moving planning to be performed in Check pulumi-terraform-bridge#2281

For example, by default we get a diff like this (because we didn't set the region value in Check)

      }
    ~ aws:s3/bucketOwnershipControls:BucketOwnershipControls: (update)
        [id=loggingbucket-0c177aa]
        [urn=urn:pulumi:chall::test-aws-bucket-migration::aws:s3/bucketOwnershipControls:BucketOwnershipControls::exampleBucketOwnershipControls]
        [provider=urn:pulumi:chall::test-aws-bucket-migration::pulumi:providers:aws::provider::2205ad1c-2cf0-4b64-9bd2-3fa6b2c1083b]
      + region: <null>

Adding the PreCheckCallback leads to a diff like this (better)

      }
    ~ aws:s3/bucketOwnershipControls:BucketOwnershipControls: (update)
        [id=loggingbucket-0c177aa]
        [urn=urn:pulumi:chall::test-aws-bucket-migration::aws:s3/bucketOwnershipControls:BucketOwnershipControls::exampleBucketOwnershipControls]
        [provider=urn:pulumi:chall::test-aws-bucket-migration::pulumi:providers:aws::provider::2205ad1c-2cf0-4b64-9bd2-3fa6b2c1083b]
      + region: "us-east-2"
  1. The second issue is that we shouldn't see the diff at all since Terraform doesn't show a diff in this case. Terraform also sets the region in a interceptor that runs after Read which means when diff runs it has the same value in the old and new inputs. So if we run up --refresh then Read runs and that post-read interceptor runs and we don't see the diff.

@@ -595,6 +595,9 @@ func TestRegress2868(t *testing.T) {
test := getJSBaseOptions(t).
With(integration.ProgramTestOptions{
Dir: filepath.Join(getCwd(t), "regress-2868"),
// TODO[pulumi/pulumi-aws#5521] `region` causes permanent diff without refresh
PreviewCommandlineFlags: []string{"--refresh"},
UpdateCommandlineFlags: []string{"--refresh"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question as above: does the PreCheckCallback not apply here?

@corymhall corymhall merged commit 919b7bf into 7.0.0-alpha May 16, 2025
24 checks passed
@corymhall corymhall deleted the corymhall/v6-beta1-upgrade branch May 16, 2025 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants