Skip to content

resource/aws_s3_bucket: fix CreateBucket LocationConstraint for S3-compatible endpoints - #49890

Open
kayrus wants to merge 1 commit into
hashicorp:mainfrom
kayrus:fix-custom-constraints
Open

resource/aws_s3_bucket: fix CreateBucket LocationConstraint for S3-compatible endpoints#49890
kayrus wants to merge 1 commit into
hashicorp:mainfrom
kayrus:fix-custom-constraints

Conversation

@kayrus

@kayrus kayrus commented Sep 8, 2026

Copy link
Copy Markdown

Rollback Plan

If a change needs to be reverted, we will publish an updated version of the library.

Changes to Security Controls

This change only affects the LocationConstraint value sent in the S3 CreateBucket request body when a custom S3 endpoint is configured with a non-standard region string. It does not alter access controls, encryption, or logging. Request signing behavior is unchanged.

Description

When a custom S3 endpoint is configured (endpoints { s3 = ... }) together with a non-standard region string — for example the region names used by Ceph RGW or MinIO such as :region-ha-premium or ceph-objectstore-region1:region-ha-premium — the provider substitutes the configured region with a compliant dummy value (us-east-1) so the AWS SDK for Go v2's strict region validation passes, while preserving the original region for request signing.

The problem: resourceBucketCreate derived the CreateBucket LocationConstraint from the (now substituted) effective region. Since that value is us-east-1 — where AWS requires that no LocationConstraint be sent — the constraint was omitted and an empty <CreateBucketConfiguration></CreateBucketConfiguration> was sent. S3-compatible backends reject this with 400 InvalidArgument, so bucket creation fails. Provider v5.x sent the configured region verbatim and worked.

This PR prefers the preserved original region for the LocationConstraint so the create request again matches the configured region:

  • internal/conns/awsclient.go: adds an exported S3OriginalRegion(ctx) accessor for the region preserved during the S3-compatible substitution.
  • internal/service/s3/bucket.go: extracts the LocationConstraint derivation into a pure, exported helper BucketLocationConstraint(region, originalRegion) and uses it in resourceBucketCreate. It prefers the original region when set, and continues to send no constraint for us-east-1.

Native AWS is unaffected: S3OriginalRegion is only populated for a non-AWS region behind a custom S3 endpoint, so on AWS the effective region is used and us-east-1 continues to send no LocationConstraint. Any non-standard region string is passed through verbatim; the equivalence of the prefixed and bare Ceph forms is enforced server-side by Ceph, not by the provider.

Verified against a Ceph RGW gateway: terraform apply now sends <LocationConstraint>:region-ha-premium</LocationConstraint> and the bucket is created (HTTP 200), matching provider v5.x behavior.

CreateBucket request body

Config used (Ceph RGW endpoint, non-standard region :region-ha-premium):

provider "aws" {
  region            = ":region-ha-premium"
  s3_use_path_style = true
  endpoints {
    s3 = "https://ceph.endpoint"
  }
  # ... skip_* flags + static credentials
}

resource "aws_s3_bucket" "example" {
  bucket = "test-aws-bucket"
}

Before (v6.63.0, broken) — the substituted us-east-1 region caused the LocationConstraint to be dropped, sending an empty configuration that Ceph rejects with 400 InvalidArgument:

PUT /test-aws-bucket HTTP/1.1

<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></CreateBucketConfiguration>
HTTP/1.1 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code>...</Error>

After (this PR / matches v5.100.0) — the preserved original region is sent as the LocationConstraint and Ceph accepts it:

PUT /test-aws-bucket HTTP/1.1

<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>:region-ha-premium</LocationConstraint></CreateBucketConfiguration>
HTTP/1.1 200 OK

Relations

Closes #46517

References

Output from Acceptance Testing

% go test ./internal/service/s3/ -run 'TestBucketLocationConstraint' -v

=== RUN   TestBucketLocationConstraint
=== PAUSE TestBucketLocationConstraint
=== CONT  TestBucketLocationConstraint
=== RUN   TestBucketLocationConstraint/native_aws_us-east-1_sends_no_constraint
=== RUN   TestBucketLocationConstraint/native_aws_non_us-east-1_uses_region
=== RUN   TestBucketLocationConstraint/s3-compatible_substituted_region_uses_original_bare_constraint
=== RUN   TestBucketLocationConstraint/s3-compatible_substituted_region_uses_original_prefixed_constraint
=== RUN   TestBucketLocationConstraint/s3-compatible_original_us-east-1_sends_no_constraint
--- PASS: TestBucketLocationConstraint (0.00s)
    --- PASS: TestBucketLocationConstraint/native_aws_us-east-1_sends_no_constraint (0.00s)
    --- PASS: TestBucketLocationConstraint/native_aws_non_us-east-1_uses_region (0.00s)
    --- PASS: TestBucketLocationConstraint/s3-compatible_substituted_region_uses_original_bare_constraint (0.00s)
    --- PASS: TestBucketLocationConstraint/s3-compatible_substituted_region_uses_original_prefixed_constraint (0.00s)
    --- PASS: TestBucketLocationConstraint/s3-compatible_original_us-east-1_sends_no_constraint (0.00s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/s3 0.208s
% make testacc TESTS=TestAccS3Bucket_basic PKG=s3

# Requires an S3-compatible endpoint (e.g. Ceph RGW) with a non-standard region
# string configured to exercise the S3-compatible LocationConstraint path.
...

@kayrus
kayrus requested a review from a team as a code owner September 8, 2026 12:53
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Community Guidelines

This comment is added to every new Pull Request to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀

Voting for Prioritization

  • Please vote on this Pull Request by adding a 👍 reaction to the original post to help the community and maintainers prioritize it.
  • Please see our prioritization guide for additional information on how the maintainers handle prioritization.
  • Please do not leave +1 or other comments that do not add relevant new information or questions; they generate extra noise for others following the Pull Request and do not help prioritize the request.

Pull Request Authors

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions Bot added needs-triage Waiting for first response or review from a maintainer. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. service/s3 Issues and PRs that pertain to the s3 service. client-connections Pertains to the AWS Client and service connections. size/M Managed by automation to categorize the size of a PR. labels Sep 8, 2026
@kayrus
kayrus force-pushed the fix-custom-constraints branch from 52a94cf to 020b772 Compare September 8, 2026 13:19
…mpatible endpoints

When a custom S3 endpoint is configured with a non-standard region string
(e.g. Ceph RGW or MinIO region names such as ":region-ha-premium"), the
provider substitutes the configured region with a compliant dummy value
(us-east-1) so the AWS SDK v2 region validation passes, preserving the
original region for request signing.

The CreateBucket request derived its LocationConstraint from the (now
substituted) effective region. Because that value equals us-east-1, the
LocationConstraint was omitted and an empty CreateBucketConfiguration was
sent, which S3-compatible backends reject with 400 InvalidArgument.

Prefer the preserved original region for the LocationConstraint so the
create request matches the configured region, restoring the behavior of
provider v5.x. Native AWS is unaffected: the original region is only set
for non-AWS regions behind a custom S3 endpoint, so us-east-1 continues to
send no LocationConstraint.

Closes hashicorp#46517
@kayrus
kayrus force-pushed the fix-custom-constraints branch from 020b772 to 535df5e Compare September 9, 2026 10:09
@kayrus

kayrus commented Sep 9, 2026

Copy link
Copy Markdown
Author

@gdavison @jar-b @ewbankkit @YakDriver could you please review this PR?

@kayrus

kayrus commented Sep 10, 2026

Copy link
Copy Markdown
Author

cc @justinretzolk I'd appreciate it if you could also have a look at this PR when convenient

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client-connections Pertains to the AWS Client and service connections. needs-triage Waiting for first response or review from a maintainer. service/s3 Issues and PRs that pertain to the s3 service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Provider Panic/Crash with Custom S3 Region Strings - CEPH

1 participant