Use terraform's native s3 state locking#1476
Merged
Merged
Conversation
cadmiumcat
force-pushed
the
native-s3-object-locking
branch
3 times, most recently
from
d84d448 to
3de1625
Compare
cadmiumcat
force-pushed
the
native-s3-object-locking
branch
8 times, most recently
from
b70091c to
79ab7cc
Compare
cadmiumcat
changed the title
cadmiumcat
changed the title
cadmiumcat
force-pushed
the
native-s3-object-locking
branch
from
79ab7cc to
0b6460f
Compare
cadmiumcat
added a commit
that referenced
this pull request
Apr 11, 2025
Since we are using terraform's native s3 state locking (#1476) we no longer need have dynamodb tables or the permissions associated with it
sarahseewhy
reviewed
Apr 14, 2025
cadmiumcat
added a commit
to govuk-forms/forms-admin
that referenced
this pull request
Apr 14, 2025
cadmiumcat
force-pushed
the
native-s3-object-locking
branch
from
0b6460f to
9c08bf5
Compare
cadmiumcat
added a commit
that referenced
this pull request
Apr 14, 2025
Since we are using terraform's native s3 state locking (#1476) we no longer need have dynamodb tables or the permissions associated with it
cadmiumcat
added a commit
to govuk-forms/forms-admin
that referenced
this pull request
Apr 15, 2025
sarahseewhy
approved these changes
Apr 15, 2025
Instead of using a dynamodb table we want to use terraform's new native locking for s3 by enabling "use_lockfile". We had previously introduced a dynamodb table to help the `init.sh` script to solve a chicken and egg problem (ff3c65c) but we should be able to remove this now.
We'll want to delete the dynamodb tables after we have enabled native state locking. For now we are renaming them with a default value so that they do not depend on values we have removed. This PR makes sure that the policies are still allowed to modify the dynamodb tables
We do this by allowing them to delete the `.tflock` object. They should not be allowed to delete state files
We need the support role to be able to `terraform apply`, therefore it needs to be able to get/release the lock on state files. The role also needs to be able to put objects in and get objects from the state file buckets, but we already gave it those permissions as part of allowing it to manage_maintenance_page.
cadmiumcat
force-pushed
the
native-s3-object-locking
branch
from
8059c3b to
10e8f86
Compare
cadmiumcat
added a commit
that referenced
this pull request
Apr 16, 2025
Since we are using terraform's native s3 state locking (#1476) we no longer need have dynamodb tables or the permissions associated with it
cadmiumcat
added a commit
that referenced
this pull request
Apr 17, 2025
Since we are using terraform's native s3 state locking (#1476) we no longer need have dynamodb tables or the permissions associated with it
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this pull request solve?
In this PR we have enabled state locking for S3 backend so that we can stop using Dynamodb based locking, as it will be deprecated by terraform in future releases.
We have added permissions to ECS, the github runner and the engineer support role so that they can release state file locks.
We also introduced a default temporary name for existing dynamodb_table to help us remove it in future PRs.
Things that will happen in future PRs:
Trello card:
Things to consider when reviewing
I tested this by manually applying the changes to the ECS permissions in dev, then allowing the pipelines to apply the
formsdeployment to dev. I could see for each deployment, theterraform applystep creates atflockfile in the same bucket as thetfstatefile. When the command finishes, terraform deletes thetflockfile, which releases the lock.Reminders
If you've made changes to the deployer role (files in
modules/deployer-access):make <environment> forms/account applyon the relevant environments (dev,staging,user-research, and/orprod)apply-forms-terraform-<environment>pipelines have run successfully